"Bite Your Tongue" was released as a single on February 2, 1999 and is found on Duncan Sheik's second studio album, Humming. The song was called "a driving hard-pop number" by Rolling Stone's Neva Chonin, "self-deprecating" by Allmusic's Roxanne Blanford and Elysa Gardner from the Los Angeles Times said: "The single “Bite Your Tongue” rocks harder and more buoyantly than his previous hits.". The song would also appear as a bonus track on the 2004 Daylight (Limited Tour Edition) CD and on the 2006 double disk album Brighter/Later: A Duncan Sheik Anthology, released by Rhino Records (Also released in 2007 as Greatest Hits – Brighter: A Duncan Sheik Collection, a single CD version.)