
Stoltz is a vegetarian and member of the Actors Studio. He lived with actress Ally Sheedy (whom he met in college) sometime before 1983, then with actress Jennifer Jason Leigh from 1985–1989 and with Bridget Fonda from 1990–1998. He currently lives in New Mexico.
Director Cameron Crowe and Stoltz became friends on the set of Fast Times at Ridgemont High, and Crowe promised Stoltz a role, however small, in every film he makes, although since Stoltz wasn't able to appear in Almost Famous (his name does appear briefly on a billboard) he hasn't appeared in Crowe's following films.