Tobolowsky was born in Dallas, Texas, into a Jewish family from Russia and Poland. [failed verification] He grew up creating imaginative games with his brother, and his story The Dangerous Animals Club paints a picture of how unsupervised children in the 20th century could amuse themselves. He showed talent at baseball, but a serious childhood illness stopped his career as an athlete before it got started. He graduated from Justin F. Kimball High School and Southern Methodist University. He received a master's degree from the University of Illinois in 1975. He is a cousin of former Dallas attorney Ira Tobolowsky, who was murdered in his home in 2016—a mystery that remains unsolved. Another cousin (and Ira's brother) is the sculptor George Tobolowsky. Stephen Tobolowsky also played in a band called A Cast of Thousands, which had two songs, "Red, White and Blue" and "I Heard a Voice Last Night", on a compilation of local Dallas bands called A New Hi. Both of these songs featured Steve Vaughan on guitar, who would be later known as Stevie Ray Vaughan.