Brett Harrelson was born in Midland, Texas, the son of Diane Lou (née Oswald) and Charles Voyde Harrelson, who divorced in 1964; he has two brothers, Jordan and the screen actor Woodrow "Woody" Harrelson. Harrelson's father, who was a contract killer, was arrested for the killing of Federal Judge John H. Wood, Jr. by rifle fire in 1979 in San Antonio. His father was convicted and eventually died during his life sentence in United States Penitentiary Administrative Maximum Facility.
In 1973, Harrelson moved with his mother to her native city, Lebanon, Ohio, where he was raised. Harrelson attended Lebanon High School but dropped out at 17 to join the Army, spending two years in Germany. He returned to Lebanon and worked as a legal clerk. Then at 22 he followed his brother Woody to California. "I came to L.A. to starify," he says. But finding nothing like Woody's success, he gave up acting for a while to become a motorcycle racer, rising to No. 8 in the 1992 pro national rankings. "After seeing a few people killed," says Brett, he became Woody's assistant and began to pursue his acting career again.