Brenner, who had been fighting cancer, died peacefully at his home in New York City with his family at his side, according to Jeff Abraham, his friend and publicist.
"David Brenner was a huge star when I met him and he took me under his wing. To me, historically, he was the godfather of hip, observational comedy," comedian Richard Lewis said in a statement. "He mentored me from day one. ... His passing leaves a hole in my life that can never be replaced."
The tall, thin and always sharply dressed Brenner became one of the most frequent visitors to Johnny Carson's "Tonight" in the 1970s and '80s.
His 150-plus appearances as guest and substitute host turned the former documentary filmmaker into a hot comedian, one who was ubiquitous on other talk shows and game shows.
He also briefly hosted his own syndicated talk show in 1987 and starred in four HBO specials.
Brenner moved with the times, trading routines about the humor of everyday life for jokes about social and political issues, and appearing on MSNBC and Fox News Channel cable programs.
Although his career faltered, he worked steadily through 2013 doing standup. A four-day gig last December included a New Year's Eve show at a Pennsylvania casino-resort in which he showcased young comedians.
Brenner, who was raised in working-class south Philadelphia and graduated with honors from Temple University, was "always there helping a bright young comedian, whether it be Richard Lewis, Freddie Prinze or Jimmie Walker, and he was still doing it until the very end," Abraham said.
In a statement, Walker called Brenner "a true comic genius" who was "my mentor and taught me about life and comedy."
Although Brenner took a brief stabs at TV fame, with the 1976 sitcom "Snip" and the talk show "Nightlife" he hosted in 1987, he didn't achieve the success of Seinfeld's self-titled NBC sitcom or Reiser's "Mad About You," and he saw Jay Leno follow Carson as "Tonight" host.
Brenner's take on his career path, as he described it in a 2000 interview with The Associated Press, was that he put family before stardom.
He said a long custody battle with a girlfriend over their son, Cole, forced him to curtail his TV appearances and visibility beginning in the mid-1980s, when Brenner lived in Aspen, Colo.
"In a nutshell, I couldn't work more than 50 nights a year (out of town) or I'd be an absentee father," he said. "That was when they were giving out the talk shows, the sitcoms."
He was asked if he regretted his decision.
"I didn't even make a decision. I didn't even think about it. How could you not do it? I don't mean to sound noble," Brenner said. "Besides, I come from the slums of Philadelphia and everything in my life is profit. My downside is what most people would strive a lifetime to get to."
Decades ago, he had burned out on filmmaking - "You don't change the world by doing documentaries," he told "CBS Sunday Morning" in 2013 - and decided to give comedy a try. He was on the verge of quitting when his effort to impress talent bookers at "Tonight" worked.
His career soared after his first appearance in January 1971. He went from being nearly broke to overwhelmed by a then-hefty $10,000 in job offers the day after he was on the show.
"I never thought this was going to turn my life upside down and give me my whole future," he told "Sunday Morning."
He also recalled how hard Carson made him work on "Tonight," asking Brenner to do a monologue each time he appeared. Other veteran comics headed straight for the couch to banter with the host.
Carson's explanation was "I like to sit back, smoke a cigarette and laugh for six minutes," Brenner recalled.
In a 1995 interview with the AP, Brenner imagined a different path with "Tonight."
"I really believe that had ... Johnny Carson retired in the early '80s, then I would be sitting behind that desk," he said. "I don't think there's any doubt."
Brenner wrote five books, including the post-9/11 "I Think There's a Terrorist in My Soup," published in 2003. His last HBO special, "David Brenner: Back with a Vengeance," debuted live in 2000.
In a statement, his family said he left a last laugh: A final request that $100 in small bills be placed in his left sock "just in case tipping is recommended where I'm going."
Besides son Cole, Brenner is survived by his wife, Ruth, sons Wyatt and Slade and a grandson, Wesley. Funeral plans were not immediately announced.
Take ABC13 with you!
Download our free apps for iPhone, iPad and Android devices