Paterno, the winningest coach in major college football, was ousted at the end of day that began with his announcement to retire at end of the season, his 46th.
It was not to be.
"The university is much larger than its athletic teams," board vice chair John Surma said during a packed news conference.
Paterno and Spanier were informed by telephone of the unanimous decisions to remove them.
"We were unable to find a way to do that in person without causing further distraction," Surma said.
Defensive coordinator Tom Bradley will serve as interim coach while Rodney Erickson will be the interim school president.
"The Penn State board of trustees tonight decided it is in the best interest of the university to have a change in leadership to deal with the difficult issues that we are facing," Surma said.
"The past several days have been absolutely terrible for the entire Penn State community. But the outrage that we feel is nothing compared to the physical and psychological suffering that allegedly took place."
Asked what Paterno did wrong, Surma said: "I can't characterize that. We thought because of the difficulties that have engulfed our university, it was necessary to make changes."
Speaking outside his home, Paterno said: "Right now, I'm not the football coach. And I've got to get used to that. After 61 years, I've got to get used to it. I appreciate it. Let me think it through."
His wife, Sue, was teary-eyed as she blew kisses to about 100 students on the lawn. "You're all so sweet. And I guess we have to go beat Nebraska without being there. We love you all. Go Penn State," she said.
Paterno said in a statement earlier Wednesday that he was "absolutely devastated" by the abuse case, in which his former assistant and onetime heir apparent, Jerry Sandusky, has been charged with molesting eight boys over 15 years, with some of the alleged assaults taking place at the Penn State football complex.
"This is a tragedy," Paterno said. "It is one of the great sorrows of my life. With the benefit of hindsight, I wish I had done more."
Paterno has come under harsh criticism -- including from within the community known as Happy Valley -- for not taking more action in 2002 after then-graduate assistant and current assistant coach Mike McQueary came to him and reported seeing Sandusky in the Penn State showers with a young boy. Paterno notified the athletic director, Tim Curley, and a vice president, Gary Schultz.
Paterno is not a target of the criminal investigation, although Curley and Schultz have been charged with failing to report the incident to the authorities.
After the firings, thousands of students descended on the administration building, shouting, "We want Joe back!" then headed to downtown to Beaver Avenue. Almost all the students were decked out in Penn State gear.
The firings came three days before Penn State hosts Nebraska in its final home game of the season, a day usually set aside to honor seniors on the team.
The ouster of the man affectionately known as "JoePa" brings to an end one of the most storied coaching careers -- not just in college football but in all of sports. Paterno has 409 victories -- a record for major college football -- won two national titles and guided five teams to unbeaten, untied seasons. He reached 300 wins faster than any other coach.
Penn State is 8-1 this year, with its only loss to powerhouse Alabama. The Nittany Lions are No. 12 in The Associated Press poll.
After 19th-ranked Nebraska, Penn State plays at Ohio State and at No. 16 Wisconsin, both Big Ten rivals. It has a chance to play in the Big Ten championship game Dec. 3 in Indianapolis, with a Rose Bowl bid on the line.
The board had already said it would appoint a committee to investigate the "circumstances" that resulted in the indictment of Sandusky, and of Curley and Schultz.
In Washington, the U.S. Department of Education said it has launched an investigation into whether Penn State failed to report incidents of sexual abuse on campus, as required by federal law.
Sandusky, who retired from Penn State in June 1999, maintained his innocence through his lawyer. Curley has taken a leave of absence and Schultz has decided to step down. They also say they are innocent.
The committee will be appointed Friday at the board's regular meeting, which Gov. Tom Corbett said he plans to attend, and will examine "what failures occurred and who is responsible and what measures are necessary to ensure" similar mistakes aren't made in the future.
Surma said McQueary would retain his job for now.
Sandusky founded The Second Mile charity in 1977, working with at-risk youths. It now raises and spends several million dollars each year for its programs. Paterno is listed on The Second Mile's website as a member of its honorary board of directors, a group that includes business executives, golfing great Arnold Palmer and several NFL Hall of Famers and coaches, including retired Pittsburgh Steelers stars Jack Ham and Franco Harris.