According to the Federal Aviation Administration, the 1981 twin-engine Cessna 414A took off from Fullerton Municipal Airport around 1:35 p.m. The facility is about 15 miles west of the crash site.
Authorities said the aircraft climbed about 700 feet before making a sudden decline down.
Shortly after 2 p.m., a spokesperson for the Orange County Fire authority said there were at least two active structure fires at the scene on Canyon Drive near Crestknoll Drive. They said they had received multiple phone calls about a small plane crashing into a home.
During an evening press conference, authorities said the male pilot died in the crash, as well as four people inside the home - two men and two women.
Because of the extensive damage to the home, authorities said the Orange County coroner made entry around 9 p.m. to work on identifying the victims.
One neighbor, Tony Tomminelli, said he heard an explosion from up above.
"I walked outside and I look up and I just see debris flying everywhere. Part of a plane, which I guess eventually was the motor, flew down into the garage like a torpedo - just hit the garage and blew up," he said.
He ended up sliding down the hillside to try and help.
Some 70 firefighters, one of whom suffered a minor injury, responded to the location. Authorities said two other people suffered moderate injuries and were hospitalized.
A witness' video shows a two-story home engulfed by a massive inferno as horrified neighbors look on and two explosions are heard.
In the footage, a piece of the plane's wreckage is burning in the street. A man is seen dousing it with water from a garden hose. Another man then uses a fire extinguisher to put out the flames.
Other debris from the aircraft was scattered on nearby rooftops. NTSB officials said the debris field was about 4 blocks long.
Anaheim firefighters assisted in the response, along with deputies from the Orange County Sheriff's Department.
Other neighbors said they had parts of debris all over their homes.
Pokey Sanchez, an OCFA assistant chief, said firefighters planned to sift through the badly burned two-story house in case there are additional victims.
Glenknoll Elementary School, which was being used as a command post for authorities, will be closed Monday as the investigation continues.
The Associated Press contributed to this report.