The building was fully ablaze when the first fire truck arrived, said New Orleans Fire Department spokesman Greg Davis. Firefighters could not tell the ages or genders of the bodies pulled from the rubble. The fire also killed two dogs.
Two survivors told firefighters that at least some of the people inside were unconscious when the building went up in flames. Davis said they might have been knocked out by carbon monoxide.
"When you burn something in a closed area, you're going to build up carbon monoxide," he said.
The average low for New Orleans in late December is around 44 degrees, but freezing temperatures aren't unheard of, said National Weather Service meteorologist Tim Destri.
The survivors were a man who escaped during the fire and a woman who had left the building before it was engulfed, Red Cross worker Tom Butler said. He said both were put up at a hotel.
The man told Butler that he heard someone screaming and trying to get out. The survivor said he tried to run back into the building, but there was too much smoke. Butler said the man didn't want to be identified.
The wood and corrugated metal building burned to the foundation. It was among several ramshackle metal, brick and wooden buildings along railroad tracks where graffiti-scrawled freight cars stood.
Many of the houses nearby appeared abandoned, but at least a few were occupied.
Ricky Gordon, who lives in the working-class neighborhood in the Upper 9th Ward, said he does volunteer work with the homeless, and they often take refuge in the buildings when it gets cold.
"I took one guy in last night myself," he said.