Officials said the fire happened on Wednesday evening at an apartment complex on Burress Street near Airline Drive.
HFD said it took about 80 firefighters from 15 different stations to extinguish the fire.
Firefighters told Eyewitness News that many residents living at the complex have disabilities, so they came in knowing they'd have to make a lot of rescues.
One person died and three people were taken to the hospital, according to the Houston Police Department. Family identified the victim as 74-year-old Rodney Gardner.
SEE ALSO: North Houston apartment residents face constant roof issues with no fix in sight: 'Here we go again'
"As of right now, our HFD arson investigators continue to figure out the cause and origin of the fire, what started it, where it started, things like that. We do believe that the fire started in the unit of the man who passed away," Brent Taylor, with the Houston Fire Department, said.
Investigators said one of the people hospitalized jumped out of their unit and suffered a broken bone.
"These guys went in and helped usher out many. To have what we have, our hearts go out to the person who died. The firefighters who came here saved many. This could have been worse than what we had," HFD District Chief R. Christ said.
HFD said the cause of the fire is unknown at this point, and about 20 residents were displaced.
"It was just a lot of fire coming from the window, a lot of smoke. It smelled electrical. You could smell the structure burning, but you could smell wires burning," Velma Salazar, a neighbor, said.
While the cause of the fire is still under investigation, residents said issues at the complex are nothing new.
Last May, ABC13 reported on the lack of electricity, which trapped some residents after a storm. Most recently, in February, the issue was the rocky roofs at the apartment.
"They had some issues in May of last year, right after the derecho, that we pointed out, and we gave them a period to cure the issue. We worked with them over the next several months, and as of August of last year, those issues had been remedied," Taylor said.
SEE ALSO: Mayor accuses management of 'abandoning' affordable housing residents without power after storm
But for people who spoke with ABC13, they say it isn't enough.
"It's going downhill. A lot of the people I knew that lived here before, they had to move out because they just don't fix the apartments," a woman said.
A woman who's lived at the complex for 21 years is sounding the alarm but does not want to be identified for fear of retaliation.
"It's hard. It's hard to live like this," the woman said. "We need help. Tell your committee we need help."
She says it's hard to get things fixed, from dishwashers to windows and ceilings.
Mario Castillo, the councilmember for this district, is pleading with the community to speak out and spark change.
"We need to know if this is going on so that we can be proactive in trying to bring these places up to code and get bad actors held accountable," Councilmember Castillo said.
The management company provided the following statement:
"We are deeply saddened by the loss of one of our residents in last night's fire, which apparently was started inadvertently by the resident or an invited guest. We are also greatly concerned about our three residents that suffered injuries due to the fire and the twenty households that lost their apartments. All of our residents are seniors or persons with disabilities who live independently on limited incomes. We are working with the Red Cross and other agencies to support everyone who was impacted by this tragedy."
However, when Eyewitness News asked about residents' concerns, they said they do not have a comment on that right now.