A neighbor at the Maison De Ville apartments in the Greenspoint area told police she found a 4-year-old child wandering barefoot outside in the rain on Wednesday night.
When she went to the child's apartment, she said she found a 2-year-old child.
Jeyson Martinez Zacapa, 26, was arrested that night.
According to court documents, Zacapa told police he had gone to pick up his wife from work while the children were sleeping.
But the same woman who called police on Wednesday told officers the children had also been left alone two weeks prior for as long as seven hours.
"It's ridiculous. You should be responsible for your own kids," Michelle Stewart, who learned about the case Friday, said.
"He is a deadbeat parent. Like, who leaves their babies? Those are babies, like little babies," another woman said.
In court documents, prosecutors note the area's drug and gang activity.
According to the ABC13 Neighborhood Safety Tracker, the Greenspoint area has had more than triple the citywide homicide rate for the last 12 months and almost twice the citywide assault rate.
Zacapa, who is from Honduras, is being held on a $1,000 bond for the abandonment charge. But U.S. Immigration and Customs Enforcement has also placed a hold on him.
For updates on this story, follow Luke Jones on Facebook, Twitter and Instagram.