Surveillance video from La Porte First Baptist Church on South Broadway captured a man stealing numerous items out of the building on July 7.
According to detectives, Noel Christian Scholes admitted to the crime and took them to some of the stolen property.
Scholes sustained an injury to his chin after fleeing from the scene and crashing his bicycle, police said.
He is being held at the La Porte Jail on a felony charge of burglary of a building.