The convenience store is surrounded by bars and restaurants, and state prosecutors believe he was using this as an opportunity to record the women.
Court documents show that constables were investigating Holt after a woman discovered the camera in the bathroom.
The constable's office said the woman took the camera and ran from the store when she discovered the camera.
During court proceedings, the state mentioned that Holt became hostile when he found out the woman discovered the recording device.
They say she later turned the camera over to the constable's office who worked with their High Tech Crimes Unit.
Investigators discovered 126 video files of unsuspecting victims using the restroom at the business location.
Constables then arrested Holt, who is currently being held on a combined $87,500 bond for the invasive recording and felon in possession of a firearm.
Holt was also found to be in possession of two firearms.
During court proceedings, Holt's appointed defense attorney said Holt claims to own a convenience store.
For news updates, follow Jiovanni Jieggi on Facebook, Twitter and Instagram.