The arrest shocked many of his neighbors.
Neighbor Alonzo Citizen said, "I was just thinking golly, I wouldn't never think nothing like that. I didn't know nothing like that."
Investigators say the accuser was a friend of Ballard's children, but say Ballard's children themselves do not appear to have been victims. Ballard, known as Leeroy to the neighborhood, was arrested and charged with sexual assault on a child.
Neighbor Nita Davis said, "I don't know really what type of person he was, but I know he wasn't like he would have done something like that."
But investigators worry that there may be more victims, because Ballard had worked as a Little League umpire for the last year or so, and not just in the Daisetta area.
Capt. Rex Evans with the Liberty County Sheriff's Office said, "We just believe he also went into some of the adjoining counties for certain games, certainly not widespread."
In fact, investigators were talking to another 15-year-old boy to see if he is a victim, and they'd like to hear from anyone else who suspects Ballard may have assaulted other kids.
Ballard, 48 is cooperating with authorities. He is still in jail with a $100,000 bond. Anyone who thinks their child may have been a victim of Ballard or who has further information about this case is asked to call the Liberty County Sheriff's Office at 936-336-4500.