The Tademas told ABC13 they were trying to make their front yard look nice, when they received a violation from the HOA.
The family was issued a letter warning them of the violation one year ago.
"I feel like I have lost my ability to choose how I want to do anything over here. I feel like I lost my freedom in a sense. We have to be careful about what we do," homeowner Dorothy "Dotty" Tademas said.
The Tademas moved to Lago Mar after their Dickinson home was destroyed by Hurricane Harvey. The family thought all of their problems were behind them until they received the violation.
"We've worked hard. We're good people, we follow the rules and we do the right things," Dotty said.
The lawsuit is asking for $200 each day the yard remains in violation. The Tademas say they've paid for landscapers to come and fix their yard, and have spent money on additional legal fees.
They're still waiting to hear back from HOA.
"Heavens to Abigail. They really are after us big time. They don't want what's good for us or the people around here. They want money," homeowner Klaas Tademas said.
So far, the family is up to $20,000 in fees and the lawsuit claims it could reach up to $100,000.
"It's more (money) than we have," Klaas said. "All over some pretty flowers. Our flower beds were perfectly good. Anybody who wants flower beds in their front yard, good for them."
Eyewitness News reached out to Land Tejas, the Tademas' HOA, for comment, but it said it couldn't comment on a pending litigation.
The Tademas are hoping the lawsuit will get dropped.
"I had no idea our flower beds would be a problem," Klaas said.
OTHER HOA DISPUTES COVERED BY ABC13:
Man turns to ABC13 for help with homeowners association
Katy homeowner fights with HOA to keep large Beto sign in front yard
Humble family fighting HOA fines over pickup truck parked in their driveway
Follow TJ Parker on Facebook and Twitter.