A passerby spotted the dog while walking in Houston Botanic Garden and alerted groundskeeper Fernando Vivanco, who called police.
Vivanco said the dog was shaking as it stood on the base of a concrete support beam for a pedestrian bridge over the bayou.
SEE ALSO: Dog rescued from floodwater left by Helene devastation | VIDEO
Vivanco told Eyewitness News he'd seen alligators in that bayou before.
He said a diver swam out to the dog, but the dog jumped into the water and tried to swim away.
When the diver eventually caught up and pulled the petrified pup to dry land, Vivanco said the dog was still scared and even tried to bite his rescuer.
RELATED: New Jersey man uses professional tree-climbing skills to rescue cats stuck in tall trees
But the story ended on a high note.
Vivanco said a woman who saw the rescue adopted the chihuahua on the spot.