If you're viewing on our ABC13 news app, tap on the photo above to see more images.
The Bengals converted a fourth down play to keep a drive alive, but four plays later, former Texan Randy Bullock's 43-yard attempt sailed wide right to secure Houston's victory.
Savage helped the Texans (9-6) rally for a win over the Jaguars last week after taking over when Brock Osweiler was benched after throwing interceptions on back-to-back possessions in the second quarter.
The Texans knew they were in the postseason with a win after the Titans fell to the Jaguars earlier Saturday. But it didn't look good for them early with Savage completing just two passes for 13 yards and them trailing 3-0 at halftime.
VIDEO: Greg Bailey's keys to a Texans success
Savage was able to move the offense much better in the second half, but Houston still struggled in the red zone.
The Texans had a first down at the 6 early in the fourth quarter before the offense stalled and they settled for a 22-yard field goal that put them up 6-3. Quintin Demps got his career-high fifth interception when he picked off Andy Dalton's pass two plays later, but Houston couldn't move the ball and had to punt it away.
it 86-yards for a touchdown that made it 10-6 with about 11 minutes left.
A 24-yard touchdown run by Blue put the Texans back on top with about 9 minutes left, but the extra point was blocked to leave Houston up 12-10. That score was set up by a 21-yard reception by DeAndre Hopkins.
Dalton finished with 268 yards passing with a touchdown and an interception.
Lamar Miller missed his first game this season because of an ankle injury and Blue had 73 yards rushing in his place.
A.J. Green, the Cincinnati star receiver who said earlier this week that he would play after missing the last four games with a hamstring injury, was inactive amid reports that he was unhappy with the decision to sit him.
After a terrible first half on offense, Houston finally got a little something going after halftime and tied it at 3-3 with a 25-yard field goal early in the third quarter.
The first half was an offensive struggle for both teams with a combined 12 punts. The Texans were a bit worse than the Bengals though and had just two first downs and 34 total yards with minus-5 yards passing because of three sacks.
The Bengals had a 3-0 lead at halftime after a 43-yard field goal by Randy Bullock at the end of the second quarter.
The Texans didn't cross midfield their first drive of the third quarter when Will Fuller caught a 14-yard pass that got them to the Cincinnati 47.
The Associated Press contributed to this report.