Mallett, who played through the injury during Sunday's game, injured the muscle in his first career start Nov. 6 against the Cleveland Browns. He practiced fully all week but was on the injury report with a chest injury throughout the week leading up to the game.
The Houston Chronicle and Houston's Fox 26 earlier reported the news.
However, a team source told ESPN that Mallett's injury still is being evaluated and no determination has been made on his status.
The Texans turned to Mallett just before their bye week. Mallett replaced former starter Ryan Fitzpatrick, a 10-year veteran. The Texans traded for Mallett on Aug. 31; he had spent the first three years of his career with the New England Patriots. Mallett threw only four passes in that time and never played a meaningful snap.
The Texans acquired him for a conditional draft pick. With Monday's news, Mallett won't reach the proper playing time needed (40 percent of the team's snaps), so New England will receive a seventh-rounder.
In his first start Nov. 16, Mallett completed 20 of 30 passes and threw two touchdowns and one interception. He had the 10th-best QBR of any starting quarterback since 2001. On Sunday against the Bengals, Mallett completed only 21 of 45 passes and was likely hindered by the injury.
ESPN.com's Tania Ganguli contributed to this report.
Sunday Blitz: Bengals-Texans Recap
Jim Basquil and Ron Jaworski break down the Bengals' 22-13 win over the Texans.