Brown, who was injured on Sunday, will not travel with the team to Baltimore. A first-round pick in 2008, Brown has started 78 games in his career.
Ryan Harris, who appeared in each game last season with two starts, will start in his place for the 2-0 Texans.
The Associated Press contributed to this report
