That's why the former New England offensive coordinator and current Houston Texans coach knows what his team is up against when it visits for Saturday's night's AFC divisional playoff game.
"It's a big challenge. It's a team that has done it year in and year out," O'Brien said earlier this week. "Last week was only this organization's third playoff win. We've got a big challenge ahead of us.
"We've got a bunch of good guys in our locker room that are looking forward to the challenge. We know that the Patriots are a great football team. We feel like if we go up there and we practice well -- we have a good week of practice -- we go up there and show up and play hard for 60 minutes, you know, we'll see what happens."
History, and the oddsmakers, are both against the Texans, a 9-7 division champion that disposed of the Oakland Raiders in last weekend's wild card game. Houston has never won at Foxborough, has beaten the Patriots just once, lost 27-0 in New England -- with Tom Brady out, no less -- in September and has lost the last two games to the Patriots by a combined score of 54-6.
The Texans come into this one as a 15 1/2-point underdog.
They're also a warm weather team playing in the cold night air.
The Texans do bring the league's No. 1 defense into this game, while the Patriots have allowed the fewest points.
Back in Week 3, Jacoby Brissett was the New England quarterback and J.J. Watt played his final game of the season before undergoing back surgery. The Texans fumbled away two kickoffs and saw Brissett run for a 27-yard touchdown.
Now, Brady will run the potent Patriots' offense, which should be bolstered by the return of wideout Danny Amendola, who missed four games with a sprain ankle. He was back at full participation on Wednesday's practice report, while Malcolm Mitchell, out last week with a knee injury, was limited.
LeGarrette Blount, who has had a strong year running the ball, missed practice Wednesday and Thursday due to illness but was removed from the injury report on Friday.
The Patriots come in 14-2 with seven straight wins and have gone 11-1 since Brady returned from his Deflategate suspension. They have spent the week doing what they always do -- not saying anything even close to bad about the opposition.
"I think that they're playing really well in all three phases," New England's Rob Ninkovich said. "I think that their running game is very strong, so we have a challenge. It starts today preparing for that challenge."
Added Malcolm Butler: "Of course we won the (first) game, but teams are not peaking early in the season, so you most definitely say they got better, we got better, so time will tell."
The Texans had eight players listed with limited participation Wednesday, six of them of defense -- including Quintin Demps (hamstring, game-time decision), Jadeveon Clowney (elbow, wrist) and Brian Cushing (ankle).
Brady finished the regular season with 28 touchdown passes and just two interceptions as the Patriots set the record for fewest picks thrown (two) in a season.
The New England defense, suspect when it blanked the Texans in Week 3, has gotten better and better and even traded away Jamie Collins, who had a big game against Houston.
"It's tough," O'Brien said. "They're obviously a team that lines up in different fronts. They challenge you with -- not only their players, which their players are very, very good. You got (Alan) Branch and (Dont'a) Hightower and (Rob) Ninkovich and (Jabaal) Sheard and Malcolm Brown. You have a bunch of really tough, hard-nosed players that are very, very well-coached so their scheme presents a problem too.
"We're going to have to have a great week of practice. We're going to have to be very focused on every single play, limit mistakes, really communicate really well because they're a tough team, run or pass, to go against."
The Patriots have outscored their last three opponents, 92-20, over the last three games, with Brady and Co. always in high gear.
"I think it's more about execution," O'Brien said of the task at hand. "We're not going to show him anything that he hasn't already seen. He's been doing it for years, 17 years or whatever it is.
"I mean, that's part of what makes him who he is. He's seen it all. He's very well-prepared. He's played against a lot of these players. He's played against this scheme. We just need to kind of do what we do and execute and play every play. Be very focused on every single play, I think that's big for us."