Houston's quarterback doesn't have time for those opinions. Besides, even his harshest critics probably aren't as hard on him as he is on himself.
"I have higher expectations of myself than anyone could ever put on me," he said.
So what does he expect from himself this season? Simple. After the Texans were bounced from the postseason in the divisional round two years in a row, he is determined to help them go deeper in the playoffs and contend for a championship.
"Whenever you're not the last team standing at the end of the day, that's motivation to get back at what you've been doing and start working toward next season," Schaub said. "Last year is in the past, and this is the time to get better individually and as a group, and if we all get better by a little bit, we'll be right where we want to be."
Last season was Schaub's playoff debut after he missed Houston's first playoff run with a foot injury.
Houston's season ended with a 41-28 loss to New England. Schaub threw for 343 yards and two scores in that game, but also had an interception in the fourth quarter ending a drive which could have got Houston within a touchdown. That miscue had many fans and media blaming the quarterback for the loss.
Coach Gary Kubiak brushed off the notion that Schaub was the problem in that loss.
"We were there because of him," he said. "So (criticism's) part of playing quarterback and that just makes you better and pushes you every day and he's doing fine. If you can't take a little bit of that in this business you're probably not going to last very long."
He doesn't think that what happened last season puts any extra pressure on Schaub this year.
"I think pressure is the same all the time, I don't think it changes," Kubiak said. "Matt's goal is to win a championship and we've got to get a team that can get that done and he's got his job to do with the football team."
The Texans won a franchise-best 13 games last season. However, they had trouble celebrating the accomplishment because of a late-season collapse where they lost three of their final four regular-season games, costing them a bye and homefield advantage.
Schaub is hoping to help the team avoid a similar fate this time around.
"We played so well throughout the season, and then that last month we kind of were up and down, we had peaks and valleys and we didn't have that consistent level of play across the board," he said. "Then we got to the playoffs and it was the same thing. So we've just got to be a little more consistent and when we get a team down we've got to keep them down."
He started every game for Houston in 2012 and threw for 4,008 yards with 22 touchdowns and 12 interceptions.
This season he'll have a new weapon on a unit that already boasts running back Arian Foster and receiver Andre Johnson in first-round draft pick DeAndre Hopkins. The Texans hope that Hopkins can develop into a top-tier receiver they've long lacked to play opposite Johnson.
"He's such a dynamic player and the run after the catch, the ability to go up and get the ball; he's just going to bring an added dimension to our offense," Schaub said.
Schaub missed organized training activities last season while still recovering from his foot injury. He has enjoyed being on the field with his teammates this week and has been impressed with the development of some of Houston's rookies in just three days of practice.
"We've had a good start and we've just got to keep going and keep bringing the young guys along," he said. "It's a fun time of year because you see young guys growing up and you see the team coming together."
Schaub can't name one thing that he's focusing on this spring as the team begins preparations for the season, but he simply hopes to do more this year to help the team reach the next level. He won't directly say that anything less than reaching the Super Bowl would be a disappointment, but it seems that he and the rest of this team have that mindset.
"Once you get a taste of the playoff atmosphere and having some success there and not getting to your ultimate goal, you keep wanting to get back there and keep doing more," he said. "So we're definitely hungry and the fire is burning pretty good."