But people still need to eat, and that is allowing some companies to profit -- filling the void left by the empty tables at your local Applebee's (where sales there are down 3 percent).
So, what are Americans filling their stomachs with instead?
For starters, McDonald's, and maybe a frozen pizza and some Oreos washed down with Sunny Delight.
Basically, the trend is that families want easy-to-prepare foods and some comforts, and parents appear to be turning to the foods they ate in childhood
"If you are selling stuff that people need -- not discretionary items -- you have fared pretty well through the recession," said Brian Hamilton, co-founder and chief executive officer of Sageworks, which tracks data for companies. "If you're selling things that people need and you can get them at a discount, that store's sales are doing pretty well."
Because of this, grocery stores are generally doing better than the rest of the economy, Hamilton said.
He also noted that a fancy frozen pizza might seem like a luxury purchase, but in reality it is still a few dollars cheaper than buying from your local pizza parlor.
When Kraft announced its earnings earlier this week, that was the clear message. Sales of its "convenient meals" rose 8.2 percent driven by DiGiorno frozen pizza and Oscar Mayer meats. Lunchables anyone?
Sales of Kraft's macaroni and cheese dinners are up "double digits" from the same period last year, according to the company's earnings statement. The company's Oreos, Capri Sun and Kool-Aid also saw strong sales.
They're not quite luxury foods, but all can make an affordable treat when budgets are tight.
It's also no surprise that fast food chains are benefiting as Americans still want to eat out but keep a tighter grasp on their wallets.
For instance, Dunkin' Donuts has benefited from consumers who can't afford Starbucks but still need their morning cup of java.
"It's a recession that's not created equal," Hamilton said. "People are obviously going shopping instead of going out."
Another big seller: rotisserie chickens from local grocery stores, Hamilton said.
And McDonald's has seen blockbuster results around the world. The company's first-quarter earnings were up 3.5 percent, driven in part by strong sales of breakfast sandwiches. (McDonald's has also been trying to steal Starbucks' customers with upscale coffees and cafes in some locations.) The company also benefited from strong sales of drinks and chicken sandwiches.
"Consumers seemed more willing to use foodservice as a special occasion on the weekend," Michele Schmal, vice president of foodservice product development at research group NPD said in a recent report. "Perhaps they are looking for a chance for a little escape via affordable luxuries, amidst the economic doom and gloom."