Right beside them, the French celebrated.
It was just like 2008 but with the roles reversed.
This time, it was France chasing down the United States -- and Lochte, no less -- to win another riveting relay at the Olympics.
"We got our revenge," French swimmer Clement Lefert said.
With Phelps looking much stronger than he did the night before, the Americans built a commanding lead over the first three legs of the 4x100-meter freestyle relay Sunday and never really had to worry about the defending world champions from Australia.
When Lochte dove into the water on the anchor leg, he was a half-body length ahead of the field and looking to add another gold to his dominating victory Saturday in the 400 individual medley.
Not so fast.
Or, should we say, not fast enough.
Yannick Agnel, playing the chaser role that Jason Lezak did for the Americans four years ago in this same event, sliced through the water and was right on Lochte's shoulder as they made the flip at the far end of the pool. With about 25 meters to go, they were stroke for stroke. But Lochte, who had already competed in 1,200 meters of racing over the first two days, simply didn't have enough left to hold off the towering, 20-year-old Frenchman, one of the sport's real rising stars.
"I gave everything in the last 50 until he cracked," Agnel said. "In the last 10 meters, I saw that he was really cracking."
Agnel touched in 3 minutes, 9.93 seconds, having gone exactly 1 second faster than Lochte over the last two laps. Lochte and the Americans dropped to silver in 3:10.38, while Australia -- the favorite -- didn't even get a medal. Russia took the bronze in 3:11.41, edging the team from Down Under by 0.22.
Phelps settled for his 17th career medal and completed his collection of Olympic colors, adding a silver to his 14 golds and two bronzes. He also moved a step closer to becoming the most decorated Olympian ever, just one away from tying the mark for most career medals held by Soviet gymnast Larisa Latynina, and has five events to go.
"At least I'm in a medal today," Phelps said ruefully, referring to a fourth-place finish in his first race of the London Games.
But silver was a bitter disappointment for the Americans, who know how the French felt four years ago.
France had the lead in Beijing and its best sprinter, Alain Bernard, going out on the final leg. But Lezak swam the fastest relay leg in history, drafting Bernard along the lane rope and beating him by a scant 0.08 seconds to keep Phelps on track for his record eight gold medals.
That was one of the greatest races in Olympic history.
This one wasn't too shabby, either.
"I was just really excited and I think I overswam the first 50 and it hurt me for the last 50," Lochte said. "But we were able to get a medal, so I guess that's good."
Lochte's reaction was much different than the one he had the night before, when he finished more than 3 seconds ahead of the other medalists and more than 4 seconds ahead of Phelps in the 400 IM.
Even though Phelps got a medal this time, he didn't look much happier. He lingered at the edge of the pool right above Lochte, before going over to congratulate the French.
Phelps put up the fastest time among the American swimmers, covering the second 100 in 47.15 and showing he still intends to be a force at these games after his disappointing start. Nathan Adrian swam the leadoff leg in 47.89, going out faster than Australian star James "The Missile" Magnussen to give the U.S. an early lead. Cullen Jones was solid, too, in the third spot (47.60).
Lochte was handed a lead of more than a half-second, but he couldn't hold it. Agnel covered the final leg in 46.74, while Lochte labored home in 47.74.
Agnel's anchor wasn't quite as spectacular as Lezak's 46.06 at Beijing, but the French had no complaints.
"It's magical, simply magical," Agnel said. "We didn't have too much pressure. We did what we know how to do. Now, Olympic champions. It's brilliant."
They climbed to top step of the podium to receive their medals, looking down at the Americans.
"It's tough," Phelps said. "We'd like to be on top, but Yannick has been swimming well all year and those guys put together a great relay. We tried to get ourselves into as much open water as we could. We had four great guys to get up there and swam as fast as we could. We were the ones that the coaches thought were going to have the best shot. We went out there and raced. That's all you can ask."
The U.S. coaches will surely come under scrutiny for going with Lochte, who had little experience in the 100 free and had never competed on this relay at the Olympics. But, coming off his dominant showing the first night, it's hard to argue about going with a swimmer who appeared to have the hottest hand of all.
"The 100 free, I don't really swim it. I haven't swum it in a long time," Lochte said. "You would think doing distance events, I wouldn't get tired. But sprinting takes a lot out of you."
In an interesting twist, Bernard will get a gold medal even though he didn't swim the final. Amaury Leveaux and Fabien Gilot took the first two legs, but Bernard will be rewarded, too, for taking part in the morning prelims. Maybe that will soothe some bitter feelings from four years ago.
Two more world records fell earlier in the evening.
American Dana Vollmer took down the mark in the 100 butterfly, then Cameron van der Burgh of South Africa broke another in the 100 breaststroke -- denying Japan's Kosuke Kitajima an Olympic threepeat.
Not even through the second night of the London Games, three world records had already been set.
So much for those dire predictions of marks standing for decades after high-tech bodysuits were banned.
This was quite a night for France, and not just because of the relay. Camille Muffat won a riveting 400 freestyle duel with American Allison Schmitt, the two virtually stroke for stroke the entire way. Muffat held on to win by about half a stroke with an Olympic-record time, while Schmitt settled for silver -- a sign of things to come.
Britain's Rebecca Adlington brought out the biggest cheer when she touched third, the home country's first swimming medal of the games.
Vollmer was third at the turn but powered to the wall for a time of 55.98, beating the record of 56.06 set by Sweden's Sarah Sjostrom at the 2009 world championships. Not bad for someone who didn't even qualify for the last Olympics, her career sidetracked by injuries and illness.
"I kept telling myself that my strength is my second 50," Vollmer said. "I kept really calm."
She dropped back her head when saw the time, then broke into a huge smile, slapped the water and pumped her fists.
"I'm on top of the world right now," said Vollmer, who qualified for Athens as a 16-year-old but was a disappointment in 2008. "I still know I can go faster."
Kitajima was trying to become the first male swimmer to win the same race at three straight Olympics. But, like Phelps the night before in the 400 IM, the Japanese star didn't come close.
Van der Burgh made sure of that, dominating the race almost as soon as his head popped out of the water for the first time. He was comfortably ahead at the turn and blew away the field on the return lap to touch in 58.46, knocking off another of the marks set at the 2009 world championships.
Brenton Rickard's time of 58.58 was among the astonishing 43 world records established at that meet in Rome, when rubberized suits took the sport to times that bordered on absurd. The suits have since been banned, with some predicting that it might take decades to go faster in textile suits.
Only two records fell at last year's worlds in Shanghai, but the Olympic meet has already beaten that number with three over the first two days.
Australia's Christian Sprenger took the silver in 58.93, and American Brendan Hansen claimed bronze in 59.49, providing a bit of salve for past Olympic disappointments. Hansen was so disgusted with his performance in Beijing that he retired from the sport, saying he was totally burned out. After giving triathlons a try, he returned to the pool and got back up to speed.
"That's as fast as I can go right now, and I'm really pleased with the outcome," Hansen said.
Kitajima didn't find the speed he needed, struggling home in fifth at 59.79. The night before, Phelps failed in his bid to win a third straight 400 IM title, fading to fourth while Lochte took gold with a dominating performance.
Van der Burgh propped himself on the lane rope, his chest heaving up and down, his hands cupped behind his head, a big smile on his face. He looked up toward the roof and pointed, a nod to his good friend, Alex Dale Oen, the world champion in this event who died suddenly in April from cardiac arrest.
"I just have to pay tribute to Alex Oen," van der Burgh said. "I know that he's been with me this year and helped me to finish the race in such a strong manner. If there is such a thing as a perfect race for me, I definitely think that I submitted it tonight. I don't even care about the world record. Once you become an Olympic champion, you join the club."
Lu Ying gave China another medal, taking silver behind Vollmer in 56.87. Australia's Alicia Coutts grabbed the bronze in 56.94. It was a tough night for Sjostrom, who not only lost her world record but didn't even get a medal, touching fourth in 57.17.
Muffat and Schmitt raced side by side the whole way in the 400 free, with Muffat clinging tenaciously to the lead while the American tried desperately to reel her in. She couldn't pull it off, as Muffat touched in 4 minutes, 1.45 seconds to win by a scant 0.32. Adlington was looking to defend her Olympic title in the event, but the big crowd seemed pleased that she rallied from a sluggish start to at least make the podium.
"I saw them guys. They were off. I just couldn't keep with them," Adlington said. "To be honest, I am so happy I managed to sneak that medal. ... Another Olympic medal is just unbelievable, and there is not an ounce of disappointment in me."
The same couldn't be said for the American men's relay team.