"Those that are shouting out are out of order," said Rep. David Obey, D-Wis., who was presiding in the chair at the time.
Neugebauer insisted in a statement that he was not referring directly to Stupak but to the agreement that the Michigan Democrat helped work out with the White House. That eleventh-hour agreement, under which President Barack Obama said he would issue an executive order pledging that no federal funds be used for abortions, helped seal the last votes Democrats needed to pass the bill.
He said his exact words, referring to that agreement, were "it's a baby killer."
"While I remain heartbroken over the passage of this bill and the tragic consequences it will have for the unborn, I deeply regret that my actions were mistakenly interpreted as a direct reference to Congressman Stupak himself," he said.
But Neugebauer also acknowledged that the House chamber "is a place of decorum and respect. The timing and tone of my comment last night was inappropriate."
Neugebauer is one of the House's most conservative members, consistently speaking out about the need for lower taxes and smaller government. Last year he co-sponsored a resolution requiring that presidential candidates produce copies of their birth certificates. That followed "birther" movement allegations that Obama was born outside the United States and not qualified to be president.
In an interview Monday with Fox News, Stupak lamented the prevalence over the past year of "uncivilized behavior" when lawmakers are trying to speak on the House floor. "We can disagree on these issues as we should and we should have a meaningful debate but personal attacks have no place on it."
The outburst was reminiscent of last September when another GOP conservative, Rep. Joe Wilson of South Carolina, yelled out "you lie" when Obama was addressing Congress on the health care issue. He also issued an apology.
Unlike that incident, Wilson was quickly identified by reporters covering the speech. This time, it was not immediately known who did the shouting and it was nearly a day before Neugebauer came forward.
In Wilson's case, the House on a mainly party line vote passed a resolution of disapproval formally criticizing him for violating basic rules of decorum and civility.
House Majority Leader Steny Hoyer, D-Md., asked Monday about Neugebauer, noted that he had apologized.
"He shouldn't have done it. ... I don't think further action is needed," Hoyer said.
Neugebauer's "baby killer" shout came at the end of a day of passionate and often vitriolic speeches on both sides of the health care issue. There were also tea party movement protests against the health care bill in which some demonstrators used racial slurs against black members of Congress and reportedly spit at one black lawmaker.