With more than 99 percent of polling stations reporting, 50.2 percent of ballots opposed the accord with the Revolutionary Armed Forces of Colombia while 49.8 percent favored it - a difference of less than 57,000 votes out of a total of 13 million. Pre-election polls had predicted the "yes" vote would win by an almost two-to-one margin.
"I won't give up. I'll continue search for peace until the last moment of my mandate," Santos said in a televised address recognizing his defeat.
He ordered his negotiators to return to Cuba on Monday to consult with FARC leaders who were awaiting results on the communist island. He also promised to listen to opponents in a bid to save - and strengthen - the deal, which he said is Colombia's best chance for ending a conflict that has killed 220,000 people and driven almost 8 million people from their homes.
The shock outcome, comparable to Britain's decision to leave the European Union in the Brexit vote, opens an uncertain outlook for the peace accord that was signed less than a week ago by Santos and the FARC in a ceremony attended by heads of state, U.N. Secretary-General Ban Ki-moon and U.S. Secretary of State John Kerry.
Opposition to the accord, led by influential former President Alvaro Uribe, argued that the government was appeasing the FARC and setting a bad example that criminal gangs would seize on. If the "no" vote prevailed, Uribe said, the government should return to the negotiating table.
But that is an option that Santos has previously ruled out.
Early in the day, FARC leaders, including Timochenko and Ivan Marquez, sat in leather recliners at Club Havana, once Cuba's most exclusive beach club, watching the referendum results on a flat-screen TV. Initially the atmosphere was festive, with the guerrillas laughing and joking while snacking on cheese-and-olive hors d'oeuvres, smoking cigars and visiting an open bar.
But the mood soured as results began to come in, and the rebel commanders talked in hushed tones on cellphones, conferred quietly and asked journalists to leave the room.
"The FARC deeply regret that the destructive power of those who sow hatred and revenge have influenced the Colombian people's opinion," Timochenko told reporters later.
He said the rebel group's commitment to peace remains intact. "The FARC reiterates its desire for peace and our willingness to use only words as a weapon for building the future."
The highly polarized campaign exposed how steep a challenge the government would face implementing the 297-page accord and bringing about real reconciliation. Colombians overwhelmingly loathe the FARC, which the U.S. considers a terrorist group, and many considered provisions in the accord that would spare the rebels jail time an insult to victims of the long-running conflict.
Turnout was low, less than the 40 percent seen in recent congressional elections, a further sign to some analysts that Colombians' enthusiasm for the ambitious accord was lacking. Heavy rains from Hurricane Matthew hurt turnout along the Caribbean coast, where the "yes" vote won by a comfortable double-digit margin.
In the past month, ever since the deal was announced in Cuba after four years of grueling negotiations, the government spent heavily on television ads and staged concerts and peace rallies around the country to get out the vote. It even enrolled the help of U2's Bono and former Beatle Ringo Starr. And for the first time in an election, it made ballots available in Braille so blind Colombians could vote.
Santos had urged his compatriots to vote early and take inspiration from Indian independence leader Mahatma Gandhi.
"We in Colombia have to adopt this culture of non-violence," Santos said shortly after casting his ballot in a washed-out Plaza Bolivar next to the presidential palace. "All of us can be protagonists in this historic change taking place in our nation."
The FARC in recent days made an effort to show its commitment to peace is real. Twice this week leaders of the group traveled to areas hit hard by violence to apologize for massacres committed by their troops and discuss with communities how they can compensate victims.
On Saturday, in the presence of United Nations observers, the FARC voluntarily destroyed 620 kilograms of grenades and light explosives. It also said they would compensate victims with financial resources and land holdings accumulated during the war.