Sarabjit Singh was attacked last Friday and had been comatose and on a ventilator for days before he died at Jinnah Hospital in the eastern city of Lahore, according to the Pakistani foreign office.
Indian Prime Minister Manmohan Singh called it "particularly regrettable" that Pakistan had not heeded pleas to take a humanitarian view of the prisoner's case and allow him to return after he had served 20 years in prison.
Singh was arrested in 1990 after bombings in Lahore and Faisalabad, Pakistan, that killed 14 people. He was convicted of spying and carrying out the bomb blasts, and the death sentence he received was upheld in Pakistani superior courts.
His family maintained Singh was innocent and had entered Pakistan inadvertently from his hometown of Bhikiwind in northern Punjab state bordering Pakistan.
Zulifqar Hameed, a senior Pakistani police investigator, said police are still determining whether to bring murder charges against the two men who attacked Singh and are questioning them to determine their motive.
India and Pakistan have fought three major wars since they achieved independence from Britain in 1947. Relations have experienced several ups and downs in recent years.
Indian External Affairs Minister Salman Khurshid said Thursday that ties "have been hurt by this terrible tragedy."
"For the present, I can only say that it is a terrible psychological and emotional setback to all of us and I believe to what we have been trying to do in terms of creating greater cohesion between people of India and people of Pakistan," Khurshid told reporters.
Rajnath Singh, president of the main opposition Hindu nationalist Bharatiya Janata Party, demanded that India scale down the level of diplomatic ties with Pakistan.
"The Indian high commissioner in Pakistan should be called back for the time being until Pakistan gives credible assurances that it will not allow its territory to be used to promote terrorism against India and that all Indian prisoners are safe in Pakistani jails," he told reporters.
Singh's sister Dalbir Kaur, who visited him in the Lahore hospital early this week, said she had asked the Indian government to seek tighter security for Singh after New Delhi hanged Mohammed Afzal Guru, a Kashmiri man, in February. Singh had feared being attacked by other inmates, she said.
Guru was convicted in a 2001 attack on India's Parliament that left 14 people dead. Several rights groups across India and political groups in Indian-controlled Kashmir have said that Guru did not get a fair trial. Pakistan's lower house of Parliament passed a resolution condemning the hanging.
Bharatiya Janata leader Ravi Shankar Prasad said he was "very pained by the gross indifference" of the Indian government for not pressuring Pakistan enough to win Singh's release earlier.
Manish Tewari, India's information and broadcasting minister, said his government had been pressing Pakistan to release Singh since 2005.
External Affairs Ministry spokesman Syed Akbaruddin said India is demanding a thorough investigation of the attack.
"This was simply the killing of an Indian citizen while in the custody of Pakistani authorities," he said in a statement. The attack "highlights the need for a concerted action by Pakistan to safeguard Indian prisoners in Pakistan."
Take ABC13 with you!
Download our free apps for iPhone, iPad, Android and Blackberry devices