Tariq Arian, spokesman for Afghanistan's Interior Ministry, said the bomb targeted the Wazir Akber Khan Mosque at around 7:25pm local time (1500 GMT) on Tuesday when worshippers had gathered for evening prayers.
The mosque is located in a high-security diplomatic area near the offices of several international organizations and embassies.
Prayer leader of the mosque, Mullah Mohammad Ayaz Niazi, was one of two people killed in the attack, Arian said. He was wounded in the attack and died later at a hospital, Al Jazeera reported.
The latest attack in the Afghan capital came after seven civilians were killed in the north of the country in a roadside bomb blast authorities blamed on the Taliban.
No group immediately claimed the evening explosion in central Kabul, which occurred at one of the city's most famous places of worship.
The mosque is located at the main entrance to the Green Zone and is accessible from both inside and outside the tightly controlled area.
Niazi was famous in Kabul and his politically charged sermons were often so well attended that worshippers would spill into the grounds outside the mosque.
Arian initially said the attack was conducted by a suicide bomber. But as investigations proceeded, it was not clear that remained the case.
President Ashraf Ghani's spokesman called the incident a "heinous" attack.
The attack follows a bombing claimed by Daesh (ISIL or ISIS) terrorists against a television station's minibus in central Kabul on Saturday, killing a journalist and the driver.
It also comes after officials said seven civilians were killed late Monday by a roadside bomb linked to the Taliban in northern Afghanistan, even as authorities pressed for peace talks with the group.