Grand Mufti Sheikh Abdul Azeem al-Afifi died in a Melbourne hospital before dawn on Wednesday, the Australian National Imams Council said in a statement.
"Sheikh Abdel Aziem Al-Afifi has dedicated himself to serving the Australian Muslim Community for over 20 years," it said.
"Sheikh Abdel Aziem Al-Afifi had been widely recognized and respected by the Australian Muslim community and the wider Australian society."
The sheikh, a founding member of the council, was only elected as Grand Mufti in March.
He had been ill for some time.
Australian Multicultural Foundation executive director Hass Dellal told AAP the Grand Mufti was a "great person" with his youth work and promotion of Australian Muslims' contributions to leave "a lasting legacy".
Victorian Minister for Multicultural Affairs Robin Scott tweeted: "My condolences to the family of his Eminence the Grand Mufti of Australia Sheikh Abdel Aziem Al-Afifi and Muslim community. May he rest in peace".
The council executive committee will act in the role of Mufti of Australia until a meeting is held within 15 days to determine a new leader.