ISWAP split from Boko Haram five years ago and pledged allegiance to Islamic State and has been fighting against the Nigerian armed forces.
Army spokesman Brigadier General Onyema Nwachukwu said troops had encountered ISWAP fighters in Borno state's Askira Uba local government area, where a fierce battle took place and several militants were killed.
"Sadly, a gallant senior officer Brigadier General Dzarma Zirkusu and three soldiers paid the supreme sacrifice ... as they provided reinforcement in a counter offensive against the terrorists," Nwachukwu said in a statement.
Borno state is at the centre of the Islamist insurgency, which has spilled into neighbouring Chad and Cameroon and has left about 300,000 people dead and millions dependent on aid, the United Nations says.
Military sources and residents said ISWAP fighters attacked Askira on Saturday morning with at least 12 gun trucks, burning houses, shops and a school and forcing some residents to flee.
The army had brought in reinforcements but the battle was still ongoing as evening approached, the sources said.
Askira is about 150 kilometres south of Borno state capital Maiduguri and lies along the fringes of Sambisa Forest, the operational base of both Boko Haram and ISWAP.
Security sources said ISWAP fighters had also separately attacked troops near Maiduguri town, but there were no immediate details on casualties.
Nigeria's army said last month it had killed the new ISWAP leader in a military operation, weeks after announcing the death of the group's former head Abu Musab al-Barnawi.