Australians who oppose the Indigenous voice to parliament are not racist, Anthony Albanese says.

It follows Opposition Leader Peter Dutton accusing the prime minister of name calling over the voice debate.

Asked if people who didn’t back the advisory body were racist, Mr Albanese responded “no” and said voters will have a range of views they’re entitled to.

“Peter Dutton is doing his best to turn up the heat on the debate,” he told Adelaide’s 5AA radio.

Asked if the voice was at risk of failing due to rising cost of living pressures, Mr Albanese said the government had provided a relief package, and was working to help people.

“At the same time we are going to give the Australian people the opportunity to vote on something that overwhelmingly will not have a direct impact on most Australians,” he said. 

“But it just might make a positive difference for some of the most disadvantaged Australians. 

“This is something that is within the Australian ethos of the fair go.”

Delivering the Lowitja O’Donoghue Oration in Adelaide on Monday, Mr Albanese said he believed Australians would wake up on that morning with “the strongest sense of ourselves”.

“A great nation that has dared to become even greater, not just to ourselves but to the world,” he said. 

His speech came ahead of federal parliament’s return, when MPs will continue debate on the proposed wording to alter the constitution. 

The legislation is expected to be voted on in the lower house this week, before it heads to the Senate.

Once it passes both houses of parliament, the referendum will be held between October and November this year.

Mr Albanese said that after a successful referendum, treaty and truth-telling would be part of the nation’s next phase of reconciliation with Indigenous people. 

“One of the things that a voice to parliament will be able to do is talk about the need for agreement making and coming together after a conflict, and part of that is truth-telling about our history,” he said.  

The prime minister used his speech to call out the “fog of fiction and misunderstanding” that had been peddled about the voice in the past few months. 

“This hasn’t been rushed into, there have been no shortcuts, nor is this something … which began in Canberra,” he said.

But Indigenous leader and prominent ‘no’ advocate Warren Mundine said there was not enough information on the voice. 

“When I talk to people on the ground and listen to people talking … I say the biggest majority of people are not ‘yes’ and ‘no’ people,” he told ABC 7.30. 

“They are people who haven’t even heard of the voice and other people have heard of it but don’t know what it is yet.”

By Maeve Bannister and Tess Ikonomou in Canberra