Opposition Leader Peter Dutton has backed in calls for further practical solutions to improve Indigenous outcomes, rather than a constitutionally enshrined voice.

Speaking from Alice Springs, Mr Dutton said more action was needed on the ground to improve safety following a spike in crime rates.

“If you aren’t out there, and you aren’t listening to what people are saying, then it’s very hard to find the solutions. Solutions aren’t going to be given to you by bureaucrats in Canberra,” Mr Dutton told reporters on Wednesday.

“I don’t believe that a Canberra voice of 24 people who predominantly come from capital cities is going to be the solution to the problems here on the ground. If it did, I’d embrace it straight away.”

Mr Dutton’s comments follow the resignation from the shadow cabinet of shadow attorney-general and Indigenous Australians spokesman Julian Leeser, who will campaign for the ‘yes’ vote.

Moderate Liberals have hoped the resignation of a constitutional conservative would help to get hesitant Australians over the line in supporting the voice to parliament and executive government.

Mr Dutton said the voice would not provide practical outcomes for Indigenous communities,

“The prime minister at the moment is putting forward a model which divides our country,” he said.

“He won’t look Northern Territorians in the eye and explain how it is that the voice is going to provide solutions to these problems and I just don’t believe that it will.”

Opposition foreign affairs spokesman Simon Birmingham said he would not resign from the shadow cabinet, despite indicating he would not campaign for the ‘no’ vote.

“I don’t wish to see an unsuccessful referendum put forward to the Australian people,” he told Sky News.

“It’s … not my intention (to campaign for ‘no’). My intention is to respect the Australian people who will go about this referendum applying their judgement to the issues they have before them.”

Mr Leeser said he would work with the government to ensure the best chances of success by bringing more conservatives onside, saying polls putting support for the voice at slightly more than 50 per cent were not good enough.

He said he would push to strip out the second proposed clause referencing representations to the parliament and executive government in a bid to allay the concerns of Australians worried about the voice overreaching.

“The way it’s drafted currently has some risk and it’s better to remove that to encourage more Australians to vote for it,” Mr Leeser said.

But senior Labor minister Penny Wong said the clause would ensure Indigenous people had input into policy that affected them.

“If we look to our history, much of what we would now say were some of the wrong decisions that governments took in relation to First Nations peoples and communities were taken by executive governments,” she said.

Former Liberal Indigenous Australians minister Ken Wyatt, who was the first Aboriginal person elected to the House of Representatives, hit back at opposition claims the voice would be “elitist” and Canberra-focused.

“It’s not a Canberra voice. It’s not elite. It’s people from the grassroots,” he said.

By Dominic Giannini and Andrew Brown in Canberra