Voters have been warned to reconsider wearing campaign T-shirts and badges when casting their ballot in the Indigenous voice referendum.

A long-standing rule bans campaigning inside polling places or within six metres of an entrance.

The Australian Electoral Commission said if a voter wore a pin, shirt or hat into a polling place it might not be considered campaigning.

But anyone seen talking about the material or gesturing towards it could be in breach of the rules.

“AEC advice is to simply avoid any potential issue by not wearing campaign material into a polling place, or to at least bring along a piece of clothing that allows a voter to cover up,” the commission said on Thursday.

Commission staff will take a “commonsense” approach to talking to voters about the matters, with people urged to be kind.

More than four million Australians have already voted ahead of Saturday’s referendum, which proposes changing the constitution to recognise Indigenous people and to enshrine an advisory body called the voice.

Polling shows the ‘no’ vote is ahead, with one in four voters undecided.

Uluru Dialogue co-chairs Megan Davis and Pat Anderson said Australians had a history of backing underdogs and fair play over “negativity and spin”.

“We remain incredibly optimistic – as optimistic as our old people were in the conversations we had throughout the regional dialogues process,” Professor Davis said.

Independent senator Lidia Thorpe said a failed referendum would be a win for the Black sovereign movement.

“We need an end to the era of powerless advisory bodies, where the role of First Peoples has been to give advice to a colonial government that can and does ignore that advice … they need to come to our table and negotiate,” she said.

She said Indigenous people who resisted colonisation and constitutional recognition would be able to start “a real healing … and truth-telling journey”.

“We’re also not talking about treaty … so I think that there is lots to look forward to and – rather than think that we’ve been defeated – see this as a victory,” the Victorian senator told ABC Radio on Thursday.

A successful referendum requires a majority of ‘yes’ votes in at least four of the six states.

Senator Thorpe said the government should still move to legislate a voice to parliament in the event of a referendum defeat – something the prime minister has ruled out. 

She said the referendum had given a platform to racists and her life was in danger after she had been targeted by neo-Nazis.

Casting an early vote in Alice Springs, prominent ‘no’ proponent and opposition spokeswoman for Indigenous Australians Jacinta Nampijinpa Price said she wanted all Aboriginal people to be able to participate in the economy effectively.

“As it stands, socialism is what has destroyed the Territory and is what continues to destroy the Territory, which is why you get large groups of people gathering to push for a ‘yes’ vote in this community,” she told Sky News. 

Speaking at a ‘yes’ campaign event in Sydney, Marcia Ella-Duncan – the first Aboriginal woman to represent Australia in netball – pointed to the strengths of the La Perouse community working with government, including the successful Gamay Rangers program.

“But it’s a very fragile relationship,” she said.

“It depends entirely on the attitude of parliament of the day.

“What you will hear about today, mark my words, is only here because currently, we have the will of government, but if there’s a change, who knows? 

“A ‘yes’ vote will enshrine the relationship between the parliament and the government of Australia and my people so that what is possible here, is possible anywhere.”

More than 7000 polling centres will operate on Saturday from 8am to 6pm.

By Dominic Giannini and Cassandra Morgan in Canberra