Anthony Albanese has posed a simple question to opponents of an Indigenous voice to parliament and called for the nation to come together.

As thousands of people took part in invasion and survival day rallies across the country – many of them campaigning against an Indigenous voice – the prime minister sharpened his pitch.

“It is something missing from our nation’s birth certificate, it’s missing from our constitution,” Mr Albanese said on Thursday.

“And I say this – if not now, when? If not now, when will this change occur? And if not the people of Australia this year, who will make this change?”

Mr Albanese said he would not engage in “partisan politics” on the matter, including the position taken by Greens senator Lidia Thorpe.

Senator Thorpe has indicated she would not support the voice unless she was satisfied it guaranteed that Indigenous sovereignty wasn’t ceded.

Last year, the National Party announced it would not support the voice, a decision which led to backbencher Andrew Gee resigning from the party to sit on the crossbench.

The Liberal Party has yet to adopt a formal position but Opposition Leader Peter Dutton continues to call for more details.

Mr Albanese said the voice was a “very clear proposition” that would constitutionally recognise Aboriginal and Torres Strait Islander people as Australia’s first peoples.

The newly appointed Senior Australian of the Year Professor Tom Calma also expressed his disappointment in the actions of some federal parliamentarians.

The Indigenous academic and campaigner for the referendum on the voice said the issues facing Indigenous people and the need to step towards reconciliation could co-exist.

“It’s important that we as Aboriginal people have an opportunity to be able to contribute to policies that impact on us and programs and legislation and that’s the first step,” he said.

“If you support the principle of Aboriginal and Torres Strait Islander people having an opportunity to have an input into public policy and into legislation, then that should be what you support.”

Professor Calma said the approach from the Nationals demonstrated naivety and not wanting to understand what was really being proposed. 

“I hope it’s just not being party-politically driven on some of these issues.”

Speaking at the official Australia Day celebration in Canberra, Ngunnawal elder Aunty Violet Sheridan called for national unity on the issue, urging people to vote “yes”.

“I am a proud Ngunnawal Aboriginal woman but I am also a proud Australian, so I want us to come together,” she said.

A referendum on enshrining the voice in the constitution will be held later this year.

By Poppy Johnston in Canberra