A multi-billion dollar fund to build tens of thousands of new social and affordable housing could be heading for defeat in parliament following concerns it would not fix problems in the sector.

Under the government’s $10 billion Housing Australia Future Fund, 30,000 new social housing domicles would be built in the first five years of the scheme, with up to $500 million spent each year.

However, critical Senate crossbenchers have raised issues with the fund, saying it was not enough to address the crisis in affordable housing.

Tasmanian senator Jacqui Lambie said she supported the fund but improvements were needed.

“We are worried about the $500 million annual cap on disbursements to the fund … (with) the amount of houses that you need to build, (it) is only going to end up about $80,000 per house,” she told ABC Radio on Tuesday.

“We are worried also with the inflation on what that $500 million looks like in the next nine or 10 years.”

Senator Lambie called for upgrades and repairs to housing in remote Indigenous communities.

The government needs the support of the Greens and at least two crossbenchers for the fund to pass the Senate.

Senator Lambie said negotiations with the government on what the fund would look like were continuing.

“We’ll be going for whatever extra money or whatever we can get up, to get those extra houses built today,” she said. 

Housing Minister Julie Collins said the fund would be essential to provide support.

“(The reform will) deliver thousands of new social and affordable homes right across Australia for people who need it most,” she told parliament.

“Australians who need it most need this bill to be passed and they need it passed quickly.”

Independent senator David Pocock said the fund represented a chance to build more affordable housing.

“It’s not ambitious enough – we’re seeing so much need across the country,” he said in Canberra.

“We’ve got so many Australians counting on politicians to do the right thing here.”

The coalition is set to oppose the housing fund, with Nationals leader David Littleproud saying the fund didn’t properly address the issue.

“This is a failure of state governments in particular. This is their bailiwick,” he told ABC TV.

“The mechanism they’re putting in place is costly and doesn’t go to the heart of the issue. And what it also does is incentivise the states to walk away from their responsibilities.”

Unions also urged the government to make the housing fund more ambitious.

Incoming CFMEU national secretary Zach Smith said more was needed, with unmet demand for social housing at 650,000 homes.

By Andrew Brown in Canberra