The prime minister on Wednesday finally made a public statement naming Mr Kelly who has been enthusiastically promoting two unapproved drugs for months on his Facebook page.

"The views expressed by the member for Hughes do not align with my views or the views of the advice that has been provided to me by the chief medical officer," Mr Morrison told parliament.

The controversial MP met with the prime minister on Wednesday after he was filmed arguing with senior Labor frontbencher Tanya Plibersek at Parliament House.

Following the talks, Mr Kelly agreed to support the government's vaccine rollout despite having earlier refused to commit to the jab.

"I have always sought to support the success of our nation's public health response during the pandemic," he said in a statement.

"I believe that the spread of misinformation can damage the success of our public health response during the pandemic."

Despite his backdown, Mr Kelly's Facebook page remains littered with articles championing hydroxychloroquine and ivermectin.

Mr Morrison said he made it clear to the Hughes MP that vaccination was critical.

"It is our primary responsibility this year as we continue to respond to the pandemic," he said.

Anthony Albanese said that after months of Labor raising the issue, the prime minister had belatedly distanced himself from Mr Kelly.

"The comments of the member for Hughes have been dangerous," the opposition leader said.

Deputy Nationals leader David Littleproud was the first cabinet minister to call out Mr Kelly over his views, declaring the Liberal was "on his own".

Earlier on Wednesday, Mr Kelly accused the national COVID evidence task force - which is made up of top medical experts reviewing science continuously - of not being across the latest studies.

"Catch up with the evidence. Yes, you looked at three studies. Go and look at the other 32 studies and come back," he told reporters.

Ms Plibersek addressed the Liberal backbencher directly as the pair clashed outside media studios surrounded by cameras.

"My mum lives in your electorate and I do not want her exposed to people who aren't going to be vaccinated because of these crazy conspiracy theories," the Labor MP said.

She said it was beyond time for Mr Morrison to tell his backbench MP to shut up.

"We're spending $24 million on a campaign to tell people to get vaccinated and we've got a taxpayer-funded nong running around telling people not to."

Mr Kelly claimed he was not anti-vaccination but said he would consult with his doctor before deciding on which jab he might have.

Reserve Bank Governor Philip Lowe told the National Press Club he would take the vaccine and believed most Australians had a desire to "do the right thing".

"Certainly some people won't want to get vaccinated and they will have concerns but I think enough of us will get vaccinated," Dr Lowe said.

Australian Associated Press