Senator Penny Wong asked the Senate on Tuesday to support a motion to request Governor-General Sir Peter Cosgrove remove Mr Heydon as commissioner by revoking the Letters Patent issued to him.

The motion said Mr Heydon's acceptance of an invitation to speak at a Liberal Party fundraiser meant he "has failed to uphold the standards of impartiality expected of a holder of the office of royal commissioner".

Mr Heydon decided not to step down from the commission last month, after the ACTU, CFMEU and National Workers Union accused him of creating apprehended bias by accepting the invitation while he was a commissioner.

The motion was defeated, with 34 senators supporting it, and 34 opposed to it.

Independent senators Jackie Lambie and Glenn Lazarus supported the motion, while senators Nick Xenophon, John Madigan, Family First Senator Bob Day and Australian Motoring Enthusiast Ricky Muir opposed it.

Liberal Democrat senator David Leyonhjelm and Palmer United Party senator Dio Wang abstained.

The unions have not publicly announced whether they will ask the Federal Court to intervene. This is now the only option left to them to have Mr Heydon removed, with the Abbott government vigorously defending the commissioner's integrity.

A spokeswoman for the ACTU and the CFMEU, and a spokesman for the National Workers Union said they had no further comment on their decisions on Tuesday afternoon.

Senator Wong, the leader of the opposition in the Senate, earlier said: "It is untenable for a royal commissioner who is conducting a politically charged inquiry to be politically compromised."

Most Australians, she said, would disagree with Mr Heydon's finding that the event, named after former Liberal member and High Court Chief Justice Garfield Barwick, was not a Liberal Party event.

"His actions contradict his words because he withdrew from the event belatedly, telling organisers he would be unable to deliver the address if there were any possibility it could be described as a Liberal party event. You have to ask yourself, 'If it wasn't a Liberal party event why did he withdraw?'"

Mr Heydon dismissed an application by the unions for him to disqualify himself last week, saying that the "fair-minded lay observer" would not apprehend he might not "bring an impartial mind" to the inquiry.

He said in his ruling that the hypothetical observer would believe he had withdrawn from the event "not because to do it will give rise to a reasonable apprehension of bias, but because the decision-maker for sensible reasons of risk management and self-preservation wishes to avoid the attacks of the suspicious and the malicious."

Attorney-General George Brandis, QC, called Senator Wong's criticism of Mr Heydon, a former High Court judge, "utterly contemptible."

Senator Brandis said that if passed, the motion would violate the fundamental constitutional principle that the Governor-General only acted on his or her ministers' advice unless they used the reserve powers of the Crown, which had only happened twice before in Australia.

"Nobody suggests that this case falls within that exception," Senator Brandis said.

"By requesting the Governor-General act contrary to the advice of ministers the resolution asks (him) to act unconstitutionally."

The resolution would also interfere with "judicial process" and violate the separation of powers because "the appropriate place for any challenge to the royal commissioner's decision not to disqualify himself is the courts."

Follow us on Twitter