Colin James Ambrym, 35, was released from custody on Thursday and had planned to return to the Border.

He had served a jail term after entering the 15-year-old's central Albury home on June 21, 2014, putting his hand over her mouth, removing her clothes and sexually assaulting the teenage victim.

He fled after the girl's father arrived home, but was linked to the incident by a fingerprint and his DNA.

Ambrym was released on parole in April 2018 but was sent back to prison after approaching a 16-year-old girl on a bus and masturbating in front of her.

He was due to be released last July, but the 35-year-old chose to remain in custody until his sentence expired on Thursday.

Authorities have successfully applied for an interim order which put a string of restrictions on Ambrym.

He has been assessed as posing a "well above average" risk of future sexual offending, and a moderate to high risk of committing other offences.

While he had undertaken sex offender treatment during his first stint behind bars, he hasn't engaged in recent programs in custody.

Supreme Court Justice Robert Hulme noted that Ambrym's response to supervision "has varied and has sometimes been quite poor".

Alcohol and ice use are considered risk factors of him committing another sex offence.

The 35-year-old is required to undergo electronic monitoring to check his movement, but the court heard such monitoring would not have prevented his earlier offending.

His phones and other devices will be checked, he must live at an approved address and he must notify authorities of where he is, where he is going, and what he intends to do.

Two psychologists or psychiatrists will assess Ambrym before the matter returns to court at a date to be fixed.