The 20-year-old was taken into custody after threatening to harm himself.
Police said he called a woman in Victoria and made the threats and she then contacted Albury officers because she was concerned he had access to firearms in the Cadell St house.
Police negotiated with the man over the phone before he was detained.
No-one was hurt in the drama which resulted in barricades being placed at each end of the street and officers wearing bullet-proof vests.
A long-arm and pistol were found in the house, police said.