Benalla residents want the Australian Rail Track Corporation to throw out plans to build a new railway overpass double the height and size of the existing structure for its Inland Rail project.
A "Better Benalla Rail" group has formed to ensure plans drawn up in 2018 don't eventuate.
The group instead want the original standard gauge line moved to the west, which would preclude the need for an overpass and make easy passage for double-stacked trains.
Group member David Moore said Inland Rail presented an opportunity to demolish the current overpass and reconfigure the site to make it better.
"In every way, Benalla stands to lose from this," he said.
"The correspondence is all about why they won't move or relocate 1.3 metres of track, which is what we have been suggesting.
"They state it's beyond their scope and will cost four to six times their budget. Signalling would cost a lot of money and would be difficult to relocate the track and they would need to build a new platform.
"There used to be tracks on the western side and reloading a track isn't difficult at all, it requires two crossovers, it doesn't need a new platform."
Mr Moore said there used to be tracks on the western side and ARTC should be prioritising community needs over its budget.
Since the Benalla station was reconfigured to introduce standard gauge 60 years ago, there has been inadequate public access with minimal car parking, no taxi rank and an unsafe bus turning area, the group says.
Eight railway stations will be upgraded or adjusted in the North East, with plans for Euroa and Glenrowan having raised concerns.
Australian Rail Track Corporation has stated from the outset that there is not enough clearance under the station overpass off Mackellar Street for double stacked freight trains.
To increase clearance, they have proposed to replace the current structure with a higher overpass built to the north of its current location.
BBR chairwoman Suzie Pearce feared the community would again be left with a station that they didn't want.
"Just because they made that decision in 1964, everything has changed," she said.
"It will be disruptive to the operation of the station and be an eye sore to the community."
Murray Smith's home overlooks the current overpass and the Mackellar Street resident it "was a bit of an eyesore" but he'd "accepted it".
"But if you're looking at double the size, if you talk about the volume, the whole situation... you do the maths," he said.
"What's the value of my house going to be worth? I love this place."
Mr Smith said he only gets half of the afternoon sun already, with the size of the current structure.
"I'd love to see it gone. The impact on residents is ... more isolation."
A working group was recently formed by ARTC and Don Firth is one of two Benalla councillors on the group and said a higher overpass would be "an eyesore".
"You have to work within the parameters of the scope of works - the scope of works is purely freight," Cr Firth said.
"[But] would they have got the $230 million [North East line] upgrade, unless the inland rail was going through?
"Personally, I would like to see the standard gauge on the other side of the station where it should have been.
"There's an opportunity there to work with ARTC to get the best thing for Benalla ... I would say they are listening."
ARTC Victoria projects general manager Ed Walker said ARTC had been engaging with stakeholders since 2018.
"Part of ARTC's commitment to Benalla is ensuring what we build complements the Benalla Station and supports opportunities for future development of the area, while remaining within the project's scope, budget and timeframe.
IN OTHER NEWS:
"The newly-established Benalla Working Group will be at the forefront of future discussions, while reaffirming our commitment to work closely with the community and revisit design solutions for Benalla through a structured process.
"While we may not be able to deliver everything the community is asking for, we are committed to doing as much as we can to incorporate what the community is telling us."