A Post Office that has been a hub of the city centre for decades is in danger of closing down, it has been revealed.