A bid to keep five flats in a former hairdressers and garage in Farnworth has fallen flat after an inspector dismissed an appeal.