The legendary actor, 76, was famously axed from the show back in 2016 after making a shocking comment to X Factor star Katie ...