Georgia’s Raffensperger says he never believed it was appropriate to talk to Trump about the election