"I will not sit idly as they use me as a prop," the former president said.