I've always felt Tony has got to be physically ripped and muscular enough to wield his suits, atleast the earlier more mechanical versions of it. So maybe he must be working out regularly to use it. There's no way he could be unfit after flying around in that heavy metal under extreme g forces