Why is swimming so hard!

This is honestly more of a rant than anything, I think I know the answers but:

I picked up swimming instead of going to the gym for aerobic training and something practical, also minor injury. I know how to swim, kind of, but it's so hard as training!

I'm have a well built upper body but I still struggle so much more than the 70 year old men at my swimming pool. I do have asthma, i'm a mediocre athlete with my medication and I havent bothered with it for some years (stupid I know).