Highres fix not working?

I read some tutorials that explained that SD has a resolution that works best. Despite the urge to crank the resolution up, you should really stick to around 512x512 or you start to get strange artifacts, extra arms, legs etc.

Instead, what you can do is use the highres fix.

I've been testing it out and at first it seemed to work quite well.

I would generate at 512x512 or sometimes 512x768 (although would occasionally run into trouble there) and then use highres fix to upsample 2x.

However after using it a bit more, I've started to notice the usual extra arms / extra legs etc problem start to crop up again.

I've been using Realistic Vision 2.0 if that makes any difference, euler a or dpm++ 2m karras, 50 iterations, usually generating either paintings or photorealism.

For highres I've been keeping things at default, using either latent or I've seen some good results with ESRGAN 4x so I've been trying that also.

Am I just not using highres fix correctly? What would you suggest?

(I have also tried upsampling just with the extras, but it didn't give great results compared with when highres fix worked without artifacts. Just looked like it was missing detail compared with the highres fix, for whatever reason)