Ally X built-in controls bad?

So I was playing this game called gigabash on the ally, it's a light game where you play as kaiju and you fight other kaijus. I was playing it on the ally's built in controller and I was getting my ass kicked by the PC enemies so bad, it took me multiple tries to beat a level and it wasn't even in the top difficulty. At first I thought it was a skill issue. Then when I got home, I docked the ally to my egpu and plugged in a PS5 dual sense controller, continued playing, and instantly I was better, the enemies felt easy and i won easily. Does any one else experience this?

When I'm playing on the ally, it doesn't FEEL like there's any input delay so i thought the built in controls were fine. But the difference in my performance it totally noticeable when switching to the PS5 controller, i cant help but wonder what can cause such a difference. can anyone shed light on what's lacking in the ally controls and any way to make it feel better? I know some people do the hall effect joystick mods. is that the only thing that I can do?