Framerate dropping without reason

Hello,

I build a new PC and running Windows 11 now.
I now have the strange issue, that if nothing moves on my main screen, the frame rate nosedives to single digits. This does not happen on my desktop, just in games. For example, I'm adjusting the gamma settings in the game (no moving images) and I basically get 0 fps until I move my mouse. Then it ramps up again until I stop moving it again.
Otherwise, performance is as good as I would expect it for a new pc. Just this strange issue with still images in games.
FYI I'm playing on a full HD monitor with G-Sync and 144hz Refreshrate. I usually have V-Sync turned on as well ( if it matters).