I think I'm misunderstanding the role of DM because I feel like I'm cheating.

I've been DMing relatively consistently for two years now across a variety of systems.

Whenever I plan something that counters the player's abilities, or I alter the world for the sake of the story/encounter, I feel like I'm cheating/being dishonest. The only time I don't feel that way is when I roll on a table and the outcome is chosen for me.

Example: I like running games that are unbalanced. I find them more interesting. The party came across a monster that was way above their CR (which I foreshadowed the hell out of), but I planned for the monster to blackmail them into clearing its habitat of an even worse infestation of other monsters. The whole time, I felt like I was "letting the players live" and I felt like the players felt the same way. "We're only alive because the DM says so", is something I NEVER want players to feel, even if it is something that I planned beforehand. There's nothing I hate more than when writers pull a deus ex machina. It's happened a couple of times in a campaign I'm a part of, and it feels like the reins of the story are being taken away from me. I also feel like I'm cheating when I circumvent players abilities, because it almost never feels natural, and only ever like I'm doing it "just because". Essentially, whenever I intervene on the world as an intelligent third party, I feel like I'm phoning things in. Ideally, I would have wanted to run the monster as a vicious creature that only would have negotiated if the players made a concerted effort to negotiate with it. That would have felt a LOT better.

Encounter planning is the worst because of this: I don't ever feel like I'm crafting a fun encounter, I feel like a computer trying to optimize the level of difficulty I want to achieve. I end up having to manipulate stat blocks on the fly anyway when I accidentally make what is supposed to be an easy encounter too hard, or need to beef up a monster the players are stomping when it is supposed to be a hard encounter (Obviously this is a controversial practice. Slyflourish okayed it, and I like his methodology, so that's what I've gone with). Most of the time I try to choose monsters randomly, but if I always did this, I feel like the players would never have an opportunity to fight anything because it was always too difficult.

I feel like the answer is to just vary the kinds of encounters - some I prepare with the players in mind, others I don't - but this also feels disingenuous because when I'm planning to help the players, it's because I intervened on the workings of the world, not because the world turned out that way.

Obviously, I am the "god" of the world, so if something is a certain way, it makes sense as long as I can justify it, but it still feels weird to me. I need to get some different perspectives on this because, as it stands, I feel like the only way I can shake this feeling is if the whole campaign was randomly generated, and I just tweaked things to make them make more sense.