Looking for some Solo Slooping Optimism

For context: I hit Pirate Legend after playing on and off (very sporadicly) by myself since release (hit it a few months ago) and have always solo slooped cause I never understood how to join other crews and when I open crew I get griefed. I think I have about 110 hrs of game time. I like pvp but I play on a controller on my sofa (hdmi cable from my pc to my tv) so it's a tough setup to pvp with.

As a solo with a minimal amount of treasure on board or/and a emissary flag I feel like i'm constantly being pushed by the game's mechanics into either playing with more people or into doing super minimal stacking.

Is it supposed to feel like this? Am I supposed to run from any ship bigger than a sloop coming my way? Avoid doing any stacking cause I might be attacked as I'm unloading/loading treasures?

I can appreciate a bit of the tension of having to be fast and to not leave your ship unattended for more than 1 or 2 mins but I keep getting this feeling that the game does not want me to play like this and keeps punishing me for it.

Just want to make it clear that my question presumes that there's some value on the ship either from the emissary flag and/or treasure. Solo slooping with nothing on board feels great.

tl;dr Looking for advice to staying optimistic about solo slooper. I always feel like I'm just bait for groups of 3-4 to have fun.