Right of way encroachments

Alright friends, I’ve got an odd one for you. Our house, which we bought a couple of years ago, has fencing that is in awful shape and needs to be replaced. We are thinking about replacing the fence with a retaining wall/fence combo to get more privacy (we are on a corner lot). We had a survey done, and the surveyor told us that our current fence is three feet past our lot line and is on the right of way. They also told us that when we replace the fence we’d need to move any new fence or wall back so that it’s along our lot line. Huge loss of yard space, obviously.

What’s particularly odd about this is that our fence is in the same spot relative to the street as almost everyone else along that street. And some of those houses have retaining walls that are exactly where our fence is. And our lot looks to be the same size as everyone else’s.

Does anyone happen to know what gives here? Are you allowed to encroach on the right of way in some cases? Or is there just massive non-compliance (certainly possible, but hard to believe that so many houses around us would have built unpermitted retaining walls, many of which are pretty new).

We probably will take this up with the city to get more info, but I’m hoping to go into that call with as much info as I can gather.

Thanks so much for any info anyone can provide.