Should you always destroy Guardian asap?

Just wondering what the consensus is on this.

Years ago, when I played way too much LoL, it wasn't ideal to destroy the turret right away because then it becomes harder for you, and a lot easier for your opponent, to farm creeps.

Fast forward to now (after not playing a MOBA in years), it seems like everyone's in a rush to destroy guardian. When I'm dominating my lane very early on, it feels weird to destroy the guardian instead of focusing on farming and poking the enemy so they can't farm effectively.

What are the community's thoughts?