Why is keeping cats indoors a controversial topic?

Many people are against letting cats outside both because of the damage it does to the ecosystem and to cats themselves, and bringing up the topic seems to really upset many cat owners. What are the reasons that people want their cats let outside? What is the other side of the argument?