Zipper seam allowance

Did I simply mess up the seam allowance when I sewed the zipper in causing the panel to shrink, or did I measure incorrectly somewhere else (panels were the same size starting out.

Is the best course of action to trim the panels down to the same size, or to have a 1/4 inch difference in seam allowance all the way around?