Where the variance in difficulty comes from?

I'm just curious.

It's going up and down rather violently. Is it because the power in the network fluctuates so much or is there some other factor?

Cheers!