I pillage when I'm on the back-foot in a war. The gold certainly helps me, and the loss of income hurts the enemy, and it's typically much easier to run around pillaging stuff than it is to capture cities. So if I'm struggling in a war, that's what I do. If I'm stronger than they are then I don't pillage - I just capture the cities instead.