Actually, my experience is similar to yours...in a real game, I've only used the individual pieces of the Caterpillar in actual battle.
However, mathematical logic shows that by linking the pieces together, they become stronger. Defensive bonuses are much more useful if they actually get used in battle. Consider the following:
1) A 3-length Caterpillar is sufficient to block forward movement of the enemy SOD, assuming correct movement of the Caterpillar.
2) A 4-length Caterpillar is sufficient to block both forward and lateral movement of the enemy SOD, assuming correct movement of the Caterpillar.
3) The low-defense tiles such as Plains can be improved by building Forts. This adds the following bonuses: the +25% bonus of the Fort, the inherent +25% city defense of the Longbowman, and the +20%/+45% city defense of CG1/CG2. This comes out to be +50% for a base Longbowman, +70% for one with CG1, and +95% for one with CG2. If you happen to be playing a Protective leader, chances are you can also have CG3, improving the bonus to +125%. And this is all from a Fort on a flatland tile.
You don't even need to be in your own cultural borders. Suppose that the enemy stack must move through "no-man's land". In this case, you can use the Caterpillar defense in both "no-man's land" and your own land, increasing its effectiveness.
1. I would rather have a cottage or a workshop on that tile.
2. Forts make the attackers CR promotions effective.