Yeah, but the point is that is one of the only times an all out war is in your best interest. Late in the game all out wars get less and less useful and actually can hurt your position relative to other players. One of the reasons pointed out is culture, which makes it impossible to utilize what you just fought for (razing could help somewhat, but again, you wont have an operational city contributing to your empire any time soon).
It comes down to purpose for war. If your purpose is to gain territory, you have to start early. If your purpose is to hurt another player, make sure you do not expose yourself to losing ground economically in the process. In a close game ... war is amlost intolerable because it will immediatly knock you out of the running. Its like a reverse lottery where you sit there trying to boom holding that lotto ticket and hoping to god your number isnt called. Then when that idiot Aztec decides he is going to war with you, you either can let him steamroll your improvements/cities or dig a big hole for yourself to make up against the other peaceful countries who are not burdened with war.
So the complaint is you have all these cool units late game, but very few reason to use them, and througout the early game you have a great need for war and not enough time to do it.