I think you're laboring under the misconception that war hampers the player against the AI. In fact, war is the aspect of the game the AI does the worst. It's shooting itself in the foot every time it does that. It's throwing away hammers in order to give your units XP. Why should the AI self-destruct itself and send you resources when you're already winning?
I will often war in order to get ahead in the game, even now. The easiest way to get ahead is to go conquer some Wonder-packed capital.
Granted, but that should be taken care of in the military comparison routines. If they have significantly more military strength that they can bring to bear then attacking might be the best solution.
But in any case, there has to be a real THREAT of attacking to push a boomer to actually divert production and money to military defenses or else the lead just continues to compound and grow.
I have seen plenty of times in BNW where the AI has more than enough to utterly wipe me out before I would be able to take effective measures had they attacked. But they don't, so I boom, they lose.
And again, I don't think they should just come charging across the borders for minor things but once I'm clearly booming and not responding to demands to cease gobbling CSs or stop taking all the wonders or whatever I should start to feel pressure that if I DONT build a competent military, I'm going to get attacked. This is especially true because SO MUCH damage can be done simply by raiding the trade routes. The AI doesn't even have to come barreling into my defenses if they go after the trade. But I definitely would need a military to protect the trade or else I can be content to do without.
That is the key IMO. There should be real consequences for over-booming like there are consequences for other ways of pursuing victory. And at that very least, those consequences should include getting less or less favorable resource trading and especially being cut off from RAs etc. Sure maybe an AI here and there won't mind and will continue to do so but that shouldn't be the norm (ie, helping me to increase my lead).
So in summary, it should be harder to maintain an economic/tech/diplo lead without consequence ranging from increased economic/diplo pressure to attacks on trade to all-out attacks if the situation warrants.