I can say there is an alternative approach to war weariness we could consider. Instead of making it a happiness thing, make it purely a supply thing (which is already a factor of weariness). Have it so that as you war your supply starts to shrink more and more, eventually meaning you either can't field a full army anymore, or you suffer the big growth and production penalties. This already happens now just make it a stronger effect and remove the happiness part.
And while the defender would lose supply as well, ultimately defenders generally need less supply than the offense does, so there is a balance there.
That removes some of the thematic concern (aka why are my people upset for my people killing the enemy). Its more "war exhaustion", it just reflects that wars are very expensive and at some point your supplies start to dwindle, and you have to stop fighting to rebuild.