Interesting test, thanks for the effort! Without having any real experience modding, I suspect the core of the issue is that the AI is implemented entirely on a per-unit basis with absolutely no kind of unit coordination (outside of the fact that each unit knows if its at war or not).-- Japan did try to attack one of the two main, productive and protected, cities. However it did little more than cycle in a lone unit - attacked by my archers - and then cycled in another unit -- attacked by my archers. Japan made no attempt at a unit rush - even though it had the capability and numbers to do so.
If this is the case, I doubt a mod is going to be able to make any serious in-roads on military strategy. One way or another, units need to be able to take "orders" from an algorithm or process in charge of a coordinated strategy. Regardless of how smart each unit may be, I don't think an "every unit for itself" implementation is ever going to work out well.
but atm I feel like it has too many collateral issues (recent cultural issue is one, settlers idle is another), I know some of them might be vanilla based. It also has too many changes that honestly I feel are not needed. I'd rather be able to pinpoint those issues myself than having a mod that tries to correct too many aspect of AI at once. Anyway, not trying to argue, do your thing the way you want it, but if you could provide a "limited" version of the mod, you would make a happy man.