Suggestions to encourage more field battles: Diminish castle defense/fortification bonuses,
This has already been done: both castle/wall defenses and city's cultural defenses are down by 50%, and also cultural thresholds are higher. Most cities have total defense bonus of only 20-55%, where in normal game they'd have 40-180%.
tweak AI to sally out and make a stand. If you can't/want this, maybe you can make siege units a little more viable. Gve them a little better stats or introduce a building (or buff the gun factory) to give starting XP to siege units also so they can get CRII right away.
This would require making the AI make stupid things, basically. It's behavior is based on combat odds, so it will sally out only if it has good odds to win, or if the unit is deemed useless enough to sacrifice. Field bonuses are meant to encourage field fighting and the AI technically "understands" them though they don't seem to have much impact on the amount of field battles.
For a history-friendly alternative for siege units, did you try using spies? Apart from the usual Incite Revolt action that strips defenses, there's also the new Sabotage Unit mission, which is there precisely to help in sieges. In my experience, it works great. It's also easier with universal EPs.
AI vassals are useless. They get into enemy territory but do not do anything.
Admitedly, vassals' main goal is their part in Unification Victory. I can't really rewrite Civ4 vassal AI. Most of the time you're supposed to be giving fiefs to vassals, rather than them doing things on their own. I've seen them attacking with some success, even stealing cities from my armies, but it's not common or coordinated. Since vassal civs aren't that good, I was thinking of an alternative in the form of military unit transfer (i.e. vassal is oblieged to give Ashigaru units to its master), but that's rather complicated to even think about it...
Vassalization is sometimes a pain bc clans would rather go down entirely than capitulate. Would you introduce the strict rule that any clan down to its last unit capitulates?
It's not possible. Cases of civs not capitulating when they seemingly should are there because of diplomatic restrictions, i.e. if your victim is also at war with other civ(s) that cannot sign peace with them, or declare war on you, capitulation is simply not possible.
The "Surely, you must be joking!" comment means they have not lost "enough" in this war. If a clan only had 1 city and a few units, then it won't capitulate easily, but that is realistic. This comment shouldn't appear if you took AI's cities and destroyed many units. Other comments usually mean diplomatic restrictions.
