Ok, lot to respond to here, so I'll try to get to each point.
I'm going to start at the beginning though--why I'm even messing with this and what has been done. Most of my testing working on this mod I've done on Monarch difficutly, no tech-trading, aggressive AI, normal size map, epic. Time and time again I noticed that I consistently out teched the AIs so that by turn 250-300, the
mid-game, I was 5-6 techs ahead of the tech leader at least, even when I didn't have a science heavy strategy. As I looked into why, turning on debug mode, noting the AI's builds, expansion, economy, research, etc. I found as I said before a primary problem of over expansion too early, poor early spice production, and poor emphasis on 'economic' buildings and research when they got into trouble, as well as over emphasis on espionage spending in the early game. This lead to a downward spiral where by the time they researched the necessary economic techs to recover, the player was well ahead.
Now, I came back to that because I want to point out that that is still the main part of what I've improved in the 'AI' work I've done. The second 'phase' we are talking about now is largely building upon that improved economic game by the AI and what that lets us do with other areas. But I always start any work with the idea that there is a problem that needs to be solved, and so that every one is on the same page, the problem statement if you will for this 'phase' is that under higher difficulty levels there is a trend towards 'unit spam'. To define that, it's an AI that builds abnormally/grossly more units than what a similar human player with that empire could build in a way that makes gameplay monotonous and victories impossible.
So, to lay out a few things about this, for reasons I've covered concerning the the vanilla handicap values
in conjunction with new unit building logic from the Better AI merge we see this more and more the higher the difficulty level selected, and increasingly towards the late game more so than the early game. This basic statement covers the reasons I've taken the steps I have.
Now it's important to note, I do think and agree that the AI requires more units than a human to pose a challenge, and as a corollary, those units need to be relatively on par technologically with the human. There should also be a moderate increase in the ratio of AI units to human units likely to be seen at each increased difficulty level. If you combine these statements with the problem statement above, you have what I'm trying to achieve in testing.
The ratio of what army an AI can support with a given empire should be within no more than 10-30% more than a human could given the same empire dependent on difficulty level for it to not reach the 'tedious' point I'm thinking. If the player does well, he should be able to pull off an invasion without needing to sink his whole economy or feel like he's playing a game of whack-a-mole instead of civilization. I'm hoping we can all agree that this is a good goal.
I think all gamers have heard many claims about AI performance even from professional game developers to be pretty skeptical about such things.
And not knowing anything about you or prior work you've done, I don't think it's unreasonable to raise a few issues to check that you've considered them or thought of the consequences.
I'm certainly no AI expert, but I'm pretty good at designing models and in thinking about unintended consequences. If you've already thought of the issues I raise, then great.
I'm also very good at reasoning through things and following the logical conclusions they might lead to, much as you describe yourself. I'm a very intelligent and capable person, always have been, so it's always an odd balancing act when stepping into a new environment where you have to prove that all over again to new people. However I also know there's no fast track to achieving that, you can only do the best you can on what you're doing and hope it's recognized in time.
I'm also smart enough to know I (or anyone else) can't anticipate everything. Again because of the large amount of variables affecting the amalgamation of AI decisions there is a lot of trial and error to see what tends to cause what, which is why I'm not going to post every possible solution I come up for discussion with before trying it. I've got to weed out ideas before finding the best ones, and any discussion of these kinds of mechanics before they are tested and balanced is pure speculation, including my own thoughts on them. If there is one thing I have little patience for it is unsubstantiated speculation.
I guess what worries me is the AI's (lack of) strategic combat ability.
You and me both
, but that's for another time. In particular behavior concerning building of effective city attack stacks, city attacks themselves, and the area you pointed to are all good areas that could be improved, but it is very time consuming work. For now, as I said above, I am and have been aiming for reducing real unit spam as I defined above, while keeping enough AI units commensurate to the difficulty level to try to keep it engaging but not tedious. It's an interesting balancing act no doubt.
My problem with inflation and non-linearities is one of transparency. The human needs to be able to figure out what the actual cost of a particular unit is, and why, and more importantly they need to be able to figure out the marginal cost of any particular decision. Inflation wrecks all this, because it means that if a city has maintenance costs of 6 but inflation is at 50%, then an effect that reduces maintenance costs by 1/3 actually saves you 3 gold per turn, not 2. So it becomes much harder to evaluate whether I should build a market, or a courthouse, for example.
One thing you'll hear me say from time to time is I'm not a big fan of catering to the min/max kind of play, meaning encouraging people to calculate every possible move down to the penny. I prefer to offer intuitive game play. The BUG actual building effect help makes it easy to look at a courthouse or market and see what it will add to the city, which seems min/max'y, but it really makes the choice intuitive without having to pull out a calculator. I honestly don't know if those values include inflation's effect on city maintenance as I've never looked, but I don't think it matters to most players because they aren't making sure every number given to them adds up to the total sum, as long as it gives them the correct 'feel' for what the best decision is, it's working.
I actually want to carry that same idea to unit cost, regardless of wether it's constant or not, so that when selecting a unit, it'll tell you what that additional unit will cost you if it was built today. I've found it a bit annoying to be checking in the early game when you can't necessarily afford any unit costs to have to go back and forth to the finance advisor screen to see how many more you can build before starting to pay for them, and this would alleviate that kind of micro and have even more positive effect if additional unit costs aren't constant (still up in the air).
. I turn it on every once in a while to make sure there are no big problems, as it does affect the pace and the difficulty. I also find that there are plenty of other reasons to play nice with your neighbors 
.
). I've watched the AI drop three different loads without any problems so far, and think I've dodged the issues with previous fixes.
. This follows the vanilla BTS behavior, for every OR tech you have (including the first, which, is a little odd... but that's how it works) you get a 20% research 'mod' towards researching that tech, so if you have BOTH techs like you needed before, you have a 40% mod, so you research it faster.
