There are a few things, this was a major goal of my mod so I've tested all sorts of combinations
The obvious parts are in the handicap infos xml file, this gives the production bonuses (player & AI) for the various difficulty levels. Unit's use the 'train' modifiers.
The kicker is the hidden 10 free hammers per AI city. That's in the global defines, you can add it to your own GlobalDefinesAlt.xml file, the tag is:
Code:
<Define>
<DefineName>INITIAL_AI_CITY_PRODUCTION</DefineName>
<iDefineIntVal>10</iDefineIntVal>
</Define>
I have it set to '1' in my mod.
Now, if you only change those the AI will be in a world of hurt. So what you really need to do is encourage the AI to behave more intelligently which can be done, to a degree, with a bunch of different XML files.
Each leader has their own defined random chance to build units. In CIV4LeaderheadInfos.xml you'll see a line like this for each leader:
Code:
<iBuildUnitProb>20</iBuildUnitProb>
This means there is a 20% chance (in this case) the the AI will build a unit whether it needs one or not. If you set this to zero the AI will still build units but only when doing so makes more sense to the AI than building something else. For my mod I have this set to zero for most leaders and 10 or 15 for aggressive leaders.
For each unit in CIV4UnitInfos.xml you have a bunch of things the AI cares about. Without making too much work for yourself there are two important ones. The combat strength of the unit (iCombat) and the power rating (iPower). The iCombat value is used in combat so be careful and make sure if you change any of them you make an equal change to all of them. For instance, I went through and doubled the iCombat value for every unit. Now, this means the AI thinks a strength 8 unit is A LOT stronger than a strength 4 unit. So in the default game where you have strength 3 archers and strength 6 crossbows this change would give you strength 6 archers and strength 12 crossbows. The result is that the AI will prefer upgrading when possible because the difference in strength is substantial even though percentage wise the difference is the same as it was before. When the AI upgrades, they don't need to build as many units. The next one is iPower, this is the value used in the in-game power chart, the AI also uses this to determine the strength of the other players in the game. Units are not the only things with power ratings, buildings and technologies also have them. To get the AI to behave better go through all of the units and make the iPower match the iCombat (even if you didn't change the iCombat values). This will improve the AI's behavior with war declarations and deciding who & when to attack.
With those few changes alone you can get away with less AI production bonuses. The weak civs will be a bit weaker than before and if you get your butt handed to you by an aggressive neighbor you can't blame the AI for cheating.
There is a lot more fine-tuning you can do once you see if you like their new behavior. Flavor values for units, buildings and leaders can be tweaked, UnitAI (and NotUnitAI) values can be adjusted for specific units and specific AI weights can be added to units, buildings and technologies. The Civ4 AI deos a lot better than most people give it credit for, with absolutely no bonuses or penalties they can hold their own even though they'll probably still loose in the end. The best part is knowing that they didn't finish that wonder 1 turn before you or outnumber your army 2:1 because they had an extra 10 hammers in that pop 1 city.