If you want to study the AI, use worldbuilder to produce some great spies, infiltrate and observe in normal game.
If you want to study the AI, set "cheatcode = chipotle" in CivilizationIV.ini and press CTRL-Z when you start the game. This does the following:
- reveals map and resources, removes fog of war
- reveals all AI units and cities (I guess this is part of the above)
- allows you to access any AI city (doubleclick as you would with your own cities)
- shows all AIs with all information in the scoreboard. That is, their research and whatever CAN be shown in the scoreboard.
- allows you to access any and all other information. Eg. when you go to Science screen you have a dropdown menu to change leader, thus accessing science screen of any AI
- mousing over the map gives extra information with shift pressed, as well as other information alt pressed. Maybe something more with control or combinations, don't remember. This includes eg.
1) "best improvement" (what automated worker or AI worker would build there),
2) found values including notes of "best found spot" (would be founded onto by AI or would be marked as blue circle for player),
3) AI strategies when mousing over AI territory (eg. Dagger, Cultural, GetBetterUnits, whatever)
4) tile pressures, eg. "cultural pressure" (may cause nearest city to build cultural building if automated or AI city), "danger value" (may cause AI to move troops towards the area to protect it, or garrison nearby cities further)
You can find out a lot about how the AIs work. I used to do this quite a lot during BetterAI development in order to understand WTH was going on. There're loads more debug screens you can access (with which you can do stuff that just might blow up your computer, and I didn't understand but a small fraction of it all - not that I'd be a game developer so not really expecting to either), but I think some information Blake's BetterAI provided isn't available in BTS.
I'd suggest anyone who finds AI to behave in weird way should do:
1) save the game when you suspect weirdness or AI stupidity (the unexpected kind, not the expected kind)
then, when you've done playing whether to win, loss, or abandonment of game,
2) set chipotle,
3) load the suspect saves one by one,
4) press CTRL-Z,
5) inspect what you suspected of weirdness,
6) play a few turns looking for symptoms or problems
7) when you find something you can't explain, take the saves that show the problems and write a report on appropriate forum with saves attached describing whatever you think needs describing
Not that I'd go to all that trouble, but hey - maybe someone will

There are lots of players, so even if just a few people going to the trouble we might find experienced modders interested in tackling the issue (and maybe even The Return Of Blake [cross fingers, knock on wood]).