AI seems to not be doing a good job of prehistoric research.

[Arkenor]

OK, I've gotten a full AI log from turn 1 to turn 485 shortly before I get Sedentary Lifestyle. Playing on Emperor, I'm well ahead of all the Civs I've met, though some are certainly doing better than the (lower difficulty setting) game which inspired this thread. I'm impressed I managed to go 485 turns without crashing!

The log file is kind of huge, but it compressed pretty well. I hope it's useful.

 

[Koshling]

OK, I've gotten a full AI log from turn 1 to turn 485 shortly before I get Sedentary Lifestyle. Playing on Emperor, I'm well ahead of all the Civs I've met. I'm impressed I managed to go that long without crashing!

The log file is kind of huge, but it compressed pretty well. I hope it's useful.

Thanks, ill take a look at it later today and see what I can figure out.

 

[Koshling]

Okay, here are the files for the game I mentioned above. The first save is the turn after I finished Sedentary Lifestyle. I then played five turns with AI logging on, and attached the log and savegame file from that point.

One quick note - I could have gotten Sedentary Lifestyle a few turns earlier, but detoured to Earth Oven and Slash & Burn to be able to take Agriculture as the free tech.

Hope it helps.

The AIs all seem to have built the right things, done ok at hunting, and have sensible civics (and civic change history). Two things struck me however:

1) How fast YOU had managed to get the Sedentary Lifestyle. You have researched a total of 80 techs in getting there, in 163 turns. That's averaging a tech roughly every 2 turns which strikes me as unusually fast (difficulty immortal, epic speed,which is faster than I am used to, but even so). As such I'm wondering if something is off that has allowed you to somehow get a large boost. Anyone regularly play on immortal/epic able to confirm that a tech every 2 turns throughout prehistoric is an unusual average?

2) The techs the AIs have indicate they have researched a few side-branches that they don't need on the way to Sedentary Lifestyle. To some degree this is ok, but they all seem to have fixations with certain techs they probably shouldn't have (see new post which is a reply to the full AI log from the entire prehistoric period posted by Arkenor, which I'll be posting shortly)

In regard to (1), I notice you're playing with flexible difficulty. What was the GAME setting (not the BUG setting) for difficulty when you started? Looking at the code there is arguably a bug with the BUG Flexible difficulty in that it NEVER adjusts the difficulty by more than 1 step every N turns EVEN if your current difficulty is outside of the range specified in the BUG settings. Thus (for example) you could start a game at SETTLER, and then turn flexible difficulty on with a min/max range of immortal->deity and you'd still stay at settler for 50 turns, then increase by ONE for the next 50 turns and so on, only reaching the MINIMUM stated in the BUG settings after 350 turns. It is therefore possible you played the start of this game at a lower difficulty than you thought you did (which would explain how you were able to research so fast)

 

[Koshling]

OK, I've gotten a full AI log from turn 1 to turn 485 shortly before I get Sedentary Lifestyle. Playing on Emperor, I'm well ahead of all the Civs I've met, though some are certainly doing better than the (lower difficulty setting) game which inspired this thread. I'm impressed I managed to go 485 turns without crashing!

The log file is kind of huge, but it compressed pretty well. I hope it's useful.

The main thing that stands out in this log is that the tech path taken by the AIs is a bit sub-optimal - in particular they beeline certain things they probably shouldn't (or at least shouldn't usually). Taking one at random (Lower Sioux) this is the tech path they followed to the end of the logged turns (some are beelines, so the choice is actually a sub-path that includes techs on the way to the target):

Shelter Building (seems ok)
Hard hammer percussion (seems ok since it enabled stone tool workshop)
Oral Tradition (fine)
Persistence hunting (ok)
Tracking (debatable, but not terrible)
Fire (I'd say not ideal, but again not terrible)
Naturopathy (IMO too soon)
Cultural Identity (good choice, but should have been earlier I think)
Natural Pigments
Slavery (why now? - this is quite a deep beeline)
Trap Fishing
Ceremonial Burial (probably after religions? Maybe ok but certainly debatable)
Tattoos
Wood Working
Carpentry
Animal Riding (the AIs all LOVE this because so many units depend on it)
Boat Fishing (ok choice, but Sedentary lifestyle definitely looks better in most circumstances)

The things that stand out (for me, but I'd welcome other opinions) are:

1. Cultural Identity (aka Elder Councils) should have been sooner
2. Religions seem to be driving the tech path a bit too much (Naturopathy, Ceremonial Burial), but maybe this is sensible, since its a first-gets there gets the benefit thing (and Naturopathy enabled jungle camps which can be significant too)
3. The units enabled by animal riding are too distorting. They will still be there when it is researched slightly later, and anyway they need other enablers which I suspect is not being taken into account when making this choice
4. The free tech from Sedentary lifestyle is being under-valued (based on checking how the code values it), probably grossly so (like it makes the tech 10% more attractive rather than 100% more)

I'll continue to look into the specifics of why Naturopathy, Ceremonial Burial, Slavery, and Animal Riding rate so high as beeline targets, but I'd also appreciate any comments and observations on the above path that anyone else has...

Edit

...ok, so a more detailed analysis turned up a number of bugs and issues as follows:

• Free techs are being under-valued as suspected
• Buildings which require the tech being evaluated have their value considered even if they also require ANOTHER tech we don't have - this causes Hypocratic Oath to be included for Naturopathy (it requires Naturopathy AND Ancient Medicine). Since that's a later era Wonder the value is very distorting
• Enabled World Wonders are being a bit over-valued I think - normal building evaluation counts them double their effect-value because of the opportunity-denial gained by preventing anyone else building it, but this is far less appropriate when the decision being made is whether to research the enabling tech rather than to actually build the Wonder itself. I will reduce this multiplier when evaluating for the purpose of tech path choice
• Buildings with free promotions are being over-valued due to a bug in the combat mod AI code, though this is not significant currently since units mostly do not have subcombat types (but will be significant when they do, so I will fix it)
• Buildings which give a production bonus by unit combat type are being BADLY mis-valued when evaluated as part of tech evaluation, since during tech evaluation they count all units of that combat type, whether they can currently be built or not, rather than just if they cannot currently be built because they need the tech being evaluated. This massively over-values the Riding School because it counts in ALL future mounted units!

 

[Arkenor]

The main thing that stands out in this log is that the tech path taken by the AIs is a bit sub-optimal - in particular they beeline certain things they probably shouldn't (or at least shouldn't usually). Taking one at random (Lower Sioux) this is the tech path they followed to the end of the logged turns (some are beelines, so the choice is actually a sub-path that includes techs on the way to the target):

Shelter Building (seems ok)
Hard hammer percussion (seems ok since it enabled stone tool workshop)
Oral Tradition (fine)
Persistence hunting (ok)
Tracking (debatable, but not terrible)
Fire (I'd say not ideal, but again not terrible)
Naturopathy (IMO too soon)
Cultural Identity (good choice, but should have been earlier I think)
Natural Pigments
Slavery (why now? - this is quite a deep beeline)
Trap Fishing
Ceremonial Burial (probably after religions? Maybe ok but certainly debatable)
Tattoos
Wood Working
Carpentry
Animal Riding (the AIs all LOVE this because so many units depend on it)
Boat Fishing (ok choice, but Sedentary lifestyle definitely looks better in most circumstances)

The things that stand out (for me, but I'd welcome other opinions) are:

1. Cultural Identity (aka Elder Councils) should have been sooner
2. Religions seem to be driving the tech path a bit too much (Naturopathy, Ceremonial Burial), but maybe this is sensible, since its a first-gets there gets the benefit thing (and Naturopathy enabled jungle camps which can be significant too)
3. The units enabled by animal riding are too distorting. They will still be there when it is researched slightly later, and anyway they need other enablers which I suspect is not being taken into account when making this choice
4. The free tech from Sedentary lifestyle is being under-valued (based on checking how the code values it), probably grossly so (like it makes the tech 10% more attractive rather than 100% more)

I'll continue to look into the specifics of why Naturopathy, Ceremonial Burial, Slavery, and Animal Riding rate so high as beeline targets, but I'd also appreciate any comments and observations on the above path that anyone else has...

The Lower Sioux are neighbours of mine, and doing comparatively well. The Canadians seem to be the biggest threat to me on my continent, but haven't overtaken me yet. I'd be interested to see what you think of how the Romans did, as they really are just sitting there with 1 city still at turn 613. I think they just got tribalism a couple of turns back. It's just us four nations on this continent, so not sure how everyone else is doing.

It's turn 613. Druidism and Shamanism have been founded somewhere. Tengruism has not. I'm 12 turns from Mesopotamism.

I've not been bothering the AIs at all. No fights, or stealth antics, apart from dealing with the rogues they send to annoy me. I daresay they've been harassing eachother though, but they're a decent distance apart so they should have had time to get established by that point.

I tend to beeline for hunting and naturopathy myself, so I don't blame the AI for trying to do the same. The myths from hunting do wonderful things for your early science rate, and you start with such high health penalties that you really need some health tech to get growing properly.

Once the AI can build more cities, they tend to start catching up pretty well on Emperor level. And then speed right past, usually, but that's a whole different problem! So the time they take to get to Tribalism is critical for their future chances. Rome, most likely, is shortly going to be a quite nice addition to my Empire, if one of the other two don't devour him first.

 

[Lenneth]

Actually, it doesn't matter if the AI knows how to tech to SL faster if it gets crippled by thiefs and rogues the way they do.

I don't think that Tracking is debatable if you can enable trackers. I always tech it before Cultural Identity unless I get unlucky with my first chasers. Trackers are what gets the animal pile going towards your capital for beakers and production.

Animal riding can wait. Thiefs should not.

Naturopathy is meh. Health techs don't matter early on. Contrary to almost all mods, focusing on prodution early is better (and starting next to a 3h tile is the best).

What is important: good hunting action, the stone tool workshop (and some techs that give it bonuses), the elder council, the bandit's hideout for money and thiefs and rogues to slow down the others, the Venus wonder and the wonder that grants the golden age. Then beeline SL getting some health and food to grow a little and build a couple of tribes.

It would be nice as well if the AI knew the power of choking and made some attrition war with it's accumulated stuff instead of allowing humans to get away with a low number of military units (like just one tribal guardian and one archer unit).

 

[Johnny Canuck]

In regard to (1), I notice you're playing with flexible difficulty. What was the GAME setting (not the BUG setting) for difficulty when you started? Looking at the code there is arguably a bug with the BUG Flexible difficulty in that it NEVER adjusts the difficulty by more than 1 step every N turns EVEN if your current difficulty is outside of the range specified in the BUG settings. Thus (for example) you could start a game at SETTLER, and then turn flexible difficulty on with a min/max range of immortal->deity and you'd still stay at settler for 50 turns, then increase by ONE for the next 50 turns and so on, only reaching the MINIMUM stated in the BUG settings after 350 turns. It is therefore possible you played the start of this game at a lower difficulty than you thought you did (which would explain how you were able to research so fast)

I didn't think I was playing with Flexible Difficulty, but had enabled Increasing Difficulty when setting up the game. I know I started at Immortal, though - I always pick it because it is the highest setting in which the AI only starts with one tribe. I also remember the message on turn 65 that the difficulty had been increased.

 

[Johnny Canuck]

Shelter Building (seems ok)
Hard hammer percussion (seems ok since it enabled stone tool workshop)
Oral Tradition (fine)
Persistence hunting (ok)
Tracking (debatable, but not terrible)
Fire (I'd say not ideal, but again not terrible)
Naturopathy (IMO too soon)
Cultural Identity (good choice, but should have been earlier I think)
Natural Pigments
Slavery (why now? - this is quite a deep beeline)
Trap Fishing
Ceremonial Burial (probably after religions? Maybe ok but certainly debatable)
Tattoos
Wood Working
Carpentry
Animal Riding (the AIs all LOVE this because so many units depend on it)
Boat Fishing (ok choice, but Sedentary lifestyle definitely looks better in most circumstances)

A couple of things:

- Shelter Building IMHO is too early. For me, I only research Shelter Building & its prereqs after Tracking, Oral Language, Hard Hammer Percussion, & usually 4 of the post-HHP techs that give +1 . The above can nearly double my output right off the bat, which is far better than anything Shelter Building gives right away.
- Fire should only be researched if you intend to research Cooking immediately thereafter, & preferably already have Herbalism for Wild Herbs (the best early / building).
- Agree on Cultural Identity being earlier.
- IMHO, there's no need to research Slavery until you need it as you make the final push towards Sedentary Lifestyle (i.e. I never research it before Tribalism).
- I've yet to encounter a situation where I would prefer Boat Fishing to Sedentary Lifestyle.

 

[Dancing Hoskuld]

I too aim for Tracking and Oral Tradition before all else. The Myth buildings make a bigger impact the earlier you can get them.

 

[Thunderbrd]

Buildings with free promotions are being over-valued due to a bug in the combat mod AI code, though this is not significant currently since units mostly do not have subcombat types (but will be significant when they do, so I will fix it)

Sorry about that. I believe I did ask you to review that part as I wasn't sure if I had maintained the correct intentions there. I made an attempt but it obviously threw things off a bit.

Without mastering the AI side of the dll modding, which I think only you really have a full grasp of the meaning of various values there, all the rest of us can do is make a clumsy attempt to blend in our projects. Since you have such an intimate knowledge of the real meaning of the math in these evaluations, I would simply ask that you keep a very close eye on any attempts the rest of us make to add our own values anywhere throughout the ai.

I personally HATE working on AI (where you've established the methods there anyhow) because I know I'm struggling to fully grasp the intentions of the existing calculations that stand as pre-existing examples and I always suspect I'm doing something wrong with it. Trying to read and comprehend THAT much math makes my head spin and I'm sure I misread those equations a lot.

I've made some attempts (that I feel are extraordinarily mathematically clumsy) that you've requested I make at some of the new unit/promo tags in my next update to come that I'd like you to take a very close look at when it gets committed. There appears to be SO much work left to do in that segment... we've only begun to break the surface of a major project there if I'm understanding it correctly.


One observation in reading this thread, only one of many I suppose, would be that the AI seems to be a little overly concerned about its starting unhealth situation and seeking immediately to resolve it despite not being able to until a number of steps are taken. Thus, they go straight for Shelter Building to get rid of Homelessness unhealth even though that's a lot of investment for nothing that will be effective until other measures can be taken to overcome the unhealth gap. Far more important to get the research up before taking that route so that it can be traversed faster and at a time when it is more possible for the reduced unhealth to have any benefit (a little later, the food output begins to make reducing the unhealth potentially valuable.)

It's also probably reading too much into all the buildings it opens up despite the majority of those buildings then requiring other techs (as you mentioned.)

Very interesting point on the Riding School btw. The value of opening up a bonus to XP or free promos to a particular CC shouldn't be as much about a flat count of how many units can use it so much as it is about the percentage of the amount of units you currently have access to (or will at that tech) that will benefit from them. Otherwise, all those UUs that COULD come from cultures you'll probably never have get thrown into the evaluation mix and... ugh. I'm sure you're thinking along the same lines but I figured I'd just state my opinion there for reflection.

 

[Arkenor]

As it's Emperor diff, all AI started with Language, Gathering, Scavenging, and Nomadic Lifestyle, which would influence its initial decision as to what tech to go for first, and give it some stuff stuff to be working on building while it sorts out the next techs.

Shelter Building doesn't seem that bad an idea to me. The AI only needs weaving and cave-dwelling to get there. If you have grass in the vicinity, you get a point of extra production from the grass houses that'll get auto-built. Does the AI also consider the usefulness of the techs on the way to shelter building? If it had caves in range, then picking up cave dwelling en route also makes sense.

Edit: Actually, I'll go look. I know where the Lower Sioux live. They have grass. Nearest caves are in the 3rd ring. I guess that's too far out.

 

[Johnny Canuck]

Here is another save game file, with associated AI logs (I had it turned on for the entire time). It was the same settings as the last game (Large/Epic, started at Immortal & up to Diety at turn 65), and I was the first to Sedentary Lifestyle again on turn 163.

Koshling - You asked earlier about my quick teching. Looking back, it might have been helped by Alexander's Megalomaniac trait, which gives +5% . Pretty nice for a 'negative' trait.

 

[Koshling]

As it's Emperor diff, all AI started with Language, Gathering, Scavenging, and Nomadic Lifestyle, which would influence its initial decision as to what tech to go for first, and give it some stuff stuff to be working on building while it sorts out the next techs.

Shelter Building doesn't seem that bad an idea to me. The AI only needs weaving and cave-dwelling to get there. If you have grass in the vicinity, you get a point of extra production from the grass houses that'll get auto-built. Does the AI also consider the usefulness of the techs on the way to shelter building? If it had caves in range, then picking up cave dwelling en route also makes sense.

Edit: Actually, I'll go look. I know where the Lower Sioux live. They have grass. Nearest caves are in the 3rd ring. I guess that's too far out.

The AI does consider the tech son the way to something it decides to beeline - the beeline decision is made on the total achieved value/turn invested of the entire sub-path

Here is another save game file, with associated AI logs (I had it turned on for the entire time). It was the same settings as the last game (Large/Epic, started at Immortal & up to Diety at turn 65), and I was the first to Sedentary Lifestyle again on turn 163.

Koshling - You asked earlier about my quick teching. Looking back, it might have been helped by Alexander's Megalomaniac trait, which gives +5% . Pretty nice for a 'negative' trait.

Ok. It didn't seem like a 5% thing to me, it just seemed very fast. However, I don't have much personal experience of that gam speed so it might just be my unfamiliarity with it, and 2 turns per tech on immortal/deity might be entirely normal on the optimal path. Anyway, thanks for the complete log - it's more useful information to analyze.

 

[JosEPh_II]

Yes you can get early techs in 2 turns on Epic or Normal Gamespeed if you've focused on research before Sed Life. 160+ turns to Sed Life is relatively normal for these 2 speeds.

If it bothers you to get tech this fast, go to a slower game speed like Marathon, Snail, or the slower levels below these 2. On normal or epic speed at game start having 600 yrs/turn lopped off and getting a tech every 1200 yrs is not too fast for number of turns for each of these game speeds.

Once you've figured out the Optimal research path then Yes you can get to Sed Life 90% of the time 1st before an AI on Noble, Prince, Monarch, and the next level up. Deity I'm not familiar with.

If you would slow the tech research rate down then by Industrial Era you would only be researchin in the Med Era techs. ls612 has done a Good job on the Gamespeed adjustments. And think about it, there are over 500+ techs to research. Normal has 1000 turns and Epic was 2000. Eternity has 14000!

JosEPh

 

[Koshling]

Just pushed a first batch of fixes (see SVN thread).

In the course of this testing I have also noticed that the AI gets itself into quite nasty financial trouble around the time it settles its 3rd of 4th city, and doesn't handle it terribly well. I will be making changes to address that over the next couple of days.

 

[Arkenor]

Just pushed a first batch of fixes (see SVN thread).

In the course of this testing I have also noticed that the AI gets itself into quite nasty financial trouble around the time it settles its 3rd of 4th city, and doesn't handle it terribly well. I will be making changes to address that over the next couple of days.

Is it getting mangled by crime penalties to gold? By that point they're all rogueing eachother, and from the cities I've conquered, they seem to make some unwise choices regarding crime-raising buildings too.

 

[Koshling]

Is it getting mangled by crime penalties to gold? By that point they're all rogueing eachother, and from the cities I've conquered, they seem to make some unwise choices regarding crime-raising buildings too.

Not really. It's more that they are getting bogged down in unit building requests when they should be building some economic buildings. Just the priorities seem to be incorrectly tuned - probably just needs more sensitivity to financial difficulties in adjusting the priorities, and possibly pay more attention to the extra maintenance costs of founding an extra city.

 

[JosEPh_II]

Just pushed a first batch of fixes (see SVN thread).

In the course of this testing I have also noticed that the AI gets itself into quite nasty financial trouble around the time it settles its 3rd of 4th city, and doesn't handle it terribly well. I will be making changes to address that over the next couple of days.

As it does for the player too. After the 4th city gold flat out disappears and it's just about the time (If you have Barb World active) for stacks of Barb Atls, Stone Axe, and Hand Rams to show up at you newest city's border. You start furiously teching for techs that give gold producing Bldgs and for the techs that open up the next level for any and all the Civic choices.

JosEPh

 

[Dancing Hoskuld]

As it does for the player too. After the 4th city gold flat out disappears and it's just about the time (If you have Barb World active) for stacks of Barb Atls, Stone Axe, and Hand Rams to show up at you newest city's border. You start furiously teching for techs that give gold producing Bldgs and for the techs that open up the next level for any and all the Civic choices.

JosEPh

With this I agree.

 

[Lenneth]

I would like to ask, even if this thread is about the pre historic era, if anybody thinks that archery should be a bigger priority for the AI. They do seem to take a while to get it after SL.

It's really easy to conquer Atls with city raider archers even without surround or diminishing city defenses. (And should archers really be able to get all city raider promos? I also think that archers are too cheap for the time, but that's another matter...)