AI improvements "Brave new world"

[Ulthwithian]

Karmah: The question on 'how much time to allow the AI to 'think'' really depends on exactly what kind of AI the devs used on the game. Hopefully, they're using a heuristic of some sort, and you would think that they would use a learning heuristic. While I favor artificial neural networks, they are far from the only option.

JtW: I'm fairly certain that the issue with having the AI use 'player turn time' to do work is that the request for resources is much less predictable, and thus the parallel processing requirements are unpredictable. Certainly, that would be an interesting 'balance point', in that the longer a player takes his turn, the more capable his AI opponents are...

Also, in general, the AI civs all have default 'options' for what they care about... one part where that should be used is 'how they choose to spend their AI time'. E.g., if you are a militaristic Civ like Japan, they should be tough to fight in war but maybe their economy will be weak. Conversely, a pacifist nation like Ethiopia may make the 'stupid tactical AI mistakes' because they have less time devoted to it, but their cities are run better.

I don't know. Until someone gets more information on exactly how the AI is implemented, this is just speculation.

 

[Trias]

Regarding turn times: I think it would be possible to make AI use the time during the player's turn for thinking. Sure, some units might get repositioned and the pure tactics would have to occur during the AI's turn, but couldn't the whole strategic layer be moved to the player's turn (which is taking several minutes more often than not).
Also "peaceful" decisions like which tiles to improve, which cities to convert etc. could be handled during that time.

Wouldn't this be a possible solution? (It's a genuine question. I don't know much about programming, but this is the first thing that comes to mind when I read about how turn times pose a limit on the AI quality).

The problem with using the player's turn to make AI decisions ahead of time is that the circumstance may change between the computation and the actual AI turn. This would require the AI to gauge whether to reconsider some of its decisions when its turn comes. There is no obvious (and computationally cheap) way to do this. (And if you don't look out checking whether to reconsider could cost as much resources as simply reconsidering.)

 

[Ulthwithian]

Trias: Why reconsider, though? You could give the AI a fixed 'budget', if you will (so many cycles), and let it run during player turns.... well, I see how it would like to respond to a changed situation...

In that case, you would do the 'less likely to need updating' AI choices (like, for example, building selections in cities?) during the player turns...

Still, though, the degree of parallelization with uncertain cycle allotments seems like it's an overall bad idea.

 

[Chris Hill]

In dealing with the AI problems, there are a number of issues that need to be considered.

1. How are the programs developed? Civ V seems to use both compiled code, in C++, and scripted code in Lua. Compiled code will be converted to executable code during development and will execute as fast as the processor can, subject to memory and other tasks. Scripted code needs to be converted into a pseudo-code before it can run. That could be every time it is run or, and better, the pseudo-code can be saved. Inherently, pseudo-code will be about 100 times slower than executable code.

2. Multiple threads, i.e. allowing multiple sections of code to operate simultaneously, can be useful but do have restrictions. On a single processor system the threads must have break points, usually during disk access or display updates, when another thread can be run. With Civ it is likely that most AI processing will be memory and processor based, and will not be able to multi-thread on a single processor. Allowing the system to use multiple processors would allow them to genuinely multi-thread.

Both C++ and Lua allow multiple threading. Programming for it is much more complex, especially when you are looking at sharing information between threads. I have looked at CPU usage on my system – i5-2500K, 4 core, 3.3GHz, 8GB memory – and I do not see Civ using multiple processors. Even though Windows/Intel will switch executable code across all 4 processors I never see the total processor CPU go much over 1 processors capacity.

I suspect that you need the programs to be coded to use multiple processors for threads. Given that it is possible for moving units on the map to be moving after the civs updating has been processed, they are certainly using separate threads to do that while working on the next civ.

3. Can you do the calculations in advance? Generally yes, you can – BUT you might need to reconsider depending on what the user, and potentially AI, has just done. At a simple level, you just need to work through each civ, CS and barbs in turn. A more complex system might decide to use multiple threads, one per civ, and effectively run them in parallel. That then again leads to the question of how to deal with changes made elsewhere – defer until the next turn or try to deal with it immediately, which leads into an RTS solution.

The current solution seems to be to allow the user to make his changes, and then loop through the civs with each being dependent on the user and previous AI actions.

4. Using multiple threads, within a single civs AI, to deal with diplomacy, trading, city growth, movement, tactics etc. seems reasonable, again subject to cross-checks between them in deciding exactly what to do. The problem is that we, as users, have an inherent ability to do this analyse and decide process. Writing that in an AI is not a simple process, which ultimately is the cause of the AI problems.

If you are using a Monte-Carlo analysis method, i.e. running the code multiple time which different options, in trying to find the ‘best’ solution, it would be possible to be able to constrain the number of options to be looked at and thus limit the length of time for the AI processes.

I would be looking at running multiple threads, each analysing various options, and coming back with yes/no or % score for the option. The difficult part would be in analysing those to make the best choices for the AI.

Even so, some of the issues with the AI do seem to be purely wrong and should be fairly easy to solve.

 

[Trias]

Trias: Why reconsider, though? You could give the AI a fixed 'budget', if you will (so many cycles), and let it run during player turns.... well, I see how it would like to respond to a changed situation...

In that case, you would do the 'less likely to need updating' AI choices (like, for example, building selections in cities?) during the player turns...

Still, though, the degree of parallelization with uncertain cycle allotments seems like it's an overall bad idea.

The problem is that most AI decisions (especially the more time consuming ones) depend on a large part of the game state. The "simplest" criterion to use, whether a decision needs to be recalculated is to check whether any of the game parameters on which it depended have changed. However given the web of interdependecies this might be true for most decision by the time they are executed, which makes the code behave a lot worse than just waiting for ones turn. (Because it has the additional overhead of tracking changes in the game state.)

You might be able to do something a little smarter, but it is just begging for buggy code.

Even so, some of the issues with the AI do seem to be purely wrong and should be fairly easy to solve.

Solving the problems is not necessarily the biggest problem, properly identifying what the problems are usually is.

 

[Chris Hill]

Solving the problems is not necessarily the biggest problem, properly identifying what the problems are usually is.

True - First identify the problem; Secondly find out where it is; Thirdly identify why the code is wrong; and finally fix the code and resolve the problem.

 

[ahawk]

The developers are often asked about improvements to the AI, but they've said very little about it. Dennis Shirk has made a few vague comments about how the diplomatic AI should be "less schizophrenic" and more amenable to long-term friendships, but nothing concrete. Whenever they are asked about the tactical AI, they usually say that they felt it was a problem in the vanilla release of Civ V, but that they've improved it a lot since then and that they're satisfied with where it is now.

I would expect that most of the AI work in BNW has been to teach the AI how to use the new features. I would not expect significant improvements to the existing problems. I do not expect that until a later patch, and not until after there has been a lot of screaming about it, because I don't think that they believe it is a problem.

The AI can always be improved. Those saying that it's not possible to improve AI without lengthening turn processing times are ignoring history; AI has improved a lot since vanilla without lengthening turn times. Algorithms can always be improved and optimized and made more efficient, especially when they are as complex as these must be.

For the most part, I agree with this. Especially the comment that the vast majority of AI tweaks will be toward how to use new features (World Congress, trade routes, tourism).

That said, there are a few (presumably simple) AI tweaks I sincerely hope they add:
-- As others have said, AI air tactics are non-existent. There needs to be less suicidal air attacks, and more intercept missions (currently AI fighters are used like AI bombers).
-- AI's need to stop sending missionaries, workers, etc. unprotected across hostile territory. It's comical that I sometimes find 4 or 5 captured missionaries surrounding a barb camp or CS. It's downright silly, and a windfall, when I find 2 or 3 workers surrounding a barb camp.
-- Defensive Pacts need to be offered by the AI, at times. Defensive Pacts aren't even a new feature, yet they too need a serious reworking of the AI.

 

[Ulthwithian]

Trias: Hrm. Well, to address the 'native code vs. script' issue that Chris brings up, since the LUA script AFAIK does not change in the middle of a game, that stuff should all be compiled so that cycles can be saved during run time. At worst, though, it should only need the scripting portion to obtain parameter values.

You are completely correct regarding the issue of tracking the game state. That's why I suggested that IF inter-turn AI processing occurred, it should prioritize those areas that are least likely to change with game state. E.g., tech choices, perhaps building choices, etc.

I do think that avoiding this whole issue is a wise move, though.

In terms of how to 'improve' the AI, I have a lot of ideas, as I work in large part in AI modeling, but until it's known how the AI is actually programmed, there's no telling how well in change would 'help'.

 

[Circuit]

Trias: Hrm. Well, to address the 'native code vs. script' issue that Chris brings up, since the LUA script AFAIK does not change in the middle of a game, that stuff should all be compiled so that cycles can be saved during run time. At worst, though, it should only need the scripting portion to obtain parameter values.

You are completely correct regarding the issue of tracking the game state. That's why I suggested that IF inter-turn AI processing occurred, it should prioritize those areas that are least likely to change with game state. E.g., tech choices, perhaps building choices, etc.

I do think that avoiding this whole issue is a wise move, though.

In terms of how to 'improve' the AI, I have a lot of ideas, as I work in large part in AI modeling, but until it's known how the AI is actually programmed, there's no telling how well in change would 'help'.

I think the AI code is available. There are better AI mods being programmed, and I'm fairly certain it is modable. You'd have to poke around in C&C, though; I don't know exactly how to access the code.

 

[Ulthwithian]

Circuit: I wouldn't mind doing this, but having access to the code is not the same thing as understanding how they set the AI up. It'll certainly help.

Anyone who does know how to access the code care to give me a brief orientation?

 

[mitsho]

Better ask here. Or I bet there's a discussion on it already somewhere I can't answer you directly though...

 

[AriochIV]

-- Defensive Pacts need to be offered by the AI, at times. Defensive Pacts aren't even a new feature, yet they too need a serious reworking of the AI.

Except that defensive pacts are useless... in order to be effective, they must deter aggression, but they don't. They just lead to you going to war when you don't want to.

How about Resource trades being offered by the AI? That's a pretty basic one that's been missing from day 1.

 

[Aristos]

JtW: I'm fairly certain that the issue with having the AI use 'player turn time' to do work is that the request for resources is much less predictable, and thus the parallel processing requirements are unpredictable. Certainly, that would be an interesting 'balance point', in that the longer a player takes his turn, the more capable his AI opponents are...

This is being done by Stardock in all its games since Galactic Civilizations 2, and works perfectly well (to the point that many here agree that Stardock AI's are usually at the top of what can be expected given the present technologies and performances). GalCiv 2 is what? 10 years old already? The only two reasons I can think of to "justify" Firaxis' "decision" not to do something similar are" lack of skills, or laziness.

 

[Shadowarriorx]

This is being done by Stardock in all its games since Galactic Civilizations 2, and works perfectly well (to the point that many here agree that Stardock AI's are usually at the top of what can be expected given the present technologies and performances). GalCiv 2 is what? 10 years old already? The only two reasons I can think of to "justify" Firaxis' "decision" not to do something similar are" lack of skills, or laziness.

What I feel a lot of people are missing isn't that the developers don't know what they are doing, but rather putting yourself in the shoes of the developer from a day to day activity.

They don't get to code all day. They don't get to sit around and figure 'one' problem out. Many times they are put under time constraints to get something 'working' and implemented into the game, of which would go through play testing and debugging. They get moved from a half finished project and over to another one. Perhaps somebody changes jobs (or ect.) during the middle of development. You now have a hole in that team to fill, and most likely that new person doesn't have the prior knowledge of how exactly the system is set up so there is a learning curve.

I honestly think that most developers are trying their best to deliver a good working product. But many times, they can only do so much with the amount of time they have. At the end of the day, most of these people have families they want to see.

 

[Aristos]

What I feel a lot of people are missing isn't that the developers don't know what they are doing, but rather putting yourself in the shoes of the developer from a day to day activity.

Your point is valid and that is exactly why I compared them with another developer of similar (4X) games... truth be told, the comparison is a little unfair with Stardock, as when the company developed GalCiv2 using multithreading to make the AI "think" while the human player was using his time, the company had very few employees and developers... far less than what Firaxis has for Civ 5 (and far less budget).

Of course, Brad Wardell did not use his scarce resources to create a million dollar lacking intro video, but to develop what is quintessential to a by-definition single player experience: a challenging AI.

 

[Aristos]

Except that defensive pacts are useless... in order to be effective, they must deter aggression, but they don't. They just lead to you going to war when you don't want to.

How about Resource trades being offered by the AI? That's a pretty basic one that's been missing from day 1.

Not only that. Civ 4 had already implemented a good block of code that made the AI value each trade taking into consideration the environmental conditions at the time of the trade proposal (who has what resource, who needs it and how much, is it strategic or not, etcetc), yet Shafer decided to disregard the already invented and working wheel... it has been demonstrated in another thread that porting that same piece of code to Civ 5 is basically very easy, yet they did not...

Any possible explanation, apart from the apparent stuborness of a yet-to-mature lead designer? Hard to believe, but also hard to think otherwise...

 

[werothegreat]

Except that defensive pacts are useless... in order to be effective, they must deter aggression, but they don't. They just lead to you going to war when you don't want to.

I have actually, in hundreds of gameplay hours, never been offered a defensive pact by the AI.

How about Resource trades being offered by the AI? That's a pretty basic one that's been missing from day 1.

I concur - especially in multiplayer, when the AI won't even bother to ask to renew them.

 

[darko82]

Guys, check out Smart AI mod that improves AI combat.

 

[japanesesamurai]

I’m playing BNW on continents plus on emperor and their grand war tactics aren’t that bad honestly. At war with Portugal atm, Im playing with Shoshones. Portugal is at the other continent. America is on mine, strugling with revolt and severe unhappiness. Inca’s have just one weak capital left.
Anyway, I invaded Portugal in the North, taking ex-Mayan capital first and planning on marching through their land to get Lissabon.
In the middle of the ocean I allied 3 CS islands, which makes a perfect buffer zone. Portugal was unable to get through, so they retreaded.

Now all of a sudden, from the shadows a giant fleet of Portuguese land units assisted by ironclad, frigates and what not, appear in the Northern part, ignoring my allied city states upnorth completely, coming right at me.

So basically, Poetugal is changing tactics: ignoring city states and rapidly trying to attack my weak area.

On top of that, they are not trying to recapture the ex-Mayan city. Instead they’re waiting, fortified, until I invade to get Lissabon.

What I also noticed in this game:
Some city states are smarter (in combat) than others. Some are dumb. Some retreat. Antwerp for example had a couple of privateers lurking in the shadows ready to flank baited Portuguese ships. And they never embark a land unit for fun either. They are so strong.

I have no mods except for artificial unibtelligent light and falcon ai doplomacy.

Maybe on king and down, AI is more stupid?

 

[japanesesamurai]

This is the first time i play on king+ difficulty and the game is way better