Improving the AI

[InFlux5]

I'm curious how people think the AI could best be improved. I think most would agree it is sorely lacking - including, probably, the developers, who made up for it by giving the AI extra units, etc. This is a cheap solution in my opinion; better AI programming would be, well, better.

The main problem, I feel, with the AI is that it always thinks long-term when developing the land. By turn 10 or 20 you can be sure the AI has irrigated some grasslands - under Despotism. And it's known that by automating your workers they will build roads to resources you can't even see yet. So obviously when the AI is making its "decisions" it looks at the entire game rather than the current state of things. Even though irrigating grassland in Despotism does nothing, eventually it will. That road leading to nowhere will eventually bring in a resource.

Well this is a huge disadvantage to the AI. It seems to me it wouldn't be hard to program the AI to make its decisions based on the present rather than the future. It should "know" that grassland is not to be irrigated under Despotism, it should maximize the output from the land - at the given time and under the given government. Nothing is more important in this game than production (of shields, food and gold) and workers. For the AI to essentially waste its workers (it's always disappointing to scout the AI's land and see pointless irrigation, etc.) puts it at a huge disadvantage.

After that I would say there should be more of an element of spontaneity and unpredictable behavior from the AI. If you always start out going for Philosophy, like I do, you know that none of the AIs are going to do that. Why? Why don't they know that Philosophy gives a free tech and is, therefore, valuable? Or why don't they even jump to the second or third tier before the basic techs are known? This is clearly not a good strategy, yet it is one that the AI is programmed to use.

Lastly I would say, like many others have, that the AI spreads its units too thin, making military conquest too easy. However, I feel that this is the most forgivable problem. Military strategy is complex and contains so many variables that this is the one area where I think giving the AI the "unfair" advantage makes sense.

But in the other areas I would prefer less "unfair" advantages and more creative programming. I would love to get to Philosophy and find out one of the AI civs already "had that idea", or find that an AI civ cut science and is offering me gold for a tech - things that players do that it wouldn't be hard to program the AI to do.

(Of course, this is why I much prefer multiplayer Civ - or would prefer if it weren't the most bug-ridden lagfest I've ever played in my life.)

So, how would you change the AI?

 

[Seth the Dark]

How about making the AI better with boats.

 

[tR1cKy]

I'm a little pessimistic about the hope of seeing better AIs in the future. Wait... to actually see AIs in the future... those of now (civ3 included) cannot be called intelligent, in fact i prefer the term AS (artificial stupidities).

The problem is not only due to uncreative programmers. A game, to sell copies, must be appealing. So huge amount of money, time and resources are spent on graphics that have little importance to the actual playability of the game, but act like the honey for the bees.

Think about all the leaderheads. Fully animated, era-specific. How much time does it take to realize all that crap? (that, in my opinion, is totally useless - a static leaderhead would be good anyway, since after the 1st week of play almost no one cares about animation anymore).

Obviously more resources spent on useless details means less resources devoted to improving the AI performance, and the result is that one of the best AI of the game existing today is still, in fact, an Artificial Stupidity.

 

[Doc Tsiolkovski]

If you manage to improve the AI even halfway in what you describe, you'll drown in cash. Seriously.
Ok, the AI has a few minor flaws that could be helped. Like attacking Armies, using artillery more frequently in the offensive (they know how to do it in general, it only needs to get higher priority), and considering Government penalties before improving terrain.

But, why do you think a 5-6 year old code, patched again and again, with new features only working by chance (see AA units), rewritten again and again, is still BY FAR the best AI you get in any game? It was an ingenious masterpiece of Soren Johnson, who even didn't make the concept, but herited it.
In 5 years, nobody managed to do something better... five years, that's an era of programming.
And Soren is working on CIV for more than a year now ...

 

[Brain]

I'm curious how people think the AI could best be improved. I think most would agree it is sorely lacking - including, probably, the developers, who made up for it by giving the AI extra units, etc. This is a cheap solution in my opinion; better AI programming would be, well, better.

I'm a programmer myself and I feel that this is very unfair. Programming a good AI system is an unbelevably complex task. Game programmers have discovered long ago that it's not worth even trying to make an AI play like a human. Instead, games will use one of two approaches - scripts or advantages. Civ chose the latter and I think it's a good choice.

The main problem, I feel, with the AI is that it always thinks long-term when developing the land.

You must be kidding! The AI does not have any concept of "long-term". It's a lot more pragmatic than this. In fact, all the automated workers do is look for the most powerful tile in the area and improve it randomly (almost). They just try to balance the number of tiles that have mines and irrigation.

It seems to me it wouldn't be hard to program the AI to make its decisions based on the present rather than the future.

It's WAY harder than you think.

After that I would say there should be more of an element of spontaneity and unpredictable behavior from the AI.

The problem with that is that the AI has no way to know whether their spontaneous behavior was a good idea or not, because there is no learning mechanism in the AI. So if you do this the AI will just try stupid random things most of the time and succede only sometimes. It's possible but extremely difficult to make an AI that learns.

Lastly I would say, like many others have, that the AI spreads its units too thin, making military conquest too easy. However, I feel that this is the most forgivable problem. Military strategy is complex and contains so many variables that this is the one area where I think giving the AI the "unfair" advantage makes sense.

Actually, military strategy is the most "doable" part of all because it involves a lot of tactics. Computers are better at tactics than strategy. To determine where to move the AI essentially makes a decision tree of all its possible moves, then all the moves that the oponent can make, then all moves it can make after that, and so forth. That's one of the reasons why chess AIs are currently the most advanced game AIs.

So, how would you change the AI?

I don't think you understand just how complex this task is. Just to give you an idea, reasearchers have been working on understanding AI since the early days of computing. 60 years later the most advanced AI systems we have are barely able to do some pattern recognition and small adjustments to their behavior based on feedback. It took a decade to send a man on the moon and a comparable amount of time to develop nuclear power. On the other hand, after six decades we are only scratching the surface of AI technology! That's how complicated it is.

 

[Dreadnought]

I agree with brain...many of your points are rather hard to change, especially on making the AI plan for the present

 

[tR1cKy]

Brain, while i substantially agree with what you say, there's at least 1 point that might be discussed better. I'm referring to this:

I'm a programmer myself and I feel that this is very unfair. Programming a good AI system is an unbelevably complex task. Game programmers have discovered long ago that it's not worth even trying to make an AI play like a human. Instead, games will use one of two approaches - scripts or advantages. Civ chose the latter and I think it's a good choice.

I think it's a good choice too, but only because of the ineptitude of the AS. Advantages are needed to counterbalance the incapability of the AS to keep the pace with an human player. While this gap will never be negated, at least in a near future, in the specific case of Civ3 things are made much worse by a crappy programming.

It doesn't need an expert programmer to figure out that, in many cases, the behaviour of the AS is just plain stupid. And this can be only be caused by an inept programming of the AS. Could you explain differently, to say one, the decision of irrigating a grassland under despotism, while a switch to monarchy or republic is 80-100 turns away? Or the absolute inability to assign to the citizens of a city the right tile placement? How many times did you spy on a size 12 city just to discover that the city is wasting 6 food surplus for nothing every turn?

These are only 2 example of moronical behaviour. There are much more:

• the AS spend a significant portion of their income in science even when the beakers outcome isn't enough to research the tech in a number of turn lower than the maximum limit of 40 turns (50 for conquests).
  
• when railroads are available, the AS never build a rail network for the intent of connecting cities. It uses rail to improve the tile outcome instead, even when it's under attack and the connection between cities would provide a significant strategic advantage.
  
• while the AS is perfectly capable to use naval units to pillage enemy tiles, it seems utterly uncapable to drop 4 cavalries on a tile to conquer or destroy an island city protected only by a rifleman. Not considering amphibious assaults...
  
• when a religious AS in in republic and at war for a long time, thus suffering heavy war weariness, it almost never switches back to monarchy, even with only 1 anarchy turn. It stays in war weariness losing huge amounts of gold, research and production every turn. But, if communism is available, the AS switches promptly, being it religious or not.

There are many more examples of moronical behaviour of the AS. I won't list more to not bore people, but countless pages of examples could be written, all of them due to poor / inept / hasty programming. With less emphasis on useless 3D animated models and more on the AI programming, we could be playing a far better game.

 

[SJ Frank]

One of the basic problems is that at the time of game developement, the methods that the programmers thought was the best way to beat the game may not turn out to be good at all. The players, in time, will become better experts at finding strongest strategies in the game.


One solution (that may or may not work) is to open-source the AI scripts. In civ4, you will be allowed to modify the AI scripts, which, I think, is Firaxis' way to testing the waters of that concept.

 

[Breunor]

I think the AI in Civ is VERY, VERY good. I've played many many computer games, and the AI here is objectively bad but VERY good for a computer game.

However, I do think that there are ways to improve it. Remember, unlike the programmers, we have years and literally collectively hundreds of games of retrospect to see what areas to emphasize. So, we knwo which areas are more important to prioritze.

I actually agree that some changes to the AI are far more important than the outstanding bugs to improve the game.

I'm not a programmer -- so I don't know what is or isn't relatively easy. I'm going to be making subjective judgements here.


I agree with Influx that a seemingly simple change would be to make the AI randomly research the next tech, or at least put in some function. Right now, players essentially know where the AI is going, and intentionally reasearch a different tech to then trade it to all of them. The player shouldn't know that all 10 of his/her competitors are researching the same thing and using that benefit. This would be the single most important change I would make and would drastically change the game.

I also agree that it could probably use a better worker rule than irrigate anything. If the computer is going to use OCP, it can probably do better with a 'mine grasslands, hills, and mountains' strategy.

It may be harder, but it may be able to change this if it is out of despotism, but it isn't a horrible rule.

Also, the AI could randomize a beginning of the game settler strategy. It may not always do OCP. Maybe it can randomize between OCP, ICS, and in between. (My guess is that this is very hard to build into the AI, though).

I'm not sure if building more artillery makes sense for the AI because I don't know if the computer really knows how to use it. Why build it if you waste it?

Could the computer build 'suicide' galleys? Probably hard -- I think it would be hard for the programmers to separate scouting with an empty ship from transport, invasion, and other functions -- but if it isn't hard, it would be a nice feature and make playing island maps much more challenging.

It may not be too hard to put in a worker priority to build railroads connecting cities within an empire as a first priority. Even this is probably hard, though, because a Civ may not have connected cities and programming this difference seems very hard to me. I think to make it really work the programmers would have to look at cities connected to the capital (as it already does as part of the luxury use calculation) and railroad them together as a top priority.

Regardless, it should build railroads like crazy and it doesn't always

It should probably build more workers. (This part is easy. determining how and where??)


However, the AI here really is quite good. Anybody remember MOO2? Remember the problem in that game -- there the AI NEVER split up its fleet, and human palyers would simply play hit em where they ain't and win easily. Thus, in Civ, to say it should bunch armies could lead to the same problem. The AI in Civ 3 conquests is MUCH better than MOO2.

Anyway, some of my thoughts.

Best wishes,

Breunor

 

[Brain]

Alright, I will give some thoughts on the subjects that have been brought so far.

First of all, yes, I concede that there are simple improvements that could be done to make the AI better. However, I will play devil's advocate for a moment to show that it isn't as bad as what some say.

On the point of worker tasks. It doesn't really matter whether the AI completes a unit because it is cheaper to produce or because it has maximized its tile usage. The bottom line is how many turns it takes for the AI to build that unit. In fact, the same thing could be said about the use of the luxury slider. In order for the AI to get a good science output you could make the AI utilize its citizens optimally, or just give it a research advantage. In the end, it's only the number of turns to complete a tech that matters. Here I think the programmers were just lazy, rather than incompetent.

About roads and railroads. I agree that AI workers could have a concept of connected cities, but I think the main problem is one of implementation. Connecting a resource is relatively easy because you are just connecting one resource to the nearest city. On the other hand connecting N cities to N-1 other cities is a much more complicated problem. In fact, it is similar to a well known problem in computer science called the Travelling Salesman Problem (TSP). But you see, this problem is very, very CPU expensive. Solving the TSP for more than a few dozen nodes usually requires super-computers to do in an acceptable amount of time. And "acceptable" means something different to a computer scientist than to a gamer.

On the other points mentionned I think it would be hard from a programming perspective to make some of those improvements. In order for the AI to lower its science slider or switch governments or plan coordinated attacks you would need to have some kind of strategic plan, which the current AI lacks completely. The AI doesn't switch governments because of a strategic choice. It only does so because it is programmed to favor some government type over another. It doesn't plan its attacks. It only reacts to state changes, looks at what units it has available, and attacks the weakest target within a certain allowable range. My point is that in order to introduce the changes suggested the AI would have to be significantly re-designed. It may seem simple to a human player, but the work to do behind the scenes is quite involved.

It is true that some things could be done that wouldn't require large changes. Suicide galleys are one of them, I think. However, I still feel that it's not so bad. In other so called "strategy" games there is always a "Silver Bullet" strategy that works every time. It's not that way in Civ. In this game you have to be creative. Even years after the release of the game, people are still inventing new strategies that have never been tried. Just for this I think you ought to give the AI programmers some merit.

 

[Praetorian]

While I would like the AI to be more compasionate, understanding, honest, and not to mention peaceful.... On the other hand it sure is fun going up against it. I am also a programmer and I feel that a wonderful job has been done

 

[wit>trope]

I wish they would make the AI as good as the AI in chessmaster or in poker academy. It seems like they only spend a couple weeks on the AI. Instead of making civ 4 they should just release civ 3 with better AI and no bugs

 

[Calder]

I'd be disappointed if in Civ 4 they haven't at least programmed in the settler factory basics. I think thats one area that can greatly strengthen the A1. There are some basics in Civ3 that rely purely on best ways to develop city tiles during a time frame of city growth, which has been mastered by players who have done the math and have displayed a consistant pattern of brilliant early victories. I'm sure these basics could be programmed, much the same way as basic chess laws are programmed into chess computers. I think there is a selected expansion theory, a method of expansion that if not realised well enough and early enough, can put you behind your opponents. As in chess, there is always a 'best move', and if an A1 programme can consistanly find this it makes it that much harder for us to play against.

 

[tR1cKy]

On the other hand connecting N cities to N-1 other cities is a much more complicated problem. In fact, it is similar to a well known problem in computer science called the Travelling Salesman Problem (TSP). But you see, this problem is very, very CPU expensive. Solving the TSP for more than a few dozen nodes usually requires super-computers to do in an acceptable amount of time. And "acceptable" means something different to a computer scientist than to a gamer.

Did you mean "N cities with N-1 nodes" ?!?
Supposing the answer is "yes", that's exactly the TSP problem, with some extra goodies that don't impact complexity (only because it's already hard enough )
However, an algorythm pooping a near-optimal solution, or even a decent one, could have been easily implemented, if some lazy programmers would have spent more time programming and less time surfing for net p0rn...

 

[Woody]

One thing your forgetting is that every "addition" you give the AI requires more CPU power. They programers have to balance how much time the AI can take for its turn to how long an average player will wait without getting frustrated. Any extra time given to the AI would also be multiplied by how many AI players you have in the game so even a small addition to the AI could use lots of game time.

 

[InFlux5]

Well let me start out by saying that I'm not as ignorant of the difficulties of programming a good AI as it may seem, or as people are assuming. At the same time, I'm not a computer programmer either. I realize that the "AI" in Civ isn't AI at all; and that many people with a lot more knowledge of the complexities of AI than me believe we will never be able to create true AI, which would learn from its experiences and create new algorithms that it wasn't originally programmed with. The Civ AI is just a bunch of algorithms and the assigning of numerical values to different game aspects (techs, units) to determine how valuable they are to the computer civs.

Having said all that, I still think it could be improved. The only thing so far that's made me rethink my original assertions was this:

It doesn't really matter whether the AI completes a unit because it is cheaper to produce or because it has maximized its tile usage. The bottom line is how many turns it takes for the AI to build that unit. ... In the end, it's only the number of turns to complete a tech that matters.

This seems like an important point; time is what matters when it comes to producing units or researching techs.

Now on to the criticisms: I'm not as demanding as it sounds! Is it really that hard to program the AI to think "short term"? If you think so, you must have misunderstood what I meant by short term. Let me use an example.

I have to admit I'm a bit ignorant as to the order/priority the AI gives to connecting resources; but let's say the AI has Iron Working but not Gunpowder. Then let's say the AI builds a road to some saltpeter that it can't actually "see" or use yet, before it connects the iron which it actually can use. This would be a case where thinking "long term" has hurt the AI. How hard would it be to make it so that "If (Gunpowder isn't researched) Then (Don't connect saltpeter yet)"?

Now I don't know if this is actually the way the AI operates when connecting resources, but that's beside the point. You can apply it to any situation where the AI is playing based on the long-term instead of the present. In a general sense it's like "If (I don't have the tech to use X) Then (Don't try to use X)". For the grasslands example, would it really be hard to program "If (Government is despotism) Then (Don't irrigate grassland)". If that's hard to do, then programmers are overpaid.

On the issue of spontaneity and unpredictability:

The problem with that is that the AI has no way to know whether their spontaneous behavior was a good idea or not, because there is no learning mechanism in the AI. So if you do this the AI will just try stupid random things most of the time and succede only sometimes. It's possible but extremely difficult to make an AI that learns.

I didn't mean that the AI is actually spontaneous or random. For it to appear to be behaving spontaneously and unpredictably from the players' point of view doesn't mean that the AI is actually behaving spontaneously. The point is to program it with some algorithms that make it appear spontaneous. (Unpredictable for a player just means they don't know what the AI has been programmed to do. As it is, the AI is very predictable, and that's the problem I have with it.)

For example, take the Philosophy gambit (something players frequently do, because it's a good idea, yet the computer never does.) After founding the capital, the AI could calculate how much commerce it's likely to produce over the next 20-30 turns or so,consider how much commerce it's producing now, then decide whether it might be a good idea to try for the Philosophy gambit. (The 50-turns-at-a-time gambit wouldn't apply any more because there wouldn't be only one civ trying it. Those who play MP know what I mean.) The AI isn't actually "learning" which "spontaneous" behaviors are a good idea. The techs in every game are static. The programmers know what's a good idea or not under certain circumstances, and they program the AI with this information. There's no learning involved.

On the complexity of military strategy, I will have to defer to those more knowledgable about programming. Maybe it is less complex than it seems, but I thought this area would be the hardest. But if it is actually the easiest, then the AI's performance is even more deplorable. Military police aside, why wouldn't the AI be programmed to keep more units in its border cities, to know that cities in the core don't need that much protection? The AI doesn't have to bunch up all its units in one city. How about keeping a mobile force within moving distance of 2-3 cities like players often do?

On the issue of building more artillery even though the computer can't use them properly, I don't know where that came from since I was arguing against giving the computer more units. And on improving the computer's trade routes & road networks, that was also something I never argued for. I realize this is a much taller order. (As I understand it, the long wait between turns on big maps is largely because the computer has to recalculate trade routes for every civ, every turn. I could be wrong about this though.) In general I think the AI does a pretty good job of connecting cities to the trade network. It's ok with me if some smaller cities are left unconnected; even some human players do this.

Returning to the point that time is the important variable, many of these issues are successfully dealt with by giving the AI the "unfair" advantage of extra units or faster tech. I still believe making the AI behave - or appear to behave - more like a human player makes the game more fun and immersive. Also, the quick-and-dirty solutions often have less-than-desireable side effects. And just think if they did program the AI to behave more like a human player, then they added the extra units and shorter research times anyway; the higher difficulty levels would be even more challenging than they already are.

 

[Mongol]

i think, whoever is saying that AI should be imrpoved should just sit down, take a particular apsect of the game that they think should be improved, take a piece of paper and a pen, and try to write down an algorithm of how they wanna do it. u dont need to know any programming language to do it. take, for example, workers' AI - and write down what u want them to do and in what sequence - there is TOO many variables to include and it is not a major aspect of the game at all...
it is much easier to write a pre-defined sequence of actions no matter what...
like today i am fighting enemy AI, having a stack of about 20 radar artillery moving around and redlining city defenders. usually i guard it with an army of 4 MA's, but one turn i forgot to move the army with it and left it guarded by an elite MI only. still that turn they sent a bunch of troops trying to take another city - with all those cavalry and infantry that dided needlessly they could take out the majority of my artillery and stall my attack fot a bunch of turns untill i could produce new artillery and ship it there (i was fighting AI on a different continent).
my point here is that that single event, like my artillery poorly guarded, is something u cant possibly take into account...

 

[tR1cKy]

i think, whoever is saying that AI should be imrpoved should just sit down, take a particular apsect of the game that they think should be improved, take a piece of paper and a pen, and try to write down an algorithm of how they wanna do it. u dont need to know any programming language to do it. take, for example, workers' AI - and write down what u want them to do and in what sequence - there is TOO many variables to include and it is not a major aspect of the game at all...

Apart from the fact that it's not clear what to do with those algorithms once they are written (paper airplanes?), nor how such an activity would benefit this topic, but the performance of the computerized player IS indeed a major aspect of the game. Unless you play only in multiplayer with only human opponents.

it is much easier to write a pre-defined sequence of actions no matter what...

Such thing could have worked well 25 years ago, with games such as Space Invaders.

like today i am fighting enemy AI, having a stack of about 20 radar artillery
...
my point here is that that single event, like my artillery poorly guarded, is something u cant possibly take into account...

You are comparing the poor performance of the AS with a mistake you made during a game... what do those two things have in common?

 

[Doc Tsiolkovski]

I think Mongol's advice is a good one...anyone complaining about poor AI performance should really sit down and try to formulate what the AI actually should do...and that is something completely different from ranting about what it should not do .
A good example for an absolutely reasonable complaint is the fact the C3C AI irrigates Grass in Despo. It seems like the code for AI terrain improvement was re-written, and this one didn't make it in (a simple routine to check for the benefits before starting to improve). Or a "Do not Settle next to Vulcanoes!" warning.
But, demanding a 'smarter' AI for higher levels, well, that's a bit silly...

Mongol: Your second point is a small flaw in AI target selection. The underlying problem is the very same as in the 'AI ignores Armies' issue. Once a target has a too high defence (def value*HPs) compared to the attacker, it won't get attacked at all outside cities. Elite MI are safe from anything but MAs, Elite Inf from anything before Tanks etc.
That is normally a wise concept, and it would be impossible for the AI to recognize if a certain tile is of strategic importance - but, it does not consider if killing this one unit would net the AI Artillery or Slaves.

On the issue of building more artillery even though the computer can't use them properly

The AI uses Artillery exactly the same way the human does. It bombards cities before attacking, units in the open from cities as well as from Artillery in the open etc. Note before C3C, this didn't work properly, since here the AI usually fired their Artillery as last .
But, for C3C all is fine - except for the fact the AI should build about 10x as much of them.
For example, in the Magificient 7+1 game, a well-guarded stack of 4 French Artillery kept besieging one of our towns; if they hit and took off a few HPs from the ELite MIs there, the town was attacked, in a turn without successful bombardment, it was ignored.

 

[tR1cKy]

Doc, this time i disagree. The most annoying glitch in the AS are due to crappy programming rather than insufficient algorithm of the AS. Just as the problem you mention (AS irrigating grassland under despotism). And while deploying a better AI would be a quite challenging task, having the programmers writing better code is obviously something much less complicated. I take again the example you mentioned:

This is what should be done:
[pre]Evaluate mining -> Evaluate irrigation -> Choose the best.[/pre]
And this is what, more or less, happens instead:
[pre]if (grassland) { irrigate }[/pre]
Now is it a problem of pratical AI limitations or a rectal-cranial inversion by some lazy programmer?

If i take a pen and a paper foil and write what the AS should do, i end up with only 4 words: play a little better. And to achieve this result, the first step would be cleaning up some obvious idiocies in the code, then, if the AS is still crappy, let's thing about something to improve its decision tree.