How would you design AI?

Putmalk

Deity
Joined
Sep 26, 2010
Messages
2,652
Location
New York
Here is the situation.

Suppose you Civilization V's lead Artifical Intelligence Designer. You currently have eight difficulties (Settler -> Deity) and you are given a new task to design a Hardcore difficulty. You have the following constraints:

- Ignore performance cost on machines.
- The AI is not allowed to receive bonuses that total more than 100% effectiveness over the player. What this means is that if you decide to allocate 100% AI advantage to its growth bonus, they cannot receive any additional advantages
- The AI must not start with any additional technologies
- The AI must be able to beat the standard vanilla deity AI 100% of the time if they played each other.

The goal is to develop strategies and decision making that would make the AI much more efficient - even over its super charged Deity brethren. Toss out any pre-existing strategies and choices the AI makes; you are in control.

Obviously I understand that an AI is an extremely complex algorithm that would be impossible to describe in one post. Choose a few things you feel are the most important thing that the AI could improve upon that would make it capable of defeating the Civ V vanilla AI.


I'll start:

- I think the AI should refuse any obviously exploitative deals that give it unnecessary amounts of happiness for 240G. In turn, these AI would exploit the CiV AI using that deal. In fact, that AI would exploit the Deity AI in everyway a human does already to it. Any monetary advantage is important for these AI.
- This Hardcore AI would conduct a very complex evaluation of every possible move for each one of its combat units (not Scout or Workers) to see where it would be in danger. In addition, each unit has a sight equivalent to its maximum movement radius + its sight (so if a unit can move four tiles and has a 3 sight, it can see up to 7 tiles away from it). This way, the unit can properly evaluate what danger it would be getting into and avoid danger whenever possible. The Hardcore AI would not suicide its units as frequently, and would bait Civ V AI into moving into suicide.
- The AI recognizes the need for science and will do whatever it takes to prioritize getting as many beakers as possible
- Assigns a much higher value for technology like Civil Service, Education, Steel, Astronomy, etc. and will "see into the tech tree" to plan a route that gets its technology + a few necessary techs along the way (like Calendar or Mining for luxury trades)
- Picks one target that needs to die and will actively pursue that target until they are no longer a threat. This can be a close neighbor that it needs for expansion or a civilization that has taken the lead and needs to be stopped. It will negotiate mutual wars or trade embargos (if such a thing existed, rather this would be a series of denouncements) on that target AI.

Let's see what the bright minds of Civfanatics have in their heads.
 
- The AI must be able to beat the standard vanilla deity AI 100% of the time if they played each other.

Not a reasonable requirement with the other constraints - there are plenty of human players who can't beat the AI advantages on Deity (I've never actually played Deity, but I can't beat Emperor or - my current standard difficulty - Immortal 100% of the time, even if not hit by an early rush).

- I think the AI should refuse any obviously exploitative deals that give it unnecessary amounts of happiness for 240G. In turn, these AI would exploit the CiV AI using that deal. In fact, that AI would exploit the Deity AI in everyway a human does already to it. Any monetary advantage is important for these AI.

This in itself isn't particularly important if the AI knows how to spend its money - the current AI can be exploited not because it makes stupid deals so much as because it's given a bonus it isn't using. It shouldn't have 240 spare to spend, at least not regularly. The AI already suffers from inflexibility - "Don't ever do X" is a problematic condition to impose, although admittedly in this case a simple one. Also, if the AI has no advantages, it won't have the excess happiness the current AI does, and so it probably won't often be in a situation where this algorithm would make gold for lux look like a bad deal.

A better option would be:

- Give AI a priority-setting algorithm where gold is concerned, so that instead of "I have gold, therefore if someone wants it for anything I can afford to give it to them", it considers the gold it has in relation to what it wants to spend it on. For instance, if it can enter into an RA, it would refuse a deal that would give it too little money to do so unless something else is higher in its priority list. This would still enable it to trade 240 gold for a luxury, but only if it has nothing else to spend money on, or its happiness is particularly low - and if it has no inherent gold advantage and is spending the money it has, it shouldn't often have the money to do so.

It's more complex, and I'm often vocal in pointing out that AIs need to be given simple rules (see below), but I think it's workable. All it really needs is a list, and the priorities themselves can be context-independent to some degree, or at least basic "If ... then" statements based on relative civ strength (if not tech leader, move science up the priority list. If not military leader, move buying units up the list etc.)

- This Hardcore AI would conduct a very complex evaluation of every possible move for each one of its combat units (not Scout or Workers) to see where it would be in danger. In addition, each unit has a sight equivalent to its maximum movement radius + its sight (so if a unit can move four tiles and has a 3 sight, it can see up to 7 tiles away from it). This way, the unit can properly evaluate what danger it would be getting into and avoid danger whenever possible. The Hardcore AI would not suicide its units as frequently, and would bait Civ V AI into moving into suicide.

All well and good if you want turn times measured in months, but chess AIs have to perform this kind of evaluation, with no random elements, no terrain features, no promotions, and full knowledge of the board, for only 16 pieces, and decent chess AIs have long turn times. All that's really needed to improve AI functioning is some ability to calculate the effects of combined attacks - so that the AI doesn't sit around or retreat with units because individually they can't deal enough damage to safely win a battle, or some might get killed, or wander aimlessly around cities until enough damage has been dealt by artillery for one melee unit to have a chance of taking the city.

Even with the knowledge of the surroundings the AI has, being able to think two turns ahead instead of one would also improve things without any other changes to AI algorithms - post-G&K the AI is so paranoid about losing units that it will always retreat from a superior opponent, whether or not that retreat would put it somewhere it could be killed in the following turn. The result is that the AI loses all the units it would lose anyway (if not more), but does less damage itself.

- The AI recognizes the need for science and will do whatever it takes to prioritize getting as many beakers as possible

It typically does this insofar as it can. However there are key things it's not capable of doing - and producing Great Scientists naturally is one of them (see my suggestions for AI improvements below). Very often AIs I face will have a higher bpt output than I do, but will be hampered by being unable to specialise Great Person production or take the appropriate Wonders at the appropriate time, as well as by making poor and inflexible teching decisions.

- Assigns a much higher value for technology like Civil Service, Education, Steel, Astronomy, etc. and will "see into the tech tree" to plan a route that gets its technology + a few necessary techs along the way (like Calendar or Mining for luxury trades)

Assigning high values to specific techs makes the AI too predictable and hence exploitable, as well as limiting variety between AI civs - certain AI civs prioritise different techs at present. Remove this, and among other things you make it much easier for the player to obtain Wonders that aren't along high-priority tech paths (and to predict which Wonders will be taken when), because all the AIs will reach the same techs in the same order and go for the same Wonders, allowing the player to grab the others (and costing the AIs that don't get those Wonders time and production, again favouring the human). This may also exacerbate an existing flaw, which is the AI's inability to react to changes in situation by reprioritising its tech progression - say, if it's being attacked by Frigates and Ironclads, and it has Caravels and Galleases, it won't make any effort to select naval techs to catch up. Nor will it prioritise Flight if an enemy has an airforce. While it will happily pursue Nuclear Fission with no uranium anywhere in its borders. And no AI civ actively beelines Globalization even if it's aiming for diplomatic victory.

- Picks one target that needs to die and will actively pursue that target until they are no longer a threat. This can be a close neighbor that it needs for expansion or a civilization that has taken the lead and needs to be stopped. It will negotiate mutual wars or trade embargos (if such a thing existed, rather this would be a series of denouncements) on that target AI.

The AI is currently moderately capable of this, but it is difficult to program an algorithm to determine when an enemy is not a threat, and even with the removal of "we're going for the same victory condition", AIs do seem to place higher priority on competing for "same victory" than on simply winning the game regardless of the rival's victory condition. Civs will nuke each other to prevent the other's Utopia Project going ahead, or declare war to prevent their rival from stealing their city-states, but the same civs won't think to nuke the other guy's spaceship because it's not perceived as a threat to a civ going for diplo or culture wins. And the various negatives for being at war, having been at war etc. tend to result in vendettas that outlive their usefulness - AI civs will very often spend valuable time beating an already defeated enemy into the ground because they now hate that civ, instead of focusing on more threatening rivals.

My own suggestions would for the most part be more specifically targeted to specific gameplay limitations rather than broad-brush "make the AI play better", although not necessarily easier to implement:

- Great People. Spy on any AI - it will not have any specialist slots filled, although it will have most specialist buildings in every city. Its GPs will be incidentally generated, either gathered from Wonders or from slow accumulation of GP points, and beyond tending to build Wonders in the capital by default, it won't try to focus its scientist point Wonders, say, in one city. The AI can now use Great People it generates, something it struggled with in vanilla, but as long as it is unable to generate them, it can never compete with a human who produces Great Scientists at a rate of one every 5-10 turns in the late game. I imagine it now knows how to faith-buy them, which probably helps, but the human can do that just as efficiently with more focused intent and superior natural GP generation.

- Evaluate the value of different Wonders in terms of their own strategy and game position, at least crudely. Some AIs will just grab any Wonder going, and watch how often a civ will build the Porcelain Tower and never enter into any research agreements. The science victory AI still works exactly as it did in vanilla - which means once Apollo is complete the civ will start building spaceship parts, and will never, ever build Hubble. Half the time Stonehenge is taken by civs that never make any other attempt to gain faith. The only real exception is that the cultural victory AIs will often hoard the cultural Wonders where possible, but even there there's no apparent priority setting regarding which to go for when - if it provides culture, take it (and the Oracle actually appears to be relatively low priority for culture civs, which appear to be programmed just to maximise culture output, not to recognise the value of free policies).

- Specialise towards a particular victory condition and keep aiming for it until achieved or impossible. This is admittedly inflexible, however I've run into a lot of recent cases where a civ will simultaneously be building spaceship parts while racing for the UN, or grabbing every CS going - cultural or otherwise - while heading for the Utopia Project. This tends to waste resources and time; as I've noted, no AI civ beelines to the UN, because they tend to go down the science tech route first. Plenty of civs build Apollo and never build any spaceship components; often the rest decide to abandon spaceship production altogether once the UN is built (or at random otherwise). And I've seen the reverse - the AI giving up a city-state race that they could have used at least to stall my diplo victory because they've decided to try building a spaceship instead. The AI launches fewer irrelevant wars than it did in vanilla, but this too should be curtailed - if a civ is at war and aiming to take a city, just holding that city can keep them distracted for the rest of the game. In one duel map game (Immortal), Wu was in a position to be competitive for science victory, but she just focused exclusively on failed invasions of a city I'd founded to access uranium - not even a capital that could have won her domination. In another Immortal game, Genghis and Nebuchadnezzar spent the entire late game trading Nippur with each other. Nippur wasn't a capital, Nebuchadnezzar wasn't in a position to shoot for any victory condition (while Genghis was doing well until that point in progressing towards diplo victory), and I was the lead rival for victory (and friend to both of the others). That was also the game where Genghis essentially conceded diplo victory to me while he focused on starting a spaceship from scratch. If he's going to go for a spaceship, and the UN is already out, he needs to be programmed to realise that he can only complete his spaceship by preventing me winning diplo victory. Instead, one of my votes actually came from ... Genghis Khan (because by that stage only three civs survived, and since Genghis and Nebuchadnezzar were at permanent war, I got both their votes by default).

- Achievable, but demanding development time wise: program the AI with an understanding of UAs, so that it will play in a way that maximises its benefits from being Civ X. At present each civ is programmed to favour one of a small number of preset strategies more often than the alternatives, but Babylon doesn't pursue its science strategy any differently from Korea, say (and naturally neither uses specialists). Genghis still prefers diplo victory despite his tendency to conquer city-states, in spite of changes to the diplo victory condition that cause conquered CSes to lose their votes. On the one hand, there's an argument that the AI should be improved across the board before fiddling with Civ-specific stuff; however it is likely to be a lot easier to program an AI to succeed by making it able to exploit its advantages rather than by training it in strategy. It amounts to giving it free bonuses by a back door route. And however much you improve the combat AI, Attila's going to lose as long as he thinks a Battering Ram should behave like a Spearman.
 
I have to nitpick here:

All well and good if you want turn times measured in months, but chess AIs have to perform this kind of evaluation, with no random elements, no terrain features, no promotions, and full knowledge of the board, for only 16 pieces, and decent chess AIs have long turn times.

Putmalk said:
Ignore performance cost on machines.
 
Ignore performance cost on machines.

I wouldn't design AI that ignores performance cost. That's pretty stupid. You'd think of an algorithm that would theoretically play very well, by just evaluating all possible moves for a given search depth. Too bad with a board the size of civ it would probably take a thousand years even if you went just 10 turns deep. This isn't useful at all.
 
...I guess you guys don't understand the concept of thought experiment.

Oh well, I'll just leave this alone then.
 
...I guess you guys don't understand the concept of thought experiment.

Oh well, I'll just leave this alone then.

Perhaps it's more a case of struggling to see the point of a thought experiment that can be summarised as:

"Assume we remove all constraints on an AI that prevent it playing like a human. How would you improve it? My idea: make it play like a human".

Sounds like a thought experiment into which no thought has been put...
 
Perhaps it's more a case of struggling to see the point of a thought experiment that can be summarised as:

"Assume we remove all constraints on an AI that prevent it playing like a human. How would you improve it? My idea: make it play like a human".

Sounds like a thought experiment into which no thought has been put...

Ouch son.


So I've been wondering along the lines of removing processor constraints on the AI.

Are there any computer science nerds who have loaded CiV onto something like a Cray supercomputer? I'd like to know what it's like to play on a mega map with max graphics, max civs, max City States on marathon.
 
Hey, I've got one, since we're talking about ignoring hardware costs, to develop a better AI. Ignore software development costs, too! Wouldn't gamers gladly pay $999.99 for Civ6, if it had a better AI? :crazyeye:
 
Are there any computer science nerds who have loaded CiV onto something like a Cray supercomputer? I'd like to know what it's like to play on a mega map with max graphics, max civs, max City States on marathon.

Given how much it cost just to get to use a part of a supercomputer, no.
 
Ouch son.


So I've been wondering along the lines of removing processor constraints on the AI.

Are there any computer science nerds who have loaded CiV onto something like a Cray supercomputer? I'd like to know what it's like to play on a mega map with max graphics, max civs, max City States on marathon.

I think at some point you run get into issue with how the software was coded. Won't be able to take advantage of the supercomputing clusters.

The ones I work on don't even have graphics cards :)
 
I think at some point you run get into issue with how the software was coded. Won't be able to take advantage of the supercomputing clusters.

The ones I work on don't even have graphics cards :)

But, do they really need one?
 
TL;DR:
Simplify the AI choices to 3-5 and the AI will have a 1 in 5 chance of picking the correct decision.

A) The tweaks need not come from the AI crunching thousands upon thousands of choices. It should come from simplifying its choices so it can make the correct ones.

AI should have 3 victory flavors, 1 flavor is chosen at random by every AI civ at the beginning of the game.

- diplomacy (befriend all city-states and make alliances with trading partners, build culture)

- science (out-tech everyone in a race to apollo program, build science, fielding a small but technologically advanced army capable of dealing massive damage to outdated ones)

- expansionist (eat land for 60% of the world domination, fielding massive but outdated army to conquer more land, typically warmongering)

- The AI will go to war (or cold war/espionage) if those victory conditions are being impeded.

all the other choices the AI is to make is in the service of those 3 victories and if it keeps the AI on track for the victory conditions. That also means the AI has to have pre-set build orders, priority buildings, etc. for those conditions.



B) AI's been called stupid because it doesn't defend cities well enough nor can't prevent being overrun by a larger neighbor. well here's a simplification of choice that would work for the AI.

1) Have line of sight for AI (assumes AI can only see part of the map: sphere of influence + fog of war similar to the humans)

2) AI will check units in line of sight near borders and up to 3 hexes away, is number of melee/range above X? if so change over city production from AI victory flavor to accelerated war production and/or rushbuy. Status on Guarded.

3) if AI has tall empire, call on allies for a round of denouncements to discourage from DOW. If possible, have pact of friendship drag allies to war or face a very harsh reputation penalty (this is better than a warmonger rep penalty i believe for many reasons).



C) AI can't use 1upt. Well 1upt has to be altered so AI can be on equal footing. Make it 2upt for melee and workers only, range is only 1 upt.

1) Load melee units into Army ala Civ3.

AI respects the HP so more HP means they'll put them on the front lines more and stop leading with range.

2) Range has less HP than Melee.

The AI will stop leading with range because they're weak.

3) Range has longer hex arc of fire but weaker effect

Range can fire up to 4 hexes away but with weaker effect.

4) Range cannot fire on close proximity.

Range must have a distance of at least 2 hexes away from its target at all times. It's all in the service of putting melee in front of range so the AI stops leading with it.

5) Range damages any unit in a stack worse due to collateral damages (not applicable to cities - cannot stack in a city)

It increases range units' firepower against melee without increasing their actual HP creating a balance and encouraging the AI to do hit and run tactics with range as its supposed to do.



D) AI uses espionage a lot.

1) Spy, sabotage, steal tech, steal maps, steal troop movements

AI will benefit greatly if they know the other players (including the Human's) movements. Also this info can be added to the check borders, switch to war production decision portion of the AI's check into the borders (dependent on intrigue and what the spy discovers like 'your spy found out that Atilla is launching a sneak attack against another civilization. your spy did not find out who the civ is')

2) AI evaluation of other Civs before going to war

Using espionage helps the AI gauge strength in taking on another Civ or even the human.



2 cents etc etc.
 
The AI in Civ V is always going to get bonuses. Call it cheating or whatever, but its how difficulty levels work in the Civ series. What Firaxis needs to do is to not have this "cheating" be so blatantly obvious. Local Happiness allows for this to happen, while global happiness holds up a big sign that clubs you over the head with which Civ has the most happy faces. The game goes so far as to have popups reminding you how much the AI is cheating. This is horrible game design.

Chasing victory conditions, while not an AI flaw, is also horribly designed when it comes to culture victories. With an entire continent available to an isolated AI that hasn't bothered to expand past 3 cities, a player isn't going to think "Hey, Montzuma's going for a cultural victory! Smart!" Instead, he's going to think "this is the worst AI I've ever seen. What's wrong with this game? Why doesn't the AI expand?"

It's the little things... :crazyeye:
 
The AI in Civ V is always going to get bonuses. Call it cheating or whatever, but its how difficulty levels work in the Civ series. What Firaxis needs to do is to not have this "cheating" be so blatantly obvious. Local Happiness allows for this to happen, while global happiness holds up a big sign that clubs you over the head with which Civ has the most happy faces. The game goes so far as to have popups reminding you how much the AI is cheating. This is horrible game design.

The pop ups serve mostly to point out that the AI isn't cheating - it's a giant sign saying "look, we gave the AI massive bonuses, and it isn't making any use of them whatsoever when it could have a massive number of high-pop cities by now!" Same with the excess gold you see in the diplo screen.
 
The pop ups serve mostly to point out that the AI isn't cheating - it's a giant sign saying "look, we gave the AI massive bonuses, and it isn't making any use of them whatsoever when it could have a massive number of high-pop cities by now!" Same with the excess gold you see in the diplo screen.

Again though - if that's Firaxis' true intention, the end result is just that the player thinks this is a poorly designed game, where even when an AI is "playing to win," they're still being very inefficient, which takes away from user immersion. At least it does for me.

You can't really trumpet "playing to win" on the part of the AI as a major selling point, and then allow things like this to happen. Have them siphon off that excess gold in an offshore account, so at the very least, even if they do have millions of gold, they don't look like complete buffoons.
 
C) AI can't use 1upt. Well 1upt has to be altered so AI can be on equal footing. Make it 2upt for melee and workers only, range is only 1 upt.

1) Load melee units into Army ala Civ3.

AI respects the HP so more HP means they'll put them on the front lines more and stop leading with range.

2) Range has less HP than Melee.

The AI will stop leading with range because they're weak.

3) Range has longer hex arc of fire but weaker effect

Range can fire up to 4 hexes away but with weaker effect.

4) Range cannot fire on close proximity.

Range must have a distance of at least 2 hexes away from its target at all times. It's all in the service of putting melee in front of range so the AI stops leading with it.

5) Range damages any unit in a stack worse due to collateral damages (not applicable to cities - cannot stack in a city)

It increases range units' firepower against melee without increasing their actual HP creating a balance and encouraging the AI to do hit and run tactics with range as its supposed to do.

I like this idea, a modder should test this and see how the AI does in combat situations. I bet there will be marked improvement.

I miss armies in Civ.
 
Again though - if that's Firaxis' true intention, the end result is just that the player thinks this is a poorly designed game, where even when an AI is "playing to win," they're still being very inefficient, which takes away from user immersion. At least it does for me.

You can't really trumpet "playing to win" on the part of the AI as a major selling point, and then allow things like this to happen. Have them siphon off that excess gold in an offshore account, so at the very least, even if they do have millions of gold, they don't look like complete buffoons.

Well, that's my point - if the AI was programmed to use its advantages it wouldn't appear to be "cheating". Yes, it is poor to design an AI that can't use the advantages it's given. It's not poor design to give the AI those advantages, nor is it a fault of the game engine.
 
Top Bottom