The future of AI

I encountered an interesting article tonight, on the topic of AI in games. The basic premise of the article (and upcoming articles in the series) is that while a focus on AI has been predicted as the next big thing several times this century, it hasn't really happened yet. And the series looks at why, and also why the author thinks it may be time for that to change.

As Civilization fanatics, this seems relevant. The AI in the Civ series really hasn't improved that much since 2001, the starting point for the article's timeline, at least in terms of the vanilla game. Civ4's AI isn't that much better than Civ3's before community mods like the Better AI mod are added in, and Civ5's AI is a step back from both Civ3 and Civ4. And yet, Civ5 may be the most commercially successful iteration of the series yet.

For longtime series veterans, AI becomes increasingly important to keeping the game interesting. While, as the article alludes to, the AI can appear more intelligent by giving bonuses (such as cranking it up to Deity level), after playing Civ enough you realize that it's using the same poor tactics, it just has production and economic bonuses. And at least for me, that takes away from the fun after long enough.

So I'm curious where this article will go, why the author thinks AI might get more of a focus now, and whether strategy games will be featured in the series. And perhaps most importantly, whether Firaxis agrees with the premise of it being the right time for good AI and will put a significantly improved AI in Civ6. As one of the most-criticized aspects of Civ5, my hope is they'll take the criticism to heart and deliver the best AI in the series... but the fact that the AI hasn't improved much over the years also makes me suspect Firaxis may not be the company to bet on for engaging AI in games.

Thoughts on the article? On AI in games in general? Whether there's a good way for companies to show off their AI so the investments in it will return more than their cost and a good AI will result in a net profit? I do feel like the latter is a challenge, as it's often only quite a bit of gameplay that you can notice the shortcomings of the AI. Civ3's AI certainly seemed a lot better in my first game when I was losing on the easiest difficulty than it does now!

For me any in game "intelligence" which is not my own intelligence is a major problem. I will explain myself: I have always played single player games and been happy with it, however after tasting multiplayer games (for instance in ARMA series) i found AI predictable and boring, so i stopped playing games where AI had a major role in favor of multiplayer games. Of course i didnt need too much time to get tired of the antics, stupidity and childish ways of human players too so i totally stopped playing most games i previously enjoyed so much.

The AI in a lot of games hasn't really improved :/ Partly because, from my understanding, AI is usually worked on near the end of development so things get rushed, but it doesn't help when they don't have any dedicated AI people either (Creative Assembly I am glaring daggers at you).

The Total War series AI is just as as it was over a decade ago. Occasionally it is a bit less , but wow playing the free weekend for Rome 2 (do NOT buy, what a steaming pile of disappointment) I swear it has actually gotten worse. The only thing it hasn't done is charge it's general headlong into my men right at the beginning of the battle and it has actually gone after units I put off to the side to flank them, but other than that jesus christ on a flaming pogo stick is it AWFUL. It *ALWAYS* makes its men run across the map to my units so they are exhausted and the only reason I lost any battles was that the enemy had more and better troops than I did (because starting off Carthage with nothing while Syracuse has their actual proper army makes so much sense, along with Syracuse looking exactly the same as my other village in Sicily).

I'm so sick of bad AI. I know it isn't easy to program and develop, but come on. It's like the #1 thing your fans have been asking for literally a decade and you can't be arsed to actually invest in it. I still remember them bragging about how they would have *one* dedicated dev working on the AI for EMpire Total War, which was obviously a lie since the game released with the AI only half-implemented and completely unable to do a number of things like move troops over water.

Maniacal said:

I still remember them bragging about how they would have *one* dedicated dev working on the AI for EMpire Total War, which was obviously a lie since the game released with the AI only half-implemented and completely unable to do a number of things like move troops over water.

Click to expand...

No it's entirely possible that they did have one dedicated AI dev, it's just sounds like they were underestimating how incredibly difficult programming a good AI is. I believe it's the hardest part of game development, and any strategy game that's aiming for a good AI should have a team of at least several people working on it.

When I was doing the AI paper in my software engineering degree, our tutor showed us a video that I think was about the history of AI. It showed that even going right back to the start of AI programming, people were thinking we'd have human intelligence levels of AI within the next decade.

Personally, I'd love to see more games use neural network based AIs. Neural networks basically allow an AI to "learn", which I think would be exceptionally interesting for strategy games.

The thing is though that they barely even programmed the AI at all for Empire. In their defense (not really though) they whole game was released far too early (it wasn't even close ot being finished, they did the same thing again with Rome 2, though not *quite* as bad) so they probably didn't have time to finish it, but they did make a big deal about how the AI would be a lot better than in Rome 1 and Medieval 2.

It is still to this day more retaded and sometimes outright broken than the AI in any of the other TW games.

It's not super difficult to program a good AI or at the very least a set of scripts in relations to events, its just that it takes a lot of balancing and play-testing to make it feel just right and not cheap or dumb. Studios are allergic to QA nowadays though.

I'm not sure I'll be buying anymore strategy titles until the devs decide to really make AI a priority. As long as everyone else keeps getting suckered in to pre-ordering "OOH SHINY," we'll still keep getting the gilded fluff that is becoming the norm. Oh well.

The only advantage an AI has over a human is speed. So they seem to do ok in zerg like RTS games and suffer in deeper thinking games like civ. Supposedly the AI in offworld trading company is quite good.

I don't expect it to get a lot better cus there are so many variables in a game as complex as civ and computers can't think abstractly. Despite all the sci fi movies that intimate otherwise I don't expect computers to ever be able to think abstractly either, they will just get quicker and be able to process more variables and situations to come up with more, better objective solutions faster.

Interesting post. My 50 pence:

I actually think the best AI I have ever come across is Half Lifes (the original – although HL2 was by no means bad). Before half life, FPSs were basically like a shooting gallery, whereupon thousands and thousands of monsters/troops would pour themselves onto your machine gun fire, or just camp where they are and snipe you off in a cheap manner. Half life was different. I vividly remember running away from some soldiers and hiding in a room, only for 2 grenades to come sailing in through the open door. That was the moment I was genuinely impressed. Soldiers would also seem to use team work and run from cover to cover, which back in 1998 was like a revelation. Half life also managed to inject some staged sequences in amongst all of this to make it appear that the AI was thinking for itself, even though it wasn’t and was scripted. The biggest thing here is that those events did not feel tacked on or as cheap tricks. They genuinely felt real. Namely:

1. The soldier throwing a Det Pack into the pipe as you are crawling in and a fireball coming back towards you.
2. You crawling through a vent, hearing soldiers at the other end, then when you are half way to the exit they shoot the vent out and you fall into the room.
3. Soldiers abseiling down vents after you.
4. Soldiers abseiling out of helicopters/troop transports.
5. Etc

The point is that Half life had superb level design, probably the best there has ever been in gaming. It also had capable (not revolutionary) AI. Packaged together this made the AI seem better than it was.

I think this is where Civ 4 is better (it is better IMO, and I don’t care how many more units 5 has sold than 4) than civ 5, and it centres almost entirely on the design of the actual game (in that Civ 4 is much better designed and balanced than civ 5 is, which is horribly skewed towards human play rather than computer play). In civ 5, the bonuses the AI gets at higher difficulties are more than the bonuses the AI got in civ 4 at the same level. This is because 5 is more complicated for the AI, so to make up for its stupidity, it needs bigger bonuses. Civ 4 was a big leap forward in terms of AI because it was the first iteration (I believe) where the AI didn’t out right cheat. But its design was superior to 5, and IMO this is all down to 1upt.

Anyway, what I mean by the above is that design is still king (not AI). And anyone who says otherwise is kidding themselves. I think this is where Rome II and the TW series in general has gone wrong. Making the game too complicated so that the AI cannot play it is a recipe for disaster. Sure it might be cool to include meaningful elevation and stealth. But what use is that if it cant cope with everything else?

I have heard that the AI in galactic civilizations is pretty decent, but cant comment directly as I have never played them.

I do hold out hope for the future of AI, but I don’t think it is the linchpin in gaming – game design and level design are. In strategy games it usually boils down to balance and how capable the AI is in formulating and executing winning strategies.

Thrawn said:

Personally, I'd love to see more games use neural network based AIs. Neural networks basically allow an AI to "learn", which I think would be exceptionally interesting for strategy games.

Click to expand...

Machine learning video game AIs have a bit of a reputation for being unpredictable, time-consuming, and very difficult to comprehensively test and debug. Neural networks perhaps more so than any others. That's a lot of the reason developers often rely heavily on scripted behavior. It gives the developers a degree of control over the user experience they don't get with machine learning.

I think the article makes a very good distinction though that many gamers say this game has good AI when really it just has well designed, challenging gameplay. The complaints in the civ series are usually just that, the AI is too easy to exploit here or there, basically game's too easy.

The AI in civ4 could be extremely difficult to beat if it was simply programmed to break diplomatic rules and exploit techs. Those are the two ways the player beats the AI. The two main ones anyway, also being more efficient/specialized with economy is the third but that's more minor.

The AI will never in civ4 research a tech while evaluating their trade returns for it and they also irrationally value techs strictly based on beaker levels plus a bit more for flavor. For example you might as a human research a tech no one has like education then trade it for lesser techs like gunpowder but then on the same turn fetch 3-4 other techs of less value bulb wise. The AI will never make those trades cus each one individually they lose beakers on but as a whole a human just tripled their return on beakers invested researching one tech.

The other major point is at a certain level of friendliness AIs will never attack you and will always do or not do certain things diplomatically. This is the other major area to exploit them. Share a religion, give a couple gifts and bam, you have an ally for life, despite how easily they could wipe you out. This however I don't think should be changed. They tried making an unpredictable diplomatic AI in the first iteration of civ5 and it failed horribly cus you never knew what they would do. It just doesn't fit the civ game design.

So some tweaks would work others wouldn't, but sherbz said it comes down to overall game design more than a super intelligent AI that can do everything.

coanda said:

Machine learning video game AIs have a bit of a reputation for being unpredictable, time-consuming, and very difficult to comprehensively test and debug. Neural networks perhaps more so than any others. That's a lot of the reason developers often rely heavily on scripted behavior. It gives the developers a degree of control over the user experience they don't get with machine learning.

Click to expand...

Oh for sure. Probably the best use of a neural network AI in games at the moment is for the developers to run it a lot of times in private, to generate that scripted behaviour. I know this is something that was mentioned in my AI paper as being possible, although I'm not sure if any developers are actually doing it.

You assume that run of the mill developers are competent enough to that. They are not. They can barely handle multi-threading. Half of the studios are powered by interns who butcher pretty much everything they touch, be it audio, video or programming.

"AI" in games isn't really Artificial Intelligence - it's just a bunch of preprogrammed rules that tell the computer how to behave.

It hasn't really been getting better for many reasons - most budget increases to a game production these days go to the creation of graphics and other content. Why would game development studios put any extra money into the development of more intelligent "AI" - when gamers accept the substandard "AI" that they dish out in every single game?

If you wanted true AI in games, that would mean a lot more time and effort put towards this in the development effort. It would be expensive. Why risk it when you can go with a methodology that works well enough? You go with well known path finding algorithms, you make "AI" harder by giving the human player handicaps, and so on - these are well known paradigms that work well - and are relatively cheap. Why reinvent the wheel so that you can maybe have "true AI" or whatever? Most game dev studios will not go down that road - they'd prefer to put their money in things that average players will notice right away - the graphics.

I don't agree with them, but they know what they do. We've accepted the substandard "AI" they dish out, so they're going to continue with it. Why wouldn't they? The alternative is too expensive.

Mind you I don't think this precludes some developers from attempting new forms of "AI" - but they'll just probably never work towards true AI. It's just too much hassle. Instead they're going to improve a rule here and there - game balance issues - not any sort of modifications to to underlying "AI" or how it operates - a bunch of if/then/else rules that tell the computer how to behave.

warpus said:

"AI" in games isn't really Artificial Intelligence - it's just a bunch of preprogrammed rules that tell the computer how to behave.

It hasn't really been getting better for many reasons - most budget increases to a game production these days go to the creation of graphics and other content. Why would game development studios put any extra money into the development of more intelligent "AI" - when gamers accept the substandard "AI" that they dish out in every single game?

If you wanted true AI in games, that would mean a lot more time and effort put towards this in the development effort. It would be expensive. Why risk it when you can go with a methodology that works well enough? You go with well known path finding algorithms, you make "AI" harder by giving the human player handicaps, and so on - these are well known paradigms that work well - and are relatively cheap. Why reinvent the wheel so that you can maybe have "true AI" or whatever? Most game dev studios will not go down that road - they'd prefer to put their money in things that average players will notice right away - the graphics.

I don't agree with them, but they know what they do. We've accepted the substandard "AI" they dish out, so they're going to continue with it. Why wouldn't they? The alternative is too expensive.

Mind you I don't think this precludes some developers from attempting new forms of "AI" - but they'll just probably never work towards true AI. It's just too much hassle. Instead they're going to improve a rule here and there - game balance issues - not any sort of modifications to to underlying "AI" or how it operates - a bunch of if/then/else rules that tell the computer how to behave.

Click to expand...

I mostly agree here, but I think there is one area that a true AI might actually be useful and is also one that we have not really seen yet in a final state. It is also one that game developers will have to someday invest some time and effort in otherwise their game will actually be sort of rubbish.

In MMOs, it actually makes sense to have a dynamic AI intelligence. I know the comparison is a bit retro, but I would liken it to the role of the dungeon master in D & D. If you had two dungeon masters in control of the same game – one was brilliant and the other one crap. They are both using the same rules, the same text, the same monsters, the same everything. But, and it is a big but, there is an enormous amount of potential on the side for them to influence the game in a positive way. Put simply, a dungeon master who was in command of all the rules, of all the nuances, and was adept enough to exploit those in a meaningful and enjoyable way, they were the gold dust in D & D. They could make an average game great, and a great game unforgettable.

AI in this sense actually becomes quite relevant. Because a human is not necessarily the best person to make those sort of judgement calls, especially in a vast world like an MMO. Granted, the parameters will be set by humans, but it will be an AI that pulls all those strings. Examples could be:

1. Monitoring a commodity market so that it can create an effective system of supply and demand.
2. Noticing the relative power of certain factions and throwing obstacles in their way.
3. Spawning computer controlled units in areas of the game that come under attack from massive numbers of human controlled players.
4. Alerting other human players of the actions of different factions in game world.
5. Using scripted encounters matched to party makeup/alignment.
6. Use of scripted events during large scale engagements to increase immersion.

An AI is best placed to make those sorts of calls because it could involve hundreds of variables. And I would expect that in the future the intelligences on offer will be able to approach the game world in a dynamic way so that they can evolve and offer diagnostics or something after a particular event.

You have sort of seen the beginnings of this in WOW. But in that case its very much developer led – Blizzard set off some event and then watch what happens. This also works in a relatively small environment. But in a huge game that’s open world, with millions and millions of players, it doesn’t, and could be rather chaotic or unbalanced if managed by humans because they couldn’t keep tabs on the action that was going on at once in a concentrated event spanning a couple of hours (say a large battle). I think this is where games like Elder Scrolls online have got it wrong. The world is too small and too scripted (the developer wants to retain a measure of control). They would do better to make it much larger and then turn control of interactions over to an AI.

You can also see the potential for such technology in a game like Elite, which is simply enormous. And with the inclusion of over 100 billion star systems in game, only an AI could possibly keep track of everything that was going on (IMO Elite is too big and what AI there is, it isn’t invasive enough, it just dishes out news bulletins about the various goings on, but that’s another issue).

warpus said:

"AI" in games isn't really Artificial Intelligence - it's just a bunch of preprogrammed rules that tell the computer how to behave.

Click to expand...

While I understand exactly what you're going for, I just wanted to point out that many kinds of intelligence, including the kind that most organisms have, is nothing more than rudimentary "if this, then this" programming. Basically instincts.

What needs to happen is the introduction of actual learning - alteration of instinctual behaviors based on prior experience:

1. Habituation - learning to ignore stimuli that result in no significant consequences. Example - the last three times I sent my entire army to chase down that terribly threatening drone, I actually didn't find any additional units. In the future I'll just send a couple fast units. Or perhaps ignore it completely.

2. Conditioning - learning to associate novel stimuli with instinctual responses.
Example - the last three times I sent my entire army to chase down that terribly threatening drone, I actually didn't find any additional units. In fact, while my army was chasing the scary drone, a bunch of enemies showed up in my base and trashed my infrastructure. I'm going to associated random drones that show up and run away with an impending attack.

3. Trial and Error learning - learning to associate certain behaviors with increased reward, and others with decreased reward.
Example - this could be incorporated into a longer-term adaptation strategy for playing against a particular player (spanning multiple games). If the player always turtles, then the AI will be punished for sending lots of early light strike forces, and rewarded for focusing on growth early on and then sending overwhelming siege units later.

etc.

The ultimate kind of learning, insight learning (basically learning from observation, "the easy way," without having to actually experience something, AKA "the hard way), is definitely very far away, but the above types should be attainable in the near future.


For now, I'm not sure any of our games show any kind of learning behaviors. What they lack is even reasonable instinctual behaviors. That is what is so unengaging.

Malachi256 said:

For now, I'm not sure any of our games show any kind of learning behaviors. What they lack is even reasonable instinctual behaviors. That is what is so unengaging.

Click to expand...

And that's what can be seen in civ4. The opponents never respond to what the player is doing, but the player sees what they are doing and can counter in a myriad of ways. For instance, you check their research to see how far they are from feudalism and then decide to axe rush em or something. Or you see they can't possibly get liberalism so you tech it to one turn of completion, then research everything else so you can pop the most expensive tech with it. The AI will never do that.

Although I'm not 100% sure I want them too... it might be too difficult! It would be nice if they responded to your types of unit stacks and techs though in war.

As far as mmo's, interesting idea cus right now mmo's probably have the absolute worst AI of all games. Everything is completely scripted and zero aiming/manuevers etc since it's all die roll based stuff. I mean you never see a creep kiting you in warcraft lol. Boss fights are 100% scripted and really it's all just a gear check and learning the scripts. It's the dirty secret behind wow that you're really just collecting shinies and not doing anything but playing simon says basically.

Malachi256 said:

While I understand exactly what you're going for, I just wanted to point out that many kinds of intelligence, including the kind that most organisms have, is nothing more than rudimentary "if this, then this" programming. Basically instincts.

What needs to happen is the introduction of actual learning - alteration of instinctual behaviors based on prior experience:

1. Habituation - learning to ignore stimuli that result in no significant consequences. Example - the last three times I sent my entire army to chase down that terribly threatening drone, I actually didn't find any additional units. In the future I'll just send a couple fast units. Or perhaps ignore it completely.

2. Conditioning - learning to associate novel stimuli with instinctual responses.
Example - the last three times I sent my entire army to chase down that terribly threatening drone, I actually didn't find any additional units. In fact, while my army was chasing the scary drone, a bunch of enemies showed up in my base and trashed my infrastructure. I'm going to associated random drones that show up and run away with an impending attack.

3. Trial and Error learning - learning to associate certain behaviors with increased reward, and others with decreased reward.
Example - this could be incorporated into a longer-term adaptation strategy for playing against a particular player (spanning multiple games). If the player always turtles, then the AI will be punished for sending lots of early light strike forces, and rewarded for focusing on growth early on and then sending overwhelming siege units later.

etc.

The ultimate kind of learning, insight learning (basically learning from observation, "the easy way," without having to actually experience something, AKA "the hard way), is definitely very far away, but the above types should be attainable in the near future.


For now, I'm not sure any of our games show any kind of learning behaviors. What they lack is even reasonable instinctual behaviors. That is what is so unengaging.

Click to expand...

See im not sure that’s the best way of going about creating a good AI. And that goes back to what I was saying about game design. It is perfectly possible with todays technology (and there are examples of it) to make AI’s play a certain way. So you can get an AI to turtle; to aggressively expand; to play defensive; to explore; to build; to research etc. The problem is that in todays strategy market it has not been effectively implemented. Perhaps the best example of it is actually Endless Legend, and that is necessitated upon because all the factions are so different. Flavours of behaviour in civ 5 basically boiled down to +2 or -2 in one of a number of personality traits (war, economy, wonders etc). The trouble is that all of this happened in secret and the player has no way of knowing one way or the other. None of the civilisations were distinct either. Sure they had different bonuses, but essentially they are all the same (certainly when the AI plays as them). What would have been more interesting is if a bonus was given to an AI and they mercilessly tried to exploit it every game. In order to do that and not break the game, you have to have near flawless game design. Otherwise something is too over powered. Civ 5 doesn’t have that balance, so went for lots and lots of civs instead that had some minor bonuses. Examples of great balance in strategy games would probably have to go to (in no particular order):

1. Starcraft franchise
2. Warcraft 3 reign of Chaos
3. Most Paradox titles (due mostly to the fact that it has staged imbalances, which form part of the challenge)
4. Sins of a Solar Empire

I think the mark of a great strategy game is where the developers, upon the first patch, do not have to reinvent the wheel in order to make the game work. This is where they went horribly wrong in BE. I mean seriously, your first affinity unit was basically like marines vs Longbowmen. Hence why people (like myself) beat Apollo difficulty on only their third playthrough. Is the AI bad in BE? Well, its no worse than civ 5s. But that is by no means an endorsement of civ 5s AI. Which I actually think is quite poor. Does it have bad game design? Absolutely yes! And this throws the AI in an even worse light because you notice it more and can exploit it more.

civvver said:

As far as mmo's, interesting idea cus right now mmo's probably have the absolute worst AI of all games.

Click to expand...

Exactly! And conversely they are also the type of game that actually needs a true AI more than other types of games because it has to deal with multiple human players. Your options are basically script it to hell (WOW), or try something else. But to do that something else you would need an AI. And not an AI that you actually see. It would work in the background and act like an old school DM.

Maybe the thing to do, would be to have the AI first.

By that I mean tailor the game around the kinds of things the particular AI can handle well (or efficiently).

Seems to me that one problem with these games if you start with a base game, then develop an AI that can "play" that game.

Maybe you should go the other way, and find what the AI does well, then implement game mechanics around that.

In the end all this stuff about cities and farms and windmills or whatnot is just flavor anyway.

The human mind will kern it, and make it fit, if they play long enough at a game.

Malachi256 said:

While I understand exactly what you're going for, I just wanted to point out that many kinds of intelligence, including the kind that most organisms have, is nothing more than rudimentary "if this, then this" programming. Basically instincts.

Click to expand...

Yeah. True AI would be able to learn as it goes, probably using a neural net or something similar to accomplish this.

Games don't use this as a rule because it would be way too expensive to set up, I'm guessing, instead of going with the tried tested and true.

It's unfortunate that we call what's used in games "AI", because it isn't artificial intelligence at all.