AI Techniques.

[Rich1986]

Hello,

I am a professional games AI programmer and I am currently working on a small research project in the area of AI that would control an empire very much like in the civilization.

My questions are what AI techniques are currently been used in the better AI mod and why? And if any what techniques have been tried and failed?

Thanks guys.

 

[jdog5000]

Hmmm ... largely this mod is just adapting and looking for ways to improve the behavior of the existing AI code. There may be lingo for the kinds of things we've done so far, but I don't know it.

A few of the changes have focused on getting the right cities to fill certain needs that the AI feels it has ... the civ city production code is decentralized, each city makes its own decisions. There was already a tiered system of dire needs (don't have any defenders), critical needs (have < 1/2 of defenders needed), serious needs (have < # of defenders needed), and then a normal production selection logic. We've made several changes to take advantage of city specialization, non-critical needs are now more likely (has to be probabilistic ...) to be filled by cities which have some bonuses in producing a particular unit.

We introduced a peace-time budget for troop upgrades so the AI doesn't burn through all its cash upgrading troops. Again this was necessary because the pay for upgrade decision was decentralized but pulled resources out of a shared pool (treasury).

One important class of changes we've made are focused on better modeling of expected utility. The prior utility model for choosing bases for bombers did not consider the full range of the bomber, for example. We also increased coordination between land and air, planes now give a higher weight to bases from which they can support troops in enemy territory (and now also run those support missions).

The largest single class of changes was getting the AI to pull planes and boats out of a threatened city. These units can't directly help defend and will simply be lost if the city is taken. Simply pulling out whenever an enemy shows up is bad behavior though, there needs to be a balance between the risk posed to the base, the risk of the unit leaving the base, and what the unit can do to help protect the base. A contingent of fighters, for example, will stick with the base till the end because the air defense they provide helps the base survive but fighters considered extra (if the city has 5 or something) will leave if the threat is serious. In addition to leaving if things are dangerous, air units also run different missions if they are in threatened cities, focusing on defending the base or weakening the incoming attackers. Situational awareness sounds like a good buzz word for that.

Right now I'm working on adding new behaviors for the AI. Previously, the AI would never pickup units which had no way to get back to any of its cities ... maybe they burned an enemy island outpost, before they would just sit on the empty island for the rest of the game. Not only is this a serious waste of resources but it also effects what the AI builds and does since it believes it has these units at its disposal. Retrieval involves correctly identifying this situation and creating a mechanism for the AI to act to address it.

Also, the AI was previously barred from using assault transport ships during a land war, it would always go over land (the one exception being transports which were full when offensive war was declared). However, the identification of land war was too simplistic, it just required the enemies to be on the same continent but overland paths could be blocked to one or both by third-party territory. Again, allowing the AI to use its ships in a land war involves identifying when they're necessary and coordinating the various unit behaviors.

Hope that was interesting.

 

[glider1]

Hope that was interesting.

Yes it was! There is plenty of research material to explore in this AI and thankfully the AI code is totally accessible. Boy would it have been nice to get some of the state diagrams from the software engineers who conceived it. It seems people like Jdog have had to deconstruct the overarching logic from an endpoint back to a beginning, just by examining the code and using a lot of intuition.
Cheers.

 

[Rich1986]

First of all thanks for the reply.

So a city makes its own decisions. I assume these decisions are influenced by the AI’s current state as well as the city’s state?

If I have assumed correctly then does the AI look ahead in time and work out a strategy that influences the decisions each city will make? Or are the AI’s decisions only influenced by the current game state?

If the AIs decisions are only influenced by the current game state then at the start of a turn or game does the AI pick a strategy according to the civ leader’s personality? Or does the AI not have a strategy that influences any of its decisions?

 

[Yakk]

The AI in Civilization is highly emergent.

Each unit or decision is make with without much of an overall plan.

Units can do things like group (which changes the unit of decision making), or change their AI state (but defender, attack, etc). The AI has a few global states, such as "zone target", "war strategy", "prepare for war", etc.

Other state that is kept track of is an abstract "win vs loss" value for a war, so that an AI is aware if it is crushing someone or being crushed by them, regardless of what it's estimate of each empire's total power.

Now, from this you do get results like "the enemy builds up troops, then goes to war, then crushes a few cities". Or, one AI decides to go to war with a weak person -- and everyone else joins in in a dog pile.

More emergent behavior happens in other resource allocation. The AI sets the value of various possible resources, then maximizes in each given city based on it's priorities.

The Trading AI is yet another component -- the AI keeps track of how much each other empire has done for them in the past, and then uses that to determine what trades to take at the current time.

The AI also uses exhaustive simulations (ie, explores the entire combat tree) to determine the victory chance of a given one-on-one combat. It also does approximations of this for faster situations, when it isn't making a "attack or not" calculation.

In effect, the AI in Civ is a bunch of heuristics and rough calculations. Each decision is made locally, with some relatively minimal global state and priorities to guide it. As Civ is a game where empires do emerge from the decisions of each city and unit, this works reasonably well.

Then there is the city site picker, which is again relatively isolated from the AI's overall strategy.

Players can beat this by macro strategies, and micromanagement that lines up with more macro state. From city specialization (which is pretty poor in the AI), to thinking about maps on a strategic level (choke points, etc), to building invasion forces and defending against invasions on an empire-wide level.

A good way I find of explaining these problems would be playing a copy of 1000 AD for BTS. By force concentration, a Chinese player can beat the Mongolian invasion by the AI. A Mongolian player can beat the Chinese similarly by force concentration. The Spanish human player can take out the Arab forces before the Arab forces fortify in Spain. Etc etc.

On top of that, the AI fails to generate the exponential power explosion of the industrial era, where production grows exponentially by constant re-investment. This is true in the ancient eras as well (lack of city specialization -- a human player will build a unit factory city using heroic epic in a well chosen location, while the AI doesn't have that level of strategy).

Now, how much of this is good or bad, I'm not certain. The AI should both play optimally, and roleplay. In Civ4, AIs have "attitude" towards other players, and treat their friends better than their enemies. This often isn't optimal, except of course because the other AIs also follow it.

 

[glider1]

@Yakk
Cool. Thanks for that assessment!

The computer science techniques used by this AI are very impressive in terms of utilising various smatters of techniques from across the science, to achieve an acceptable simulation of "intelligent" behaviour in a very complex game environment. The AI code is not locked in by one overall approach to this like a chess algorithm is, and the scope for different approaches in different contexts is huge in this beautiful game.

I would love to say that the AI was designed from the top down under a scheme like a "black board AI" which would have suited CIV very well (expert specialisation with central coordination), but it is not so. As a result, this AI could never be used in the real world to solve complex problems for business or planning.

(Note that there are game companies that have built game AI's that they have ported across into industry like Millenium Interactive with the brilliant artificial life simulations in Creatures that was eventually used in defense applications)
http://en.wikipedia.org/wiki/Creatures_(artificial_life_program)

As a project, want to rewrite the AI as a blackboard? (ok I'm kidding I think)
Cheers.

 

[jdog5000]

Yeah, emergent is exactly the term for it. Good one Yakk.

There's a lot of state information passing between the various pieces, especially city production decisions which have special behavior for various diplomatic stances. It's not driven by war/not war, but by how the AI is approaching war on the island/continent that the city is based on ... ie, whether the AI is on the defensive or offensive locally, or whether they're building up troops for a naval invasion, or whether they're not at war. Unit behavior is similarly effected by area war state. It's a pretty good system I think.

 

[glider1]

Is an ant colony governed from the top down? (don't think so?) Most of the state information transferred around the colony from ant to ant is chemical I think. Is this then what the Better BTS AI is like? How adaptable is the BTS AI? How adaptable is an ant colony?
Cheers.

 

[jdog5000]

The BTS is somewhat adaptable, that is why you can add units or civs or whatever through XML and everything works. The fields of the XML tag describe the ant colony chemical message passed around BTS, if you will. There definitely is a similarity in terms of the decentralized approach where each "ant" functions independently as part of a team.

 

[glider1]

Each category of ant has an instinctive coded behaviour (MISSION_REPAIR_HIVE) for example. If you prod a stick into the hive interior, there is apparently no central directive coming from one location that drives all the ants into the process of protecting the queen (THE CAPITAL) in the deep interior of the hive. The central nervous system appears to be nothing more than chemical messages moving from ant to ant.

Yet their decentralised mission based behaviour is breathtaking in it's "appearance of" being a single-minded cooperative determination AS IF there was a general commander in chief MR BIG ANT making all the decisions. They do a great job if you ever tried to prod a hive and are amongst the most spectacularly successful species on the planet. A few hundred million years of AI development is probably not a bad head start.

I will begin to think of the BTS AI in a similar fashion. Honestly, my commonsense tells me that an AI should really be built from the standpoint of having a centralised coordination interior where all strategic directives are issued, but apparently this does not have to be the case according to nature and the builders of the CIV AI.

Cheers.

EDIT: thanks for all your efforts on releasing 0.45!

 

[Yesod]

Sorry to resurrect the thread, but an ant colony may be able to adapt quicker than a system with a centralized concious, but it's limited in the range of it's adaptability. The colony can only perform functions not outside the realm of the complexity of an individual ant's coding. That's why the Civ AI can never perform the complex gambits that a human will. It's simpler to write, but limited, and any serious changes are gonna need to be implemented top-down.

 

[glider1]

Ant colonies and BTS is a really interesting analogy at least to me! Your points are very good. It certainly is an interesting question regarding "adaptability". It would be an interesting exercise if it were possible, to quantify the range of behaviours of a single ant verses the range of behaviours of the entire colony, and see if there is any difference. Is that also applicable to BTS if you were to compare the behaviour of a single warrior to that of a single civilization?

It would be interesting to study the life cycle of individual ant colonies in the eco-system. Perhaps any particular ant-colony will fail because of an inability to adapt to a sudden change just like a BTS civilization suffers. At the scale of an individual colony and an individual ant, your observation about adaptability is really interesting.

However once you scale up the scope of the study and examine the entire ant species in the eco-system, "adaptability" itself seems to come under question. An initial glance of the incredible success of the ant species seems to show that they are perfectly adapted to their environment even if it was a process that took them millions of years. What other adaptation of the ant species would ever be required of it? You could consider them, "perfect".

In some respects, if you think of the BTS AI as an artificial organism, then as a "species" of BTS AI's, it may actually have adapted incredibly well too! After all it has spread and multiplied itself through millions of computers, and has become a permanent feature of the global computer system, like ant's have become a permanent feature of the global ecosystem. What other adaption of the BTS AI would ever be required of it? Despite the individual imperfections in the BTS AI code, could you not consider the artificial organism of BTS as "perfect" too?

Cheers.

 

[Yesod]

I suppose the issue only comes when the human comes into play. In both the real world, and in civ, we stomp on nature. I think for the AI to hang with humans, outside of the bonuses given to it, it needs to be a group of systems with a centralized top-down executive; dualistic like the human is (vast subconcious coupled with a concious mind).

 

[PieceOfMind]

All this talk of ants reminded me that several years ago when I was still playing vanilla Civ IV, I was secretly hoping someone would make an Ant mod of Civ4, sort of inspired by Sim Ant. You'd only need a few basic types of units, and cities could act like different ant holes etc. Different civs would be different ant species or rival colonies (do ants ever fight among their own species?)

 

[glider1]

do ants ever fight among their own species?)

That PieceOfMind, is a good question. I do know that they will defend their colony to the death, but whether they will actually go into an ANT_WARRIOR_OFFENSE mission AI, I dunno. It's a good question! We'd have to ask mother nature

@Yesod
Good point about humans being dualistic. It sure seems that way to me. The big debate in my head is whether there really is any centralised executive at all in us humans in reality.... There may only be an "appearance" of one. This could be a reason why societies that have tried to dictate absolute authority via a central command, don't seem to work at all well in the long run, because the idea is based on an illusion and out of kilter with reality.

There still may be a case for saying that decentralised command AI's like the BTS AI, will be the one's that provide the only "workable" solution in complex AI systems like BTS, even into the distant future. No single computation unit sees the big picture, yet when they all combine, they do something incredibly useful like give us a good game of CIV to waste our time with!

I just dunno the answer to that.
Cheers.

 

[phungus420]

Check this out glider1: Slave Master Ants.

 

[glider1]

@Phungus
Woooow. How cool is that? Incredible stuff. The question now is whether ant "Legionnaires" or other types of ants are able to switch from MISSION_DEFENCE_AI to MISSION_OFFENSE_AI and back, as the situation demands. This would require an appearance of "judgement" as our AI civ's appear to have.

Absolutely for sure is this. If you go outside away from your civ machine even for a minute (sorry to have to do that to anyone), and look at any patch of real mother nature dirt out there, anywhere even in the middle of a city, there is a real game of civ going on down there whether we want to observe it or not. At a guess, it's far more highly tuned and possibly sophisticated too, because evolution has had millions of years to get the behaviour right, while Jdog has had only a few months.

Cheers.