[BTS] Development thread

[jdog5000]

glider:

The function which has the bug Yakk's talking about is CvTeamAI::AI_getAttitude. As Yakk said, it does an average over master and vassal relations using just the Furious, Annoyed, etc categories and not the underlying +- relationship. That would tend to mute the really positive or negative effects, producing more middling relations. It certainly sounds to me that like played a role in Napoleon's decision, checking out the saves now.

The AI also does not currently consider how popular its potential victim is ... it does consider how powerful a vassal's master is or how powerful a defensive pact bloc is, but it doesn't consider the friends of the potential enemy who might come to its defense. That's a tough calculation for the AI to make, because it's an unknown future decision by the third parties. On average, it's also likely that other players will join the aggressor's side of a war.

I don't have an answer for why Catherine was in so many pacts ... I haven't changed anything there directly.

 

[glider1]

That's a tough calculation for the AI to make, because it's an unknown future decision by the third parties.

Jdog thanks. This hopefully will be some fun on a coding level! Getting the AI to wear a suit and tie and conduct some diplomacy that is at least reasonably coherent in an intercontinental context. It really is the sweet interior of the cake.

Here is a save from another critical moment in that wonderful simultaneous intercontinental invasion by Napoleon and Saladin. Saladin carries on the invasion from the island to Ham's mainland. On the very next turn Ham capitulates and Saladin accepts, which triggers off the whole Napoleon debarcle and demise down the track.

Honestly, the reasons as to why Saladin seems so ready to accept capitulation from Ham are not at all clear to me. The capitulation arrangement is tenuous to say the least because Saladin as master, holds just a fraction more territory than Ham. Intercontinentally, the future of the vassal arrangement is too fragile to be worth it I'd say. Saladin hangs onto the vassal arrangement for some time and perhaps there are resources Ham has, that are useful to Saladin. However the envitable eventually happens and Ham breaks free after Saladin cannot hang onto his single mainland city I think because of anarchy.

Saladin would have perhaps been better off continuing on the war against Ham, which would have won the continuing support from the Christians on home soil at a guess....It is difficult to say. Will have to check out Saladin's war weariness and economy...

These saves are also important to "tune" in the context of "Revolutions" as well.
Cheers.

 

[glider1]

Duplicate post, see below.

 

[glider1]

Saladin would have perhaps been better off continuing on the war against Ham.
Cheers.

Switching player to Saladin right at the moment of invasion of Babylon at 1760AD. Here are some stats:
Saladin has no war weariness problems
Saladin's gold, research and production are middle of the road.

It depends on the results of the battle at Babylon. It appears that:
If Saladin wins Babylon:
Ham capitulates and Saladin accepts

If Saladin does not win Babylon:
Ham vassals to his friend Willem and we have a big intercontinental war!

Saladin's Babylon attack stack is well rounded but injured and is at least logically coherent on a tactical level on basic level. Saladin has no back up support so essentially in this light Saladin made the right decision to end the war before it esculates!

Here is the save from Saladin's perspective.
Cheers.

 

[glider1]

Strategically, I'd say Saladin's decision to continue on the attack from Ham's island to the mainland was a strategic mistake. Perhaps it is worth a look at the code that made Saladin push on from the island to the mainland in the first place....

Here is the save at the point I think Saladin decides to reform another fleet at Dur-Kuri on the island and attack the mainland. He does something a bit weird. He reforms a nice little fleet then instead of sending it from Dur-Kuri to the mainland at Babylon, he first directs it to Opis, Napoleon's prize city on the island, then the fleet hits Babylon.

The point is that Saladin had a good chance of capturing Borsippa the southern most city on Ham's island, instead of hitting the mainland. Did he consider Borsippa in his calculations?

This is high level complex stuff. Enjoy. I will try to contribute more concretely any way I can.
Cheers.

 

[jdog5000]

Thanks for all the analysis and saves glider, I'll take a look at the newer ones this evening.

One change I'm thinking of is blocking the AI from declaring war on a distant vassal if the master is one of its neighbors. It's a poor strategic decision for the AI ... the AI does consider the strength of the master when deciding to go to war, but it will send its main attack stack off to the distant vassal because that's the primary target, leaving itself wide open for attack from the stronger master. If the AI is neighbors with both the vassal and the master, then it's not so bad ... in terms of troop positioning it still focused exclusively on the primary target (the vassal) and can be effectively surprised by the master, but can adapt better if the war is local. I'll block just the worst case, foreign vassal and local master, and look into building up defenses along a border with a master in the local/local case.

 

[glider1]

Sounds good to me! Really what is going on here is the poor old AI has no ability to learn from past experience. It is not stupid, it is not smart, it just cannot learn. We have no philosophical justification to say that the AI is stupid, unless it actually had a learning faculty available to it....

So it has to be given the "gut instinct" and this "rule" you are thinking about, is really part of the instinct the AI needs!
Cheers.

 

[BobTheTerrible]

It would be cool if eventually down the road, the AI was able to learn (simple) things, like "City attack failed, bring more siege units next time," or "war failed, make sure we have more units next time." This would also serve a dual purpose to give AIs some dynamic personalities that change based on the AI's experiences in the game.

 

[glider1]

@Bob the Terrible
Yep was thinking the same. In fact, this whole exercise that Jdog has been doing is to improve the basic instinctual algorithms that exist in the AI. We as humans, take for granted just how many instincts govern our behaviour too....

The problem with previous attempts to build an adaptive learning AI is that it was conceived at too low a level of learning. The AI had to learn everything including problems of micro-management. This is a galaxy of complex learning. We humans could build a SETI like distributed network and let the AI evolve on it's own from a basic foundation but we might as well just leave all the PC's on the planet turned on and plugged into the network, leave planet earth, explore our solar system and come back two million years later to see whether the AI can micro-manage.....

However, Jdog and others are really making great progress on basic gut "instincts" the AI needs to have to survive, which literally cuts the problem down each improvement, by 1000 years of evolution for that AI problem......

We are reaching an interesting point where there is a hybridisation possible now between the instinct AI and the learning adaptive AI I believe. The idea would be to let the AI continue to micro-manage and conduct the basics of all it's activities instinctually, but let it learn adaptively on the higher levels....

We have a theoretical mechanism for that now....It's called WoC. It's probably way to slow but it is theoretically there. If the WoC core were to be built so that the high level decisions made inside the AI core, could be accessed via Python, you could hypothetically "plug" in python modules that govern high level learning in particular areas (diplomacy, religion etc) and to allow the AI to record it's decisions down and the "success or failure of them" and then to refer back to them in some kind of data structure....

If you think about the context of high level diplomacy, it's not that daunting a resource effort to save and record and refer to. Compared to issues of micro-management, it's no more complex than some of the things workers and units have to decide. The problem is that diplomatic context has to be stored efficiently and accurately, so that the AI can refer back to previous contexts and try an alternative decision, based on previous failures.

It's a difficult problem but really, the AI is getting to the point where it should be able to learn some high level stuff. Imagine the very scenario we have been talking about with 0.6H. The AI must not declare war on an intercontinental vassal, if the master is it's next door neighbour and has power and influence. This is still a big problem AI learning wise to establish context and do it quickly enough, but it is possible especially since diplomatic decisions like this do not always have to be made and are only a small but very significant part of an AI's ability to thrive.

This is at the point of saying that today's PC's are verging on being fast enough and the storage systems sufficiently sized and accessible, to make learning at this high level theoretically possible within conceivable time frames. The fall back the AI has is it's instinct. That's a very important starting point.

Cheers.

 

[BobTheTerrible]

Yeah, I wasn't talking about an AI that learns tons of high level things or is predominantly learning based. Using the current infrastructure, simple things can probably be done: Basically take a goal that is already in place (take city X) and have the AI evaluate the outcome (either it took the city or it didn't). If it didn't, then tweak the infrastructure already in place to add more weight to siege weapons/more offense units in the AI's next city capturing stack. Of course, I'm guessing things like that are difficult to code since they're not already in place.

 

[jdog5000]

glider:

A couple of things I learned from poking around in the various saves you posted:

- Napoleon actually chose to go to war with Saladin. The vassal/master attitude averaging influenced his decision by lowering his effective attitude to Saladin from Pleased to Cautious (works that way both with and without the changes Yakk proposed), but he was considering war with Saladin anyway. Even in the 1760 save you posted where Saladin has no vassal, Napoleon's top potential target is Saladin though with a low percentage chance. So the vassal part played a role, but not as big as it seemed. (You can see what the AI is planning by holding down ALT and mousing over the different entries in the scoreboard, I brought that over from the warlords better ai mod a little while ago).

- When I tested out the 1760 save, Ham always seemed to vassal himself to Willem. Saladin would then capture Babylon, and a couple of turns later capture Borsippa while losing Babylon to Willem.

- I can't tell why Saladin landed a second invasion by Babylon instead of Borsippa from the saves as it's already happened, but I put in a change to make the AI value invasions targeting cities in areas it already has cities over new area ... that should make the Saladin more likely to the whole island before going to the mainland in the future.

- When I tested the 1812 save, Napoleon and Saladin fought to a draw and their neighbors got distracted and joined the turmoil on the other continent. So with this set of random numbers, it wasn't a good decision by Napoleon but it didn't cost him any cities either. That's the trouble with predicting who might enter a war in the future ... Napoleon also elected to go into a "limited war" with Saladin, meaning that if he had captured a city or two he would have been happy to make peace and end the war.

- I'm pretty sure the defensive pact fest you were wondering about is largely due to the dominance of Christianity on eastern continent. There are a lot of players who all like each other over there, it's an unusual setup.

 

[Palinathas]

Not sure if this has been covered somewhere, but if we keep making the AI better it will get FAR too strong for a middling player such as myself. To this end I ask would it be possible to set the AI strength with the difficulty level. Remove some or all of the current AI boni based on difficulty, and instead have AI penaltys. Say at Deity level the AI has 100 % chance to make its best choice, at Settler say only 10% chance to make it's best choice, and only 50% chance to make it's second best choice.

I would much prefer playing a fair game with an excellent AI, that may occasionally make a mistake, than to have artificial means to make the game "fair"

 

[Arlborn]

Not sure if this has been covered somewhere, but if we keep making the AI better it will get FAR too strong for a middling player such as myself. To this end I ask would it be possible to set the AI strength with the difficulty level. Remove some or all of the current AI boni based on difficulty, and instead have AI penaltys. Say at Deity level the AI has 100 % chance to make its best choice, at Settler say only 10% chance to make it's best choice, and only 50% chance to make it's second best choice.

I would much prefer playing a fair game with an excellent AI, that may occasionally make a mistake, than to have artificial means to make the game "fair"

I'd prefer artificial means(like it is now). Why? Well, a game with a weak AI is essentially a boring game. To play with an AI that can still surprise you and do its tasks well, even if crippled, is better than an AI who does nothing good and basically is there only to slow the player down and not to actually win.

I am a Noble/Prince player myself and I'd hate to play with a "dumb" AI.

 

[glider1]

@jdog
Wow! I wasn't picking up quick enough on that wonderful feature you have brought in to study a civs planning. Beautiful. Cool!

Agree that it really boils down to critical probability forks in the tree that determine the various outcomes. This is good. We need variety, and we need AI's to appear to make mistakes as well in order to keep it fun and diverse.

Agree that at the 1812AD save posted above, Napoleon is most likely thinking about a limited war on Saladin his neighbour. He is also considering a 50% total sea war against Ham and a 90% total sea war against Willem. The basic probabilities all favour Napoleon to at least resemble a sane decision....

What happens next is where the aberrant behaviour appears...

If by 50% probability, Napoleon decides on a sea war against the buddhist Ham, he does not prepare properly at all for the automatic declaration of war with Saladin, the master of Ham. Therefore I think there is still a case to control Napoleon's consideration of going to war on an intercontinental vassal whose master is his neighbour, even if the probablity is lower. It looks really stupid more often than not and definetly takes the fun away for me to see this behaviour, unless it is balanced by your other suggestion that Napoleon at least defends his border with Arabia. This is a difficult mechanic to code!

Is it correct to say that the problem is that we have a limitation in the war plan mechanics where a civ can only execute one war plan? If a civ decides to go to war on an intercontinental vassal, it surely has to be limited if the master is his neighbour. The triggered war on the master would have to be then total!? In other words, Napoleon if he wanted to go to war on Ham, should have executed a limited sea war on Ham and total war on Saladin. That would have been the only sane thing to do even if the idea is stupid because of all the Christians on the mainland. I still think there is a case to put more cold water on the probability of Napoleon going to war on Ham, Saladin's vassal....

The whole probability thing is curious. The numbers don't seem to reflect the reality when you play through this 1812AD save. It could be just the old problem of how often do you roll the dice to get the result you want?

In any case there is still a definite issue about Saladin accepting Ham's capitulation offer...It is a huge grey area here and we probably don't want to go there! This is high level stuff. It is nice. Thanks for addressing the issue about Saladin not consolidating on the island first, before hitting the mainland!
Cheers.

 

[glider1]

Not sure if this has been covered somewhere, but if we keep making the AI better it will get FAR too strong for a middling player such as myself.

I definetly take your point about keeping it "fun". The theoretical way to go is to have a strong AI where we program in the probability of stupidity based on difficulty level. At this point, the focus for most of us would be to balance the AI to the human more across the game. At the moment, most experienced humans always have to catch up to an AI because of the extra bonuses the AI gets. Those bonuses need to be reduced over time, and then a probability of "stupidity" introduced in for the various difficulty levels. Humans make stupid decisions too and this is what we would try to simulate in the AI.

Cheers.

 

[Roland Johansen]

Thanks for all the analysis and saves glider, I'll take a look at the newer ones this evening.

One change I'm thinking of is blocking the AI from declaring war on a distant vassal if the master is one of its neighbors. It's a poor strategic decision for the AI ... the AI does consider the strength of the master when deciding to go to war, but it will send its main attack stack off to the distant vassal because that's the primary target, leaving itself wide open for attack from the stronger master. If the AI is neighbors with both the vassal and the master, then it's not so bad ... in terms of troop positioning it still focused exclusively on the primary target (the vassal) and can be effectively surprised by the master, but can adapt better if the war is local. I'll block just the worst case, foreign vassal and local master, and look into building up defenses along a border with a master in the local/local case.

If an AI is considering whether it should attack a coalition of civilizations (through defensive pact, vassal-master relation, permanent alliance or whatever), then why does it consider the various civilizations in this coalition as separate targets? When it attacks one, it will be at war with all of them, so it should pick the best target to attack from amongst all cities in the coalition.

I don't really see the fundamental difference between attacking a coalition of civilizations and attacking a single civilization. Every city in the coalition and every unit in the coalition will be fighting you just like it's the case with a single civilization. In the case of the coalition of civilizations, the opposition might be a little less organized and the coalition can fall apart, but that's not something that you can depend on. The AI should base its decision on the worst case scenario where it is fighting one big enemy civilization.

Not sure if this has been covered somewhere, but if we keep making the AI better it will get FAR too strong for a middling player such as myself. To this end I ask would it be possible to set the AI strength with the difficulty level. Remove some or all of the current AI boni based on difficulty, and instead have AI penaltys. Say at Deity level the AI has 100 % chance to make its best choice, at Settler say only 10% chance to make it's best choice, and only 50% chance to make it's second best choice.

I would much prefer playing a fair game with an excellent AI, that may occasionally make a mistake, than to have artificial means to make the game "fair"

On difficulty levels: I could imagine that if the AI were improved enormously, then it might be able to function on prince level without bonuses equally well as it presently does with the bonuses. So a one level shift in bonuses could be reachable, but more than that? Is the BetterBTSAI team planning to create a genius AI that is more than 'one level better' than it presently is? I would be amazed if that were possible. But if that is what the team is envisioning, then I really hope that they can do it.
At the moment, the AI is still a lot worse in conducting wars and in perfecting the output of their cities (especially late game) than a capable human which means that the AI needs a handicap to have a chance at victory. I don't think that the AI will ever become the equal of the capable human player in these areas. It's just too complicated.

 

[glider1]

I don't think that the AI will ever become the equal of the capable human player in these areas. It's just too complicated.

Yep agree, no question....The AI is a long way off from really top quality play, and even if it was getting good, humans will find ways to make new progress....For now it is not an issue. At the moment it is about pruning away some overtly stupid AI behaviour that definetly do reduce the enjoyment for humans. After all, what we humans are doing to this beautiful game, is attending to it like a gardener would attend to their garden. Always to make it more beautiful, not necessarily absolutely aligned and correct.

The experience with the chess algorithm already for a few years is that humans did find new ways to combat the AI until finally, resistance was futile...Now in the chess world, the geniuses of chess play between themselves because it is a beautiful thing to do and in any case, strategy games are not just about "correctness" but about "character, emotion and art".

They use the AI to discover new avenues, that a human would not have considered possible, in full knowledge that they cannot beat the AI unless for a miracle. Some of the "appearances" of new ideas the AI comes up with, are breathtaking and it is good thing to have a super brain available to the game...For the rest of the chess community, it is about introducing imperfection into the AI chess algorithm, so as to keep people enjoying playing chess and learning more about themselves, as much as about the game. You could quite easily say that self-knowledge is the only true intelligence anyway.

Cheers.

 

[Roland Johansen]

Yep agree, no question....The AI is a long way off from really top quality play, and even if it was getting good, humans will find ways to make new progress....For now it is not an issue. At the moment it is about pruning away some overtly stupid AI behaviour that definetly do reduce the enjoyment for humans. After all, what we humans are doing to this beautiful game, is attending to it like a gardener would attend to their garden. Always to make it more beautiful, not necessarily absolutely aligned and correct.

The experience with the chess algorithm already for a few years is that humans did find new ways to combat the AI until finally, resistance was futile...Now in the chess world, the geniuses of chess play between themselves because it is a beautiful thing to do and in any case, strategy games are not just about "correctness" but about "character, emotion and art".

They use the AI to discover new avenues, that a human would not have considered possible, in full knowledge that they cannot beat the AI unless for a miracle. Some of the "appearances" of new ideas the AI comes up with, are breathtaking and it is good thing to have a super brain available to the game...For the rest of the chess community, it is about introducing imperfection into the AI chess algorithm, so as to keep people enjoying playing chess and learning more about themselves, as much as about the game. You could quite easily say that self-knowledge is the only true intelligence anyway.

Cheers.

Wow, great thoughts, I especially like the view of the gardener.

 

[Arlborn]

I definetly take your point about keeping it "fun". The theoretical way to go is to have a strong AI where we program in the probability of stupidity based on difficulty level. At this point, the focus for most of us would be to balance the AI to the human more across the game. At the moment, most experienced humans always have to catch up to an AI because of the extra bonuses the AI gets. Those bonuses need to be reduced over time, and then a probability of "stupidity" introduced in for the various difficulty levels. Humans make stupid decisions too and this is what we would try to simulate in the AI.

Cheers.

Again, bad AI = boring AI

 

[jdog5000]

If an AI is considering whether it should attack a coalition of civilizations (through defensive pact, vassal-master relation, permanent alliance or whatever), then why does it consider the various civilizations in this coalition as separate targets? When it attacks one, it will be at war with all of them, so it should pick the best target to attack from amongst all cities in the coalition.

I don't really see the fundamental difference between attacking a coalition of civilizations and attacking a single civilization. Every city in the coalition and every unit in the coalition will be fighting you just like it's the case with a single civilization. In the case of the coalition of civilizations, the opposition might be a little less organized and the coalition can fall apart, but that's not something that you can depend on. The AI should base its decision on the worst case scenario where it is fighting one big enemy civilization.

The current system is an interesting hybrid of coalition considerations and separate considerations. Attitude is averaged (for vassals, not defensive pacts) which determines the % of chance a particular target will be considered each turn. The comparison of military power is done by coalition and the AI will not consider attacking coalitions which are too powerful. The weighting of attractiveness of territory to capture is done separately (though attitude plays a role, power may as well) and, if all the war rolls pass, the highest weighted victim that is considered that particular turn is chosen.

The AI then picks a single target, which makes sense since there can be anywhere from ~10 to ~40 turns (Epic) before it actually declares war so the coalitions can change. This has one bad effect in that it won't bulk up defenses along its borders with other members of the coalition (yet). The other main effect is where the attacker's surprise stack moves to start the war: it always attacks the chosen single target as it should, since that's the territory it found most attractive (with some attitude-based randomness). The attacker then also prefers to keep that war and is more willing to end wars with other members of the coalition who attacked it.

On difficulty levels: I could imagine that if the AI were improved enormously, then it might be able to function on prince level without bonuses equally well as it presently does with the bonuses. So a one level shift in bonuses could be reachable, but more than that? Is the BetterBTSAI team planning to create a genius AI that is more than 'one level better' than it presently is? I would be amazed if that were possible. But if that is what the team is envisioning, then I really hope that they can do it.
At the moment, the AI is still a lot worse in conducting wars and in perfecting the output of their cities (especially late game) than a capable human which means that the AI needs a handicap to have a chance at victory. I don't think that the AI will ever become the equal of the capable human player in these areas. It's just too complicated.

Yeah, although it's a very nice compliment, I agree with you Roland that the AI will not shift so dramatically in ability as to make any dumbing down necessary. The AI will simply never be able to plan strategically at the level to routinely beat anyone who's invested enough in civ to browse these forums at or below Noble ... it might happen occasionally, but with equal in-game resources the human brain will usually win. If it takes players a couple of games at a lower difficulty to adapt to the new AI, then I'd say that's a job well done.