Artificial Unintelligence

How about setting the value FLAVOR_RANDOMIZATION_RANGE to 1 instead of 2 in the GlobalAIDefines?
I used to play G&K with the less randomized leader flavour mod and it improved the AI significantly. However, I am not sure if this would improve the AI in BNW, too.

 

I actually changed the flavor randomization algorithm in this mod: the randomization happens in a binomial distribution instead of a flat one.
A flat distribution means that with a flavor randomization range of 2, there's a 20% chance of getting each of the five possibilities: -2, -1, 0, +1, and +2. A binomial distribution means that with a flavor randomization range of 2, there's a 6.75% chance of getting -2, a 25% chance of getting -1, a 37.5% chance of getting 0, a 25% chance of getting +1, and a 6.75% chance of getting +2. Essentially, there's an 87.5% chance that the result will be within a randomization range of 1.
This solves the drawbacks of having a large randomization range, as the AIs will still tend to get values near the original amounts with only a few extremes, while still providing the high degree of randomness that a small range simply cannot offer (since that 12.5% chance of going outside the smaller range can still happen, it just doesn't happen often).

I also created a reroll mechanism to stop flavor values from bunching up at extremes. Normally, if a random roll would take the AI above or below the allowed thresholds for flavor values (i.e. below 1 or above 20), the game would just set the flavor value to whatever threshold it overtook. The problem is that this reduces the randomness of AIs with low default flavor values, as the flavor threshold has a much greater chance of being picked than the value one above it. I solved this by having the AI keep rerolling binomially by however much the previous roll went outside the total until the reroll no longer lies outside the threshold.
Example: an AI with default flavor 2 rolls a -3, which would bring its final value to -1. In the unmodded game, that value is below the minimum of 1, so its value is set to 1 and the game moves on. In this mod, the value is first set to 1, then a binomial reroll happens with a range of 2 because the original roll went 2 over the threshold; if the new value is still below the threshold, the cycle keeps repeating until the new value no longer goes below the threshold.
It's a bit unoptimized (I'm sure I can cut down the maximum amount of cycles it runs by making sure it never rolls a result in the reroll that's simply a repeat of the previous roll), but since the algorithm executes only once while the game is loading, I can deal with that.

 

Sounds good to me!

 

First of all, fantastic work so far! AI modding has to be incredibly challenging/frustrating, especially when there's only so much that can really be done.

What are the chances that this project could ever successfully remove the large head starts that AIs receive in favor of per era bonuses? If done well I'd imagine it could make the AI feel more competitive throughout a game rather than just being immensely ahead at the beginning.

 

In Civilization 4 when I did modding AI, I found that I could make the AI extremely competitive but at a huge cost. If you sort of brute force a tech tree, and decisions for the AI you can make even Prince have competitive AI.

It just makes the AI extremely consistent, and boring. I would make very difficult AI in Fall From Heaven that is way more complicated, just by brute force decisioning the tech tree.

For example in Civilization 5, there's really only a few core strategies for Deity. There may be a few who deviate, but it's basically the same.

Brute forcing the tech tree to follow the core Deity Strategy, plus the AI improvements that Artificial Unintelligence has already added should make the AI competitive even at low levels.

But again, the AI becomes boringly predictable. Just like playing Deity is boringly predictable.

Maybe I'll make a setup of how I used to do this and test it out.

 

I know the Community Patch has implemented this to an extent, might want to do a quick powwow with Gazebo's Github page first? Granted, the CPP is a complete overhaul designed with AI competitiveness in mind (perhaps like what you mentioned) so it could be like comparing apples and oranges.

As a Deity player (at least when I had more time to play ) consistently is what I'm sick of, though I realize it may be an impossible battle. It always stinks knowing that on Deity, or even Immortal, a vast majority of early wonders are out of the question before you even generate your map. At least having the option (whether or not it is a good option is another question) to build said wonder would go a long way for a lot of players, I'd think

 

C++ is versatile enough that I could technically implement anything and everything I want, it just takes a lot of time and effort to write and test everything.
The primary source of frustration is not that there's stuff I cannot do, it's when something happens and I have no idea why, not even after hours of crawling through code with a debugger. It's what's currently holding me up from properly testing my DangerPlots stuff, as the pathfinder constantly misbehaves (read: it seemingly disobeys code, resulting in fun things like infinite loops and null pointer exceptions), so I can never get more than 40 turns into a test game before the pathfinder derps out and forces me to quit the session.

It's definitely doable, just not something I'm interested in. One of the main reasons is that I'm not good enough at Civ5 to play at higher than King . The more acceptable, second reason is that I dislike both stat-based advantages and disadvantages for AI players, as having either essentially means that the AIs are playing a different game than the human player (certain buildings, units, wonders, and policies become more or less useful depending on how what stat-based bonuses or penalties the AI receives, obvious examples being anything that grants a free tech, a free policy, or free units).

While revamping the game's difficulty system to revolve around AI knowledge instead of stat bonuses (eg. Deity AI might know the result of every random roll ahead of time) is something I'd definitely want to do, I've relegated it to wistful thinking the same way I've handled my other brainstorms, such as an ideology revamp revolving around control vs. autonomy/efficiency (short version: players would have to choose between having OK bonuses over which they have maximum control or excellent bonuses over which they have little/no control; if you're familiar with Victoria 2's 4 economic policies, you know exactly what I'm talking about), a settler/culture revamp (short version: map has neutral cities, settlers "settle" your culture instead of cities, letting you culture-shift cities in addition to founding cities in untamed lands), a policy revamp that turns policies into a ring-shaped, culture-fueled tech tree that is city-specific instead of player-specific (large empires would progress through culture slower not as a result of increased policy costs, but because getting all their cities up to speed with the latest cultural tech takes longer), a trade route revamp that would tie into Great Merchants and would place more focus on the actual trade unit (eg. trade income would be gold shipments that you'd receive only when the trade unit enters your city's tile, raiding a trade route would require actually catching the trade unit), and a city-state revamp that would allow promotion and demotion between major civs and city-states as well as allowing humans to play as city-states (a necessity if they can be demoted to a city-state).

For now though, I'm strictly focusing on AI algorithm improvements for now with the occasional foray into code optimization and/or code cleaning (plus that one, new, binomial RNG), as these are things that everyone can appreciate and that are generally compatible with others' modding efforts, even if they aren't as creative as my other Civ5 daydreams.

This approach essentially turns the research and policy bits of the AI into a neural network framework, albeit replacing thousands of test runs with information gathered by human players. However, I feel that the lack of AI variety has more to do with poor game design than being a framework drawback: such a result, if it also objectively improves the AI's performance, might indicate that the game is designed in such a way that there are very few (possibly only one) ideal paths for every initial set of conditions (civilization, starting plot, map setup). It may also indicate that not enough tests were run, but that's always a risk you run when using working with a neural network AI.

The two main problems I have with this approach are both caused by the fact that you're relying so heavily on top-down analyses. The first is time-precision: running millions of test games to gather all the data required for precise prioritization would take ages; though you can speed up implementation if you rely on player-developed strategies instead of iterating through millions of test scenarios, you'll be limiting the AI to strategies humans have documented instead of allowing it to find those strategies (and possibly undocumented ones) on its own and do the fine-tweaking algorithmically. The second issue is lack of modularity: the instant a person uses the AI mod alongside a mod that adds, changes, or removes any building, unit, wonder, policy, or civ-specific bonus, hard-set priorities become a liability, since you can no longer guarantee that your prioritization is ideal for the new ruleset.

 

How can I use this mod in multiplayer for the AI? Someone has a step-by-step how to activate that option in the menue?

 

You essentially have to package the mods you want enabled into a "fake" expansion pack, allowing you to use the regular menu for games instead of having to go through the mod menu. See this thread.

I've slowed down a bit with development, both because Visual Studio is becoming increasingly unwieldy for my low-RAM computer and because of a larger-than-usual university workload. Also, debugging v10's two big overhauls (DangerPlots and Great Works swapping) requires me to disable compiler optimizations, meaning that test games run at least two times slower than before.

 

Running a game with only AI civs to try and understand their decisions at this point with Artificial Unintelligence.

Every civ used worker first as first build

Settlers when built waited outside of their capitol for an escort. This could be a ton of turns sitting and waiting.

Sweden settles 2nd city, decides to build a water mill in the capitol. Not a great building at this stage.

Spain has a settler stuck outside the capitol. There is an escort there, but perhaps it has another job it's doing? Spain builds another settler, so now has 2 at the capitol, will be interesting to see what happens.

Russia builds a library and then immediately after the great library.

Warrior garrisoned in Spainish capitol. 2 settlers, 1 outside 1 inside, both going no where.

Spanish Warrior finally decides to move with Settler to found a city. Spearman is coming back to escort the other settler it looks like

One settler is moving around without an escort, but walked over to a location, then decided to walk an entirely different direction. We'll see where he ends up

Turn 83 - Almost every civ has Currency, while neglecting other techs. Strange they'd all rush for Currency.

Turn 100, not a lot of libraries built, interested to see when the first national college comes up.

Austria has 3 libraries built in her 3 cities, but doesn't have philosophy.
Celts have Philosophy, and 3 cities, 2 with libraries. But they are building a worker in the 3rd city and it'll take forever to get that worker out. No hope for national college any time soon with that build.

Austria still no philosophy, building a settler in the capitol so going to end up delaying national college even more.

Russia has everything needed now to build National College. She's building National Epic instead shakes head

Ottomons building writers guild in a city with no food, and that city needs the last library to be able to make national college.

Denmark has all libraries, but no Philosophy.

Austria building National Epic, still no Philosophy

Turn 130 - Russia starts on National College! It's in the wrong city, but who cares at this point, at least it's being built.

Ottomons and Austria both are 1 policy from finishing liberty, and are instead going in other policy directions. Bold move Cotton, let's see if it pays off for them.

Sweden is unhappy and building Aqueducts. Might want to consider a happiness building there!

Russia constructs Haglia Sofia with no religions left to found.

Vikings Building Petra in a city with 1 desert tile. They don't have calendar yet, but have techs all over.

Ottomons start building National college! woohoo

 

Surprised you didn't catch Terracotta getting built in a city that has one warrior garrisoning it.
Besides the usual blaming of flavors and the flavor system in general, production choice still has a whole slew of problems that I have yet to solve. You've seen one of the more obvious ones: since the flavor system is the only way the AI checks whether it wants to build something or not, it has no idea how to handle situational wonders, or national wonders that need early prioritization because of their building requirements. This is why, for example, you'll see the AI building Petra in a city that has one desert tile, or the Hagia Sophia when it doesn't need any more prophets, or Terracotta when it only has one unit to clone, or Borobudur when it doesn't have proper missionary targets.
A more harder-to-notice problem that still has a large effect is that the AI only builds wonders in cities that have the "wonder" specialization, and it will only ever allocate up to one "wonder" city. This effectively means that the AI will only ever work on one wonder at a time, including national wonders; if the AI is building Petra in that city with one desert that tile, that means it will not even consider building a national college until Petra either completes or gets built by someone else. Great Engineers are exempt from this limit (the AI will still use them in non-wonder cities to rush out a different wonder), but the AI will never use them to build national wonders.

It's more likely that Spain built the first settler, then the second one popped out from that Liberty policy. The spearman that didn't come back to escort was likely marked as a scout, while the warrior was likely recruited into a barbarian attack operation that was waiting around. I think I can speed up future cases similar to the warrior to switch to escort more quickly, but I don't think I can speed up future cases similar to the spearman unless I rework both the scouting and sentry code.
The hard logic behind escorted settlers will be problematic no matter what. Replacing it with something else would first require overhauling the sentry system though (since units would act as sentries to scout ahead of the unescorted settler), and that's not something I've even glanced at yet.

Settlers that are waiting for an escort patrol around via Homeland AI or stay put (if they are meeting up with the escort or the Homeland AI's patrol code tells them to stay put).

I kind of want to blame the flavor system again, but in the AI's current state, I'd actually say that policy finishers, especially Liberty's, aren't as useful for the AI as they are for humans, especially since so many of them are not the stat-type bonuses that the AI can weigh properly. A free GP of your choice, for example, is a lot worse for AIs than for humans: not only can the AI not plan for the free GP ahead of time (it'll always be "surprised" by the free GP after it unlocks its final Liberty policy), but it also cannot handle most GP's half as well as a human player (eg. it won't save up scientists to bulb, it won't maximize its generals and admirals properly pre-v10, it won't even select the best wonders to rush with its engineers). Four free aquaducts are nice, but the AI has no way of anticipating it, so it'll still build aquaducts in its larger cities before it gets the Tradition finisher that would make them free. Heck, even Firaxis noticed that this sort of behavior could be quite problematic, which is why the last policy in Rationalism has spaceship flavor instead of science in addition to a lower-than-usual flavor value (Firaxis' XML comments write that they did this to stop the AI from unlocking it in early Industrial instead of waiting for Atomic or Information to get the most beakers out of the free tech).

 

There were some very close calls, but the civs (randomly) created additional units while Terracotta was being built, so every test looked reasonable.

I think I saw a civ building a wonder, and a national wonder at the same time. I believe it may have changed flavors or some other change in between. To me the harder-to-notice problem is that it seems the Civs don't evaluate their build decision after they've made it, making them continue to build something that may not make sense. They get stuck in their previous orders.

I watched them being built in their queue.

I think scouts should be included in the escort. It should solve some problems. Ideally the solution would be that if a City has a Settler being built, that someone is assigned at that point to return to the city to escort.

I've also seen a unit as garrison in a different city, not come over and escort a settler.

I ran similar tests on community patch, and they had fewer problems here, because scouts were escorts

I don't think this was the case in this particular instance. I've watched several games now, and there are times where Settlers do not wait. I don't know what causes this distinction. I do agree the majority of the time they follow the logic you've outlined.

I don't think the flavor system is bad, I think the values in the flavor system are bad. I've been doing editing of them, and I can get the AI playing significantly better by increasing the flavors towards things that Deity players do.

Example - Currently all the civs ignore Philosophy. This means they don't build National College, and makes them far behind.

By increasing all flavors value for it, they get it, get national college much sooner, and have more than double the science earlier.

You just have to be heavy handed with the flavors, and force their hands. Right now, all Civs go after Currency early. It's illogical, and poor gameplay.

If they all go after Philosophy early, they are enabled to accidentally luck into being far more powerful (National College).

If you compared the game I forced them into Philosophy, vs. the default, it's night and day. A massive difference.

The negative here is that, war based civs lose their ability to Zerg Rush a bit. The positive is, they are far more competitive than if they Zerg Rushed. Sort of depends on your outlook there.

I also think because you have Ranged fixed for move and shoot, ranged unit Flavors need to be far higher. I think they may have had them low because ranged didn't work all that well.

 

My apologies, I looked into the production code, and it turns out the AI can build wonders in multiple cities at once. The AI simply treats all buildings, wonder and non-wonder, alike for its regular production decision, but will only build wonders in the city to which it has given the wonder specialization. The AI also reweighs production flavors based on the turns required to complete a build order, which is why wonders with their long production times rarely ever get chosen in cities other than the one with the wonder specialization; the actual adjustment is: new flavor = (old flavor) / (turns to construct)^(0.15 + 0.015*(turns to construct)), where 0.15 and 0.015 are XML-modifiable values AI_PRODUCTION_WEIGHT_BASE_MOD and AI_PRODUCTION_WEIGHT_MOD_PER_TURN_LEFT respectively.
If you expressed the denominator as f(x) = x^(ax + b), where x is turns to construct and a and b are your two XML constants, then df(x)/dx = x^(ax + b) * (a ln(x) + a + b/x); this means the reweighing can get rapidly out of hand. For example, if two buildings have the same flavor, but one can be built in 5 turns, while the other can be built in 2 turns, the 2-turn building's flavor values will be about 26% higher than the 5-turn building's; for 15 turns vs. 25 turns, the 15-turn one has almost twice as much flavor as the 25-turn one. Thankfully the derivative's value doesn't exceed the function's value until e^(167/3) = about 1.498*10^24, assuming original XML values (it's e^(1/a - a - b/a) for modded ones). Still, this can have a huge effect on national wonders whose production costs go up with each city, and is probably why the whole wonder specialization thing was needed (the AI would never build world wonders otherwise due to severely reduced weights from production time).

The fact that the AI doesn't reconsider build orders is not a problem I have yet considered, but I do admit it definitely is one, especially in Marathon games and/or for smaller cities.

Strange. I'll consider hard-capping settler production for v10 so that the AI doesn't build new settlers if there is already one that is not moving out or waiting to meet up with its escort.

I've already sort of implemented it as dynamic UnitAI flipping (a lot of roles that the AI chooses are based on a unit's UnitAI value, which almost never changed once a unit was created, the only exception being land non-scouts set to UnitAI_Explore switching to their primary UnitAI if there were enough scouts). In fact, I already implemented a check that will switch the UnitAI of a non-scout if it's needed for an operation, but I guess I'll need to revisit it to make it more aggressive.
As a side note, dynamic UnitAI flipping also solves a lot of other problems, such as letting Roman Legions and Samurai build improvements when necessary (via flipping their UnitAI from UnitAI_Offense or UnitAI_Defense to UnitAI_Worker or UnitAI_WorkerSea when improvements need to be built, and flipping back when they are needed for soldiering again).

I swear I have as well, and I've yet to trace the source of the issue.

When a settler is created, it immediately creates a settler operation (there are also analogues for Great Merchants that want to go to City-States and Great Musicians that want to have Concert Tours); this operation is a special, single-hex operation, so its movement and targeting rules are explicitly defined in TacticalAI's code. Single-hex operations also rely on there only being two slots in the operation, one for a civilian unit and one for a military escort.
After the operation is created, the settler is immediately recruited into the operation's army (each operation controls one groups of units, or army, although a lot of residual code seems to indicate that Firaxis originally planned operations to be able to handle multiple armies). The operation then looks for possible escorts to fill the second slot. If it finds a potential escort, even if it is very far away, it recruits the escort into the army and lets TacticalAI handle its movement code from there (it usually involves the settler waiting in the same hex until the escort enters the same tile as it, then the two moving out to a target determined when the operation was created). However, if it cannot find an escort, different things may happen: in the unmodded game, the settler would almost always move out immediately, though residual code indicates that Firaxis only wanted AI's with a Boldness of 6 or higher to do this. As of AuI v9, the AI will either wait with its settler or move out depending on the result of a random roll: if (AI Boldness) + (binomial RNG roll between 0 and 6 inclusive) > 9, the settler moves out, otherwise it will wait for an escort to become available. The random roll happens each turn with only one success needed to have the AI send out the settler, so the amount of turns the AI waits before sending out a settler is a negative binomial distribution whose gradient depends on the AI's boldness (the binomial RNG used in the roll amplifies the differences in probability between neighboring boldness values). Right now, the code is compromise between unmodded functionality and what appears to be Firaxis' original plans, though I might consider bumping up the binomial RNG roll's range so that even AI players with minimum boldness will eventually send out unescorted settlers. Ultimately, I would still prefer to rewrite single-hex operation escorting, but it would require a lot more time and effort than just simply tweaking a binomial RNG range.

It's passable at best; sure, you can get the AI to be unstoppable at higher difficulty levels, but that's only due to its stat-based bonuses (eg. extra units, extra buildings, extra techs, extra yields). If you want an AI that can stand toe-to-toe with human players without any stat-based bonuses, a flavor system is just not doable in a game so full of conditional bonuses: Petra's value depends on Desert tiles, promotions' values depend on what terrain the AI will be fighting in with the units it creates (yes, promotions also use a flavor system, though none of it is available for tweaking outside the dll AFAIK), the values of policies that give free units depend on how easy it is for the AI to produce units with similar strength, etc. You'd have to implement so many different flavor checks and modifiers for the different possible effects that you might as well just scrap the entire flavor system and approach the problem from a different direction (maybe a yield/stat exchange rate of sorts?).

Though modifying individual flavor values for Philosophy and Currency is definitely one way to solve the problem, I do wonder if there are perhaps more thorough solutions that I can implement thanks to C++ code. For example, currently the flavors of techs are determined by their flavor values, the flavor values of techs they unlock (which, in turn, are affected by the flavor values of techs they unlock, so on and so forth), and whether or not the tech unlocks a unique item; Maya get a unique flavor boost to Calendar. If I could implement a method to have the flavors of the contents of a tech propagate to the tech itself (and remember that the flavors of buildings and units is modified by cities, so this means that the flavor of a tech changes as the game progresses), it would not only handle Currency vs. Philosophy problems, but possibly others as well (eg. Education vs. Physics).

Keep in mind that flavors have a player half as well (overall production weight for units is sum((player flavor)*(unit flavor)) + (flat weight based on the operation the unit is being produced for that is affected by offense flavor), all reweighed by production time), so if the problem is with all ranged units, you could also just increase Ranged flavor for all AI leaders and achieve a similar effect.
Speaking of ranged units, I really should set up a scenario of sorts to see if my parthian addition (move-shoot-retreat) actually works; I implemented it quite a while back along with my other improvements to Ninakoru's original move-and-shoot algorithms, but I've yet to see it in practice.

 

This would be an improvement in so many cases. For example, the Civ that's building wonders that is attacked, yet can't build anything new in any of it's cities for 20 turns.

The AI loses it's settlers waiting for an escort more often than if they were to go out on their own. If the Settler stayed in the City, this would be an improvement.

My solution at the moment is to keep the minimum number of units defending the homeland much lower, and have all unit types as optional escorts for the Settler. I would post the full solution if I felt it was fantastic, but there are still issues with it, and I don't feel it is "ready" yet.

That's great. The issue I have is that the first unit starts as explore, and remains as explore no matter what. This is odd when a unit upgrades from a goody hut, and remains an explorer when he could be used to 1. escort a settler, 2. defend the homeland, 3. be a part of an attack. (clearing barbs, or whatever)

There is also a large problem in my opinion that a Settler determines where it will settle, and then starts to move there. The position they would settle is only reevaluated when they arrive. This causes settlers to move many turns to a location, only to turn around and move many turns to a new location.

I'd prefer a settler reevaluating after every step, it's that important of a decision.

All of my work is done on Prince. I am trying to make competitive AI on prince. When I run a Deity test of the work I've done, it's absurdly unstoppable.

I don't disagree there are a challenges, but I don't think it's as difficult as you're expressing. There is so much in the default setup that is completely unoptimized.

Running a game of your mod, vs. a game of my edits + your DLL is a completely different experience. My AI is leagues harder due to simple changes.

I.E. I agree it won't be perfect, but we're not even past bad, and I think we can reach good.

The main problem here is a bit complicated to explain as I see it, but I've essentially corrected it for my usage.

Instead of going from small numbers, to larger numbers in the flavors, I've flipped it. With higher numbers earlier in the tree.

This allows me to force the AIs hand in directions. The problem with flavors currently is the numbers are so high deeper in the tree, that the AI will start beelining for a Tech without getting important techs before it.

When I implemented this in Civ 4, I used Python and if thens to get what I needed. I don't see how to do that with Civ 5, so I am using the Flavors to do a version of this same concept. It's working, but it's annoying to debug. You literally have to test it repeatedly, and I don't see logging to indicate to me which Civs are running which flavors. More logging info would save me tons of wasted time, if you are aware of how to get explicit AI details.

I'd say it doesn't work, or I'd have seen it given how many tests I'm doing.

 

One thing that consistently drives me nuts is the AI doesn't select a build order when they build a city, or complete a new build order at times.

You can see it easily whenever the AI settles. They don't select a build order until the next turn. This slows down their ability to rush items, and all sorts of other things.

I haven't done testing to see if they are actually losing hammers because of this, if they are actually slower than a human player. Can you look into this? This seems like it would be a big help, but perhaps I'm misguided here.

 

Major issue that I see.

1. Settlers and Workers have to be protected. They are walking towards barbarians they've seen before. The ideal solution would be to have them consider that area extremely dangerous, and then send out a barbarian squad to deal with them.

Today, they walk directly into obvious trouble. I'd take the AI knowing everything about Barbs over this situation. A simple AI always takes into account Barbs as if they knew where they were is a significant improvement, and is fairly close to human perspective.

I just think it's closer to right than current code.

2. The AI goes directly for Guilds because of the weight of the Guild buildings. It rushes the wrong part of the tech tree to be competitive.

It has to go Philosophy no matter what.

My idea is to just increase everyone's science flavors, and make the science flavors match the ways to get Science.

There is no good strategy that involves ignoring Science.

You have to build Libraries, the National College, Universities, and Public Schools. The faster you do it, the better military you'll have.

Rushing Currency just kills your competitiveness.

3. UPDATE Defines SET Value = '12' WHERE Name = 'AI_OPERATIONAL_BARBARIAN_CAMP_DEPLOY_RANGE'; -- Default 2

The AI will completely allow Barb Camps to spawn and won't defend themselves. Just take out the threat.

I have been playing around with having city states do this too. They might as well get some experience.

Also, the AI doesn't have any clue when it's civilians get taken. You'd expect a squad of the guys they have to go fix it, but they do nothing. They could have their entire territory filled with units ready to war, and not go get the 3 settlers they lost.

 

Found how to get all the logging on, makes my life much easier.

Part of the Settler problem is they are patrolling.

Settler (32770) patrolling to, X: 30, Y: 25, Current X: 30, Current Y: 26

It would be better to just leave them in the capital until a unit is available. Patrolling puts them in harms way without an escort.

 

Sorry for not replying earlier, it's been a busy week at university. Gotta love statistical thermodynamics. I'm writing this with a significant sleep debt, so my apologies for typos and if I seem a bit flippant or have missed something.

Wonders are the only thing the AI is allowed to interrupt, but it only does so if an operation has been requesting units from the city for the past (AI_CITY_SPECIALIZATION_REEVALUATION_INTERVAL) turns.

This was one of the main reasons I started rewriting the DangerPlots code from scratch in mid-January; in the unmodded game, each tile keeps track of the sum of the combat strengths of all visible, enemy units that could attack onto the tile within 2 or 3 turns. It currently works exceptionally in my v10 dev build in most cases (the AI not only distinguishes between units that can move onto a tile and those that can only ranged attack onto a tile, but it also knows precisely how much damage any unit could suffer on a tile, leading to more accurate healing and retreat code), but there are still plenty of rough edges to smooth out. For example, the AI cannot properly recognize enemy units that can attack from fog, leading to some hilarious situations where an AI will sit its crossbowmen within range of enemy longbowmen and just leave them to die; this was still the case in the unmodded game, it's just amplified tenfold by the new DangerPlots due to how much other systems rely on DangerPlots' new accuracy.

I'm not sure how you got all unit types to act as optional settler escorts: operations only support one primary and one secondary unit type (adding multiple secondaries should not work), and single-hex operations (which include both the regular settle city and the settle-city-on-another-continent operations) are not programmed to be able to account for more than one land unit escort (sea escorts are fine), the same way they're not programmed to account for multiple civilians or sea-based civilians (eg. admirals).

I'll be honest, the way non-scouts are automatically allocated and unallocated to and from scouting is still a bit mysterious to me. Yes, I've read over the algorithms a bunch of times and even made a few modifications to fix some bugs (eg. if an unmodded AI accepts the embassy of a player on another continent, it will start allocating a large portion of its land army to scouting and then send them out to attempt to scout the fog surrounding the other player's capitol, even if they'd need to cross an ocean and don't have ocean embarkation yet), but the whole thing still doesn't *click* for me the way, say, operations have.
Hmm, and now that you mention upgraded scouts from ruins, I also haven't checked to make sure there aren't any problems with scouts upgrading to units that do not have a UnitAI_Explore flag.

That's definitely doable, especially since I swapped out settler evaluation to use the pathfinder instead of distance. I know that the Community Patch has implemented something similar, though AFAIK, theirs only reevaluates on arrival to the original target. I'll see what I can do after I figure out why Great Generals just sit around in cities all the time in v10. It's probably a mistake in the new DangerPlots. It has always been a mistake in the new DangerPlots. *sigh*

Hmm, I may be underestimating the power of flavors then. It's still not something I'd prefer to rely on, but if the unmodded flavors are truly this unoptimal, I guess a few tech flavor changes won't hurt.
I've been eying some of WHoward's Lua script system insertions and wondering whether I can create something for flavor modifiers, ie. allow mods to modify the flavors of items based on Lua scripts. The main advantage would be for mods that add items with very specific effects: for example, a Petra-like wonder would be able to modify the wonder's food and production flavors based on the amount of eligible tiles within working range using a Lua script, rather than forcing the modder to either deal with static flavors or have to rely on C++ code. It would certainly have its limits (Terracotta, for example, since it not only needs to check for nearby units when selecting the wonder, but also make sure enough units stay in range when it completes), but it should do for most cases.

Looking at the actual flavor values, it seems like Firaxis just filled them out on a whim with very little thought into how much a flavor value of, say, 20 or 50 actually means. Based on AI log files, it seems that techs' flavor propagation doesn't work as consistently as I thought it would. I also just realized another problem stemming from the way lategame techs' flavors propagate downwards: since the AI always compares total flavor values for techs, useful lategame techs will always have less total flavor values than less useful earlygame ones simply because there are less future techs whose flavors will propagate downward. I'm sure there's a way to solve both problems at once by altering the way flavors propagate in the tech tree, I just haven't figured out how... not yet, anyway.

If you enable logging and AI logging (via setting AILog = 1) in the Civ5's config.ini, the game will print detailed .csv logs into My Games\Sid Meier's Civilization 5\Logs about almost every AI-related decision. One of those logs, TechAILog, displays all information related to what techs the AI chooses: whenever the AI must choose a tech, all available techs' flavor values are logged as well as what tech the AI chose (it'll be the one with the highest flavor).
If you want to have separate logs for each civilization, make sure to set PlayerAndCityAILogSplit = 1 in config.ini. Also, make sure you clear the log folder after each game (you'll need to quit Civ5 to do so), or else subsequent games' logs will be appended to existing logs; this a minor nuisance at worst if you don't split logs for each civilization (since everything's just appended to one log file), but it can be quite a pain if you split logs.

It's a very situational thing: the code will only fire for units that can attack after moving after they've attacked a target that has not died from the units attacking it that turn. The unit must be able move to a safer spot from which it can move and attack the target next turn. Due to these conditions, it's highly unlikely to trigger for melee units, and since I still haven't gotten around to having the AI make sure it keeps units with a lot of promotions alive, Keshiks are really the only case where the code will probably fire.

Unless I'm wrong, the AI will choose a city's production right before production is tallied up and spent, so no turns or hammers are actually being lost. If anyone wants to double check, cities' production is tallied and spent in CvCity::doProduction(), which fires off an AI_chooseProduction(false /*bInterruptWonders*/) right at the beginning for AI/automated cities that are not producing anything or are working on a process (Wealth, Science, World Congress Project).

... Guilds' 20 Gold flavor is just ridiculous, especially since things like having negative income boost gold flavor as well. Outside of AuI's special, multiplicative science flavor boost (all building, policy, and tech science flavors are multiplied by 8, 4, or 2 if the AI is in at least Classical Era and missing techs marked as "crucial" to their prominent Grand Strategies: Archaeology and The Internet for Culture, Globalization for Diplomacy, etc.), there are no real ways any of Philosophy's three flavor types (culture, science, religion) get boosted throughout the game; Culture and Spaceship Grand Strategies boost them a little, but it's not nearly as much as all the flavor modifications of Gold, Growth, Expansion, Production, Offense, and others from city strategies, economic strategies, and military strategies. Tech flavors were assigned with no regard to what they unlock: even though Philosophy unlocks one of the most important science buildings in the game, it actually has less science flavor (3) than Optics (5).

I don't think altering that value has the effect you think it does. The deploy range is the maximum allowed distance between an operation's true target and deploy target. The true target is the operation's actual target (enemy city, barbarian camp, etc.), while the deploy target is the tile to which the operation's army moves; when the army reaches the deploy target, its movement code is handed over to the tactical AI, the AI class that handles all non-passive, non-scouting, and non-worker moves (those are handled by the homeland AI). If you increase the value to 12 from 2, what you're doing is allowing destroy barbarian camp operations to deploy 12 tiles from the target camp instead of 2, possibly leading to unintended side-effects of units wandering off away from the camp instead of attacking it, or units with more movement speed than other units in the army rushing ahead to engage the camp alone before the others can catch up (the operation AI performs "move-as-a-group" corrections, but the tactical AI does not).

It's pointless, city-states don't have an operation AI from what I can tell, so they will never create or run a destroy barbarian camps operation, among others.

This is a problem inherent to the model of operations controlling groups of units; if you wanted to stay within the operation model, you would have to create a new operation specifically for stealing back civilians from barbarians, then make sure that operation can accept every military unit type (which would either require rewriting the formation recruitment code or creating four clones of the same operation to cover every unit type), then make sure there are no conflicts with other operations (a much harder task, since you might get overlap between this operation and the destroy barbarian camp operation, which gets increased priority if the camp contains a captured civilian that originally belonged to the current AI player).
Instead, I may start experimenting with "hunting" algorithms in the far future. Even if I let the AI cheat and always know where its captured civilians are, it would still need to know what unit(s) to send. The primary purpose would be to farm barbarians for XP in peacetime, since this is probably the easiest to test and debug, but they could be used for a bunch of other stuff, including hunting down captured civilians.
Plus it'd be nice to say that the mod has a hunter-seeker algorithm.

Patrolling is just simply what the homeland AI does when it can't do anything else with a unit. I don't exactly know off the top of my head how and when the homeland AI gains control of settlers (since it should be just the operation AI and tactical AI handling them), but patrolling shouldn't normally be an issue in AuI (civilians would patrol to dangerous tiles in the unmodded game, this was supposedly fixed in AuI v8) so long as its DangerPlots are accurate... which brings us back to the dysfunctional DangerPlots system pre-v10...

 

My understanding was that was dictated in AIMilitaryStrategies Col AIUpdateCitySpecializations. I've turned it on for several it was off for. I also keep the Specialization REevaulation Interval at 1.

Reasonable. There are so many hilarious AI screw ups when it comes to military, it's very difficult to come up with much that will help it outside of major DLL work as you suggest.

I've tried multiple variations using the values available in the DB, and it's still a consistent trainwreck.

I made multiple unit types all false, with every variation of unit type. It's allowed it to grab whatever is available for the escort, if the settler waits, and if the unit is allowed to.

Often times Garrisons won't change over. Sometimes they do. Recon almost never switches over, although in Community Patch tests, it feels like I've seen it happen more often.

The reality is though that my reports are slightly suspect since I didn't have logging working properly. That's now fixed, so I have a much better grasp on the why something is happening. There's too much wrong with AI to guess at it.

I've shut off Warriors Recon value, and that's helped things a bit. I need consistency more than anything, and having Warriors be a random gamble doesn't help me. I know if a Scout is being built, that Recon took it over.

Obviously having proper logging now helps tremendously.

Great generals for me mostly patrol, same issue with Settlers. They get captured walking around the edge of the borders for no reason.

Great Generals, and Military pathing is just so bad that I'd rather they were used as Citidels since that code works. Not a solution, but a better than bad case.

I think a lot of options can work. I have experience with this sort of thing from Civ 4, so I immediately knew what to do.

I can with flavors force it to do an optimal route, but it's not elegant at all. It's a crappy brute force implementation worse than what you are suggesting with Lua script.

And Terracotta Army is a garbage wonder, and I just set it's flavor to -999.

It was horrible in Civ 4 too. There are problems everywhere in here, they just don't give it the kind of time and repetition that we do. It's a passion for us, and a job for them.

I had this set, and it didn't work for me. I opened up the config and randomly turned on other options, and it finally worked. I don't know if it was a cache issue, or some other option that is required. I don't care to find out, it works now, and I am very grateful for it.

So much is wrong with any AI attack, that something that useful would be irrelevant anyway.

I've been leaning more and more towards the AI beelineing for Air Units, since it's the only units it can use effectively without huge losses. Or perhaps I haven't caught a situation for how bad the AI air is.

I do question rebaseing from what I've seen...

If I run a turn automatically, and repeat this with nothing queued in a city, nothing ever queues, and no hammers are held. This was what started me on that as an issue. I don't know that it is infact an issue outside of testing turn by turn.

It would take an extra turn to buy something, which would be a waste. It would be great to decouple purchasing from the build order.

That's not all, the writer's guild has a 100 building flavor. Just ridiculous.

Thus, I just force the AI down the path a Deity player would take. This has negatives.

In Civ 4 in python, I had if then for the resources around the player. I don't have if then in the SQL, so I have to literally just go down the tech tree in an order I think will mostly cover everything.

Thankfully, it doesn't really matter since every AI worker will be stolen by barbarians, so putting off resource techs in the tree is almost meaningless in reality.

I agree with you, this was before I had logging working. It has helped though when I watch games, so either it's chance (possible), or moving it to Tactical AI is actually a far better decision from far out (possible)

Now that logging is working, my biggest issue there is that the AI considers it's army worse off than the barb camp with multiple units. I think that's just crazy. A competent player can take out a barb camp with ranged units not engaging.

I can't get the AI to engage the enemy often after making the trip towards it, which is frustrating.

There's an option to make them do it in the SQL. They seem to engage the Barbs more with it set to on. It may not actually work at all, I will have to do more testing.

I have significantly increased early game unit creation. Usually the City States don't have enough units to engage in an operation, but if it's turned on, and they prioritize units, they seem to engage the barbs away from their territory more.

Could be a fluke.

That would be the ideal situation, but that's not really required in my opinion to drastically improve it. The AI simply won't engage the barbarians in a meaningful way.

In my opinion, if there is any barb, all non-civilians without a current mission should engage them. That would solve it already in my view.

You're right again, but I don't think it has to be as good as you indicate. All the AI has to do is send it's units. Barbs can't heal. If they attack, with ranged first, then melee, then heal and repeat, issue is over.

Barbs get out of control because the AI doesn't kill the camps, and instead sends caravans directly for Barbs.

Which is another topic, the AI needs to stop sending Caravans on routes that include barbs in the path. It's just ridiculous to watch.

They are patrolling. It's in the logs.

The problem is, when units Patrol, they don't even engage targets often. My best AI games involve .5 for homeland units, because they are absolutely terrible at dealing with anything. Better to just keep them in missions.

You may have this fixed now, but I watch this stuff, and it's just dreadful. Garrisoned units are horrible, because they don't do anything useful either unless they are ranged, or city_bombard, and the barb happens to walk close enough the City could have handled it.

Just a mess to watch. All fixable, but it's a long slog as you surely know.

 

Question for you, is there a way to shut off AI plot buying, or make it rare (natural wonders), and extremely competitive tiles.

I have the AI going full tradition, and thus they don't really need to buy plots. I'd rather the money saved to rush units and buildings, and keep them out of the negatives as much, allowing them to not go GOLD crazy.

Any thoughts there?