I cannot understand, how is Ai now trash when there are literally AI cores being integrated into even smartphones CPU-GPU cores???
I'm not a software engineer, what difference there is between AI "tokens" and old school floating point raw calculation power...
I can't grasp this paradox.
As magicq99 says above, there is no underlying logic or consistent reasoning being used when these models are used - they are simply exceptionally good at being able to predict a similar output given a massive amount of input -> output examples to analyse in the learning stages. This means that they will only be able to predict output similar to that which was included in their training data - for instance, if your training data is everything ever posted on a forum for an emo band from 2005, the output will always be text that sounds like that mid-2000s era subculture. If you asked the model to write like a Shakespeare character, it probably wouldn't have enough mentions of the word Shakespeare to form a meaningful statistical association with other words, and would just end up sounding like the same mid-2000s emo writing, maybe with some slight difference it had learnt from people posting about Shakespeare there.
If we're now applying it to a 4x game's AI, what is our training data? Remember, these are
not undertaking reasoning, so they have to infer everything from a significantly large enough basis of training data. If you give it something simple, and with objective measures of good and bad choices, it can learn it quite quickly. So, for example, if you give it a settler on turn 1 of the game and give it data on all reasonable places one could settle within the first ~3 turns, it could likely learn from that as to which was an appropriate place to settle based on some scoring of 'goodness' you give it. That scoring metric will massively change what it thinks is good, however. Is that metric the sum of all yields after 15 turns? In which case it will almost certainly miss the relative importance of different yields. It's easy to get a bunch of food, but it's not that important - and it's hard to get science and culture, and their benefit doesn't show in 15 turns, so it will massively undervalue those. Do you bias it, and give it a scoring metric of "the sum of total yields after 15 turns, weighted by what I think their importance is"? Then it might appropriately value +1 science more than +1 food, but it's also dependent on what your views on the game are. Is science more important than culture at the game start? Is that universally true, or are you biasing the AI towards your default playstyle? Maybe influence seems the most important stat to you because you always play city state-heavy games, but it might be less relevant in other games. Regardless of which of these you do, if you're cutting it off after 15 turns, you're absolutely going to get a result where 0 military troops are trained, and probably minimal scouting is done - they probably don't impact yields in these first 15 turns meaningfully. If you cut it off later, after say 50 turns, you then have to try and figure out if you can get a statistically significant effect on those turn-50 yields from how you settled your capital without other factors dominating. You can entirely do that if you provide enough examples to analyse, but the more things you're varying, the more data you'll need to be providing - learning which hexes within a few turns is best to settle on for a specific map is going to be pretty easy. For a random map, you'll need more data. To learn meaningful differences for different map generation scripts will require a tremendous amount of data. To learn meaningful differences for different map generation scripts aND taking into account the civ and leader you're playing will require even more data. How do you gather all that data? Do you ask for playtesters to play out hundreds of thousands of matches where everything is tracked to learn from? If so, when do you do it - once the game is otherwise fully complete, delaying the release significantly? Do you do it earlier and hope that what is learnt is still appropriate for the release version of the game? It's difficult to automate the generation of this data, because there's no AI to run these civs without the end result of this training, so we can't really assume it's available beforehand. If we don't provide enough data, you'll either get the generated civ AI missing important factors about non-standard conditions (e.g. it has learnt that you shouldn't have too many scouts, but still follows that rule when playing a civ with a useful unique scout where you should have more), or it will overfit from limited data (e.g. it will learn that if you play as a civ with a unique infantry, you should always spam them at the expense of your economic development, because the few games with that combination of civ + other parameters happened to spawn near a bunch of weak civs and conquering made more sense).
There's also the question of how you train these models; in the example above, we were just looking at the impact on total yields in a very brute-force metric, but that doesn't give the AI a lot of variables to try and determine why these results are the way they are. For instance, do you train these models providing inputs like "granary built turn 10 giving +4 food", or "at a cost of 60 hammers, a granary was built in the city centre, providing a +4 food bonus after adding in the +3 warehouse bonus"? The second will require significantly more computation if you do that level of detail for everything, but it might allow the model to learn something a bit deeper - that the granary is a good choice here because of the large warehouse bonus specifically. The first is much cheaper and faster, but it's less flexible to change - if you nerf granaries in patch 1.1 by making them cost 80 hammers and only making their bonus apply on adjacent farms, the first model will have next to no way of altering ints predictions given the new conditions. You'd have to retrain it entirely to re-learn whether a granary is worth it under these new conditions. However, using the second one requires more information as time goes on - building a building in the modern age that has great work slots in it has a value dependent on the raw yields of the building, the adjacency of the building, how many great work slots you currently have available, how many great work slots you'll need for the game overall, whether you're trying to win a cultural victory, whether you've discovered the relics in that continent, and probably more I'm not thinking of that right now. And each one of these factors adds to the computational cost - and to the size of the training data you need to provide it for it to be able to tease out statistical correlations between all these things. And again, that training data all needs to come from somewhere.
Frankly, I don't think using machine learning models to create the entire AI for a game as complicated as a 4x is a worthwhile endeavour. What could make more sense is to have an underlying logic-based AI like we have now, and then tweak the parameters based off of the results of machine learning. If the ML model doesn't have to learn what the benefits of every possible place your first two scouts could move is, but just needs to learn the effect of tweaking the `strengthDifferenceToAllowWarDeclaration` parameterer, there are so many fewer parameters to be learning that the amount of data required is somewhat manageable, that data can be generated automatically from games run entirely automatically by the logic-based AI, and the logic-based AI means there's always a layer of some degree of logic being used in the process. Realistically though, this is not what is needed to fix the AI in 4x games - it's the game design to begin with. The Old World military AI is not better at doing 1UPT 4x combat than the civ 5-7 AIs because of a complicated machine learning technique they did on their AI, but because the game design was built with AI in mind from the ground-up. The orders system allows for massively larger movements to be occuring for a single unit in a single turn, which means the AI doesn't need to try to calculate tactics 5 turns in advance and anticipate potential player actions across those 5 turns; it can simply choose to invest its orders in executing that tactic immediately, which is profoundly more simple for the AI to do, and then the difficulty scaling can primarily come from ensuring the AI have more orders to work with than you do. This is absolutely not the only factor in The Old World's AI performing significantly better than civ 5-7, but the game has clearly been designed with the AI in mind. Doing so for Civ would force the removal of some fun aspects of these recent games - I really enjoy civ 6's city-building minigame, where you want to spot a strategic resource and an aqueduct location next to each other, build those and then put mines around the central spot to get a really got industrial zøne, and then do that consistently across multiple cities so you can slot the double adjacency yields on IZs policy card and get a massive production bonus. That's really difficult for the AI to do, and it would be much better for relative difficulty to replace that with a different mechanic. However, I enjoy that adjacency-finding minigame, so the question becomes: is it worth removing that to get a better AI? That's personal, but given that about ~75% of people have gotten the civ 6 achievements for doing the basic things you'll do in any given game, and about ~7% of people have beaten deity in any context (including people cheesing it with duel-sized maps stacked heavily in their favour, which I have seen discussed commonly), it seems like only about ~10% of the playerbase is playing at the hardest difficulties even with them being relatively easy if played well at the moment. Would civ 6 be a better game for the majority of the playerbase if its design were changed to be more compatible with logic-based AIs, like The Old World? In some ways, absolutely. In others, I think it might not be. It's a subjective choice dependent on what you value in the game too, and how long you expect to be playing it (as well as how much experience you have with other 4X games in general).
Still no Gullfoss or Iguazo though...200+ hours so far.
