I have to jump here, since there is a lot of misconceptions about machine learning, and deep learning.
This is pretty much what I do, and sure deep learning is a nice tool that can make a lot. But is far from the best tool to use for a game like this.
Deep Learning is mostly a special kind of Neural Networks, with a set of sets of convolutional filters that extract features in the data. That is, patterns that can lead to a good description of the data that then feeds a NN, that works basically minimizing an error metric or maximizing a goodnes or similarity metric. This is very good for regression, clasification, interpolatiom, prediction, clusterization and many other problems. However not for this game. Where playing is in general less about exploring a solution space, and more about providing a believavble world and characters that have a set of goals.
To aproach Civ like a optimization problem is however, I think, posible, and probably around 1% of players play this way. It would need just a pair of years of full time AI developement, and the result would be most likely unplayable for anyone includding that 1% of players.
First, you need to understand that a civ game is not best understood as a numemical problem that needs a solution. It is a combination of a lot of small problems that need to be solved in a combination of different ways. Very few of them with an optimal approach.
Lets use an example, when going for a Science Victory, there is arguably an optimal approach to building and developing. To maximize science output, and to prevent the player from stoping said production. Every decission made regarding many game systems can be made to this end. And honestly, this is the last thing the player wants. And among othwr things will require a redesign of all civs, to negate all major advantages in certain game styles.
We dont want an AI where in every game there is a Religious Maximizer, a Science Maximizer, A War Maximizer and so on where the end goal is just a race against numbers. And no way of recovering from a bad position.
On the other hand, when approaching pathfinding and similar problems, there is a optimal approach to solve getting from point A to point B in the fastest way. This is a very old problem that can be solved optimally. And that does not require at all any Sofisticated technique. Sure, Civ UPT restriction make this problem much more difficult, but an improvement here would not be hard and will benefit the game.
Other example of a problem that should not be optimized is city building. The optimal aproach is to build as much as cities as posible, and to optimice each city distribution for the current goal of the civilization. This is another thing we dont want. We dont want the game to turn into a race for geometrical expansion in the number of cities where using a production turn for a wonder that we fancy is a mistake. This approach can work for a Starcraft, where expansion and conquest is the only goal. But not here, where the player (at least I do) stops expanding when feeling comfortable with their empire layout and builds some stuff because is cool.
There are many subproblems in a game like Civ. Trade, City Location, PathFinding, TechProgression. None of them are truly complicated and none of them require any fancy Deep Learning Aproach.
We also want, however that none of the Civs play the same, that they dont go always for the same Wonders, that they dont try to conquer City States, that they use trade and Diplomacy In a Coherent way, but not in an optimal way, since we dont way the AI to be able to backstab, or conquer their neighbours in the first turn. The best way to code this behabior are probably behabioral trees. Which is I think the system the game already uses.
The actual problem in the civ AI is, to suboptimice in a beliebable way and challenging way, rather than use the best strategy. To solve this, the developers worked very hard trying to make the Civs to make a lot of decissions the players do not take. And yes, the AI has a lot of bugs and underplays too much in many ways. For example caring too much about grievances. But overall, this is the right approach for AI. It just need minor adjustments.
When I say minor adjustments, i mean it. The players here tend to think the IA is stupid because makes some big mistakes. It does, but 90% of the decissions the AI makes are not mistakes. And are not stupid.
The AI needs to be more aggresive in wars, to plan cities smarter, to solve some issues in war, to improve pathfinding, to take into account more factors in diplomatic exchanges, to add more sofisticated combat strategies, and just some nuisance... Believe it or not, this can be done with a couple of man-month resources in dev time, as the foundations for the AI are currently solid. (Yes, I mean this too)
The AI needed more resources, and designers should have been less worried about playing safe and be more agresive and focuss on challenge. Civ VI design filosofy has this problem all over the place, and it lacked the proper care and atention to detail, not only in AI. But this is honestly quite easy to do without any convoluted Pattern Recognition approach.
In summary. There is a reason why almost no game uses Deep Learning for AI. It just almost never works and there is almost always a better way to solve the problem.
