[Development] Code Rewrite

[Leoreth]

Remember unidentifiable C++ exception error messages that crop up every once in a while? I finally figured out why they're happening and fixed the root issue.

 

[Temüjin VIII]

Remember unidentifiable C++ exception error messages that crop up every once in a while? I finally figured out why they're happening and fixed the root issue.

I'm curious. Is this related to the recent commit you pushed? What exactly causes that occasional C++ exception?

 

[Leoreth]

The latest commit has a bunch of stuff, I don't bother keeping the commits limited because I fix many small errors along the way that I run into. The main purpose of the commit was to fold the old Areas stuff (i.e. getCoreArea etc.) into the Core module. But yes, the fix is included there.

The error came from a function that flipped units from one player to another. Internally this works by destroying the units for the previous owner and then recreating them for the new owner. I've known for a long time that this error came up rarely when a unit that is being destroyed is on an invalid plot (i.e. not on the map), but I could never explain it, because if you check beforehand, it is where it's expected to be.

The explanation is that when a unit is destroyed while carrying cargo, the cargo is destroyed along with it (makes sense, e.g. ships sinking). But in those rare cases, the cargo units were still in the list of being flipped, even though they were already destroyed. In other words, they were references to units that were already invalidated. I rewrote the code to skip killing units in cargo and determine the list of units to be created beforehand.

 

[Leoreth]

The plan for this is to merge the current state of the rewrite branch into develop very soon. It's currently very stable, and only missing some last finishing touches for the last thing I am working on.

This is before any changes to autoplay and rise and fall. Initially, I wanted to take a break from this at this point and return to the new map and bugfixes. But I wasn't sure if I wanted to keep this code in a separate branch or not. But now I don't see any benefit from holding off on letting people play with it and discover additional issues that aren't discovered during autoplay and short trial games.

This means that I will also fix the previously reported bugs on top of all these changes. This is because it would be pointless to fix issues in the old code first and then attempt to port them forward to the new one. Savegame compatibility has been maintained and the bug either is reproducible with the rewrite (in which case I can still fix it) or it isn't (in which case it doesn't apply anymore). This should save a lot of time for me and is also a good way to make use of the benefits the rewrite is meant to provide.

 

[Leoreth]

The rewrite branch has been merged into develop.

 

[Leoreth]

Updated roadmap:

0. Fixing bugs (unrelated and related to the rewrite)
As I mentioned earlier, I will fix all bugs in develop based on the new code. Unlike usually, I will try to address issues in reverse order, starting with recent ones. This way I can hopefully address problems that come from the recent changes immediately.

1. Clean up and improve autoplay
The current code for autoplay is more complicated than it needs to be, and I would also like some additional functionality, like:

1.1. Minimal autoplay code
1.2. Easy functionality to enable autoplay for n turns even after spawn (for testing AI only games)
1.3. Maybe even an observer mode that does not require to pick a specific civ to run AI only games​

2. Clean up and improve Rise And Fall code
I've talked about this a lot before, essentially throwing out all the old rules and implement them as needed from scratch, including:

2.1. Remove unit defections, replace with other mechanics to protect new civs from easy conquest
2.2. Additional mechanics to help some civs on spawn, e.g. when you want them to expand quickly
2.3. Greater differentiation around spawn situations based on historical circumstances
2.4. Consolidated and more generalist code for conditional spawns
2.5. Consolidated and more generalist code for civ specific pre-spawn preparations and advantages on spawn​

3. Transition from scenarios to custom games and improve tooling for multiple start dates

3.1. Game starts from Play Now / Custom Game instead of Scenarios
3.2. Start date becomes a game setting instead of a consequence of the scenario you pick
3.3. Only one file describing the initial state of the map
3.4. Differences between the initial map and later start dates implemented in separate files with scenario infos or via Resources.py etc.​

4. (Optional) Rewrite of Victory.py where victory goals are decoupled from civs but instead goals themselves are a proper unit, e.g.:

4.1. The goal check itself, hook for the event handler, texts, progress display etc. are grouped in goal classes/objects
4.2. Mechanism to group multiple subgoals into one goal
4.3. State of the goal is self contained and does not need to be tracked separately​

As usual, the goals here are long term: code easier to maintain, more consistency, easier to add or change civs, goals, start dates, and so on. The Rise and Fall changes are the exception in that they would actually change how the game behaves.

 

[BaneFire]

For all the flashy graphics of the map thread and high minded discussion of the civ and mechanics threads, this one is always my favourite to see an update in

 

[merijn_v1]

In the core module, both class PlotFactory and class Plots have a core() member function. IIRC, plots.core(iPlayer) returns the core defined in Areas.py and plots.all().core(iPlayer) returns the core that is used in the savegame. (with plot.isCore(iPlayer)). Is this correct?

Most functions use the method from the PlotFactory. That means that applying changes to the core (by plot.setCore()) will not effect these functions.

 

[Leoreth]

Interesting. It's probably a consequence of me trying to keep function names short and to the point that makes this ambiguous. Plots.core came first and basically means "select all plots that are currently in core for the player". Conversely, PlotFactory has a bunch of functions as convenient wrappers around core, normal, broader etc. areas which were added later. I didn't really consider that those identical names suggest their behaviour should be the same, especially because many PlotFactory functions wrap Plots functions.

You're right to point out that using plot.isCore is the preferable solution and better than going through Areas so that things remain dynamic but I would have to check where PlotFactory.core is used to see if that really is intended in all cases where it is used.

 

[Leoreth]

Updated roadmap:

0. Fixing bugs (unrelated and related to the rewrite)
As I mentioned earlier, I will fix all bugs in develop based on the new code. Unlike usually, I will try to address issues in reverse order, starting with recent ones. This way I can hopefully address problems that come from the recent changes immediately.

1. Clean up and improve autoplay
The current code for autoplay is more complicated than it needs to be, and I would also like some additional functionality, like:

1.1. Minimal autoplay code
1.2. Easy functionality to enable autoplay for n turns even after spawn (for testing AI only games)
1.3. Maybe even an observer mode that does not require to pick a specific civ to run AI only games​

2. Clean up and improve Rise And Fall code
I've talked about this a lot before, essentially throwing out all the old rules and implement them as needed from scratch, including:

2.1. Remove unit defections, replace with other mechanics to protect new civs from easy conquest
2.2. Additional mechanics to help some civs on spawn, e.g. when you want them to expand quickly
2.3. Greater differentiation around spawn situations based on historical circumstances
2.4. Consolidated and more generalist code for conditional spawns
2.5. Consolidated and more generalist code for civ specific pre-spawn preparations and advantages on spawn​

3. Transition from scenarios to custom games and improve tooling for multiple start dates

3.1. Game starts from Play Now / Custom Game instead of Scenarios
3.2. Start date becomes a game setting instead of a consequence of the scenario you pick
3.3. Only one file describing the initial state of the map
3.4. Differences between the initial map and later start dates implemented in separate files with scenario infos or via Resources.py etc.​

4. (Optional) Rewrite of Victory.py where victory goals are decoupled from civs but instead goals themselves are a proper unit, e.g.:

4.1. The goal check itself, hook for the event handler, texts, progress display etc. are grouped in goal classes/objects
4.2. Mechanism to group multiple subgoals into one goal
4.3. State of the goal is self contained and does not need to be tracked separately​

As usual, the goals here are long term: code easier to maintain, more consistency, easier to add or change civs, goals, start dates, and so on. The Rise and Fall changes are the exception in that they would actually change how the game behaves.

So, I don't think I have said this outright but I am now back to working on this. The plans have changed slightly since then:

• Point 0 is now done for the most part, of course there are probably still bugs somewhere but it seems we are past the growing pains phase of the transition.
• Instead of moving on to what is point 1 above, instead I decided to do point 4 marked as "optional" first. I realised this is actually necessary to untie civs from slots completely and will also solve some other problems, and I will learn things that are needed later on.
• Then, there's another additional thing I will be doing. When fixing some issues, I noticed that the game at this point has many different "scripted spawns" of various types, for example: time based conqueror events (Greeks, Romans, Seljuks...), contact based conqueror events (New World conquistadors, Mongol conquerors), building/tech based conquerors (Trading Company event), tech based settlers/conquerors (Russian far east and American west coast), maybe more that I am forgetting right now. While they are different to some extent (e.g. what triggers them and if the player can benefit from them), they have a lot in common, such as figuring out where to place units in a way that makes sense. However, they all use completely different code which makes fixing issues hard. Also almost every one of them does one thing better than others, for instance the Mongol conquerors had a neat algorithm to make units spawn at the borders coming from the right "direction". Unifying these events should simplify the code and improve at least some of those events by bringing them up to par with the rest.

So, taking the list from above, the new priority list is:

1. Victory.py refactoring
2. Conqueror events refactoring
3. Clean up and improve autoplay
4. Clean up and improve Rise and Fall code
5. Transition from scenarios to custom games

 

[Leoreth]

This is still happening. I can finally say with good confidence that the victory rewrite will be completed in the next weeks. I have completely implemented every UHV, and now I am working on the goal descriptions and progress displays. I also still have to rework how the overlays and tile tooltips for area based goals are implemented.

Then what is also missing are all URV goals, which have not been specifically implemented yet, but mostly follow the same patterns as the historical goals, and should be completed quickly once everything else is in place.

 

[rmontaruli]

 

[Leoreth]

Best failed attempt at automatically generating goal descriptions, for the second Roman goal: "Control two Iberias, two Gauls, a Britain, two Africas, four Anatolias and two Egypts in 320 AD"

 

[merijn_v1]

Single-game-multi-map-mode confirmed.

 

[hnrysml]

As long as there's only ever one Britain we'll be fine

 

[Leoreth]

if you vote SNP you can reduce it even further

 

[Leoreth]

How about spreading 30 Islams to the world's population?

I am just now realising that these anecdotes are a way to track my progress through the goals.

 

[1SDANi]

How about spreading 30 Islams to the world's population?

I am just now realising that these anecdotes are a way to track my progress through the goals.

I read that as you proposing 30 denominations of islam and wondered how posting in the dumb ideas thread helped track progress

 

[Leoreth]

Spreading Islams is actually what made my progress grind to a halt. The final special case that couldn't be worked around anymore.

 

[Leoreth]

I am done autogenerating the descriptions for all UHV goals now, barring some small errors such as incorrect plurals etc. that I can easily fix later. I don't know yet if you can set the description in the civ selection screen at the beginning of the game in this way too, but if possible, it will be a lot less likely that the description can diverge from what the actual code does.

When I go back to this, I will do the same for the bullet points for the individual parts of a goal, but that should be simple at this point because it uses the same building blocks as the description itself.