[BTS] Upper Civ Limit of 53 in World Builder Saves/Scenarios? Going over = Runtime Error

Universe188

Chieftain
Joined
Sep 16, 2018
Messages
14
Hello all,

I'd just like to ask if anyone is aware of an absolute maximum limit for civilisations that can be loaded by a scenario.

During my little experiment I downloaded a 100 Civ DLL and placed this into a mod (there was no pre-existing DLL so nothing was overwritten). I then started a game with 70 civilisations and it loaded fine and plays well so far. I saved the game and I am able to reload this save without any issues.

The problem, however, lies in making this into a World Builder Save. When I attempted to load the .wbs, the game froze at the civilisation selection screen and a Runtime Error popped up, before crashing once I pressed OK. This seems to me as if the game cannot load scenarios with too many civilisations, even if the DLL allows for more.

I made a couple more saved games without replacing the DLL (so it was still a 100 civs DLL) but this time, I only created games of around 10-20 civilisations. All could be loaded both as a save file and as a world builder save/scenario without a problem.

I then reduced the number of civilisations in my 70 Civ WBS to 10 and it could be loaded as a scenario. I added 10 more and it was still loadable. At 51 civilisations it loaded fine as a scenario: I was able to select a civilisation and load in without a problem. The problem arose when I added the 54th civilisation, which gave me the Runtime Error. So it seems to be that 53 civilisations is the cut-off limit for scenarios.

Additionally I experimented with replacing the 100 Civ DLL with a 77 Civ DLL, and I made sure the WBS files contained 77 player entries instead of 100 (otherwise it would crash.) The same issue: more than 53 civilisations results in a Runtime Error when loading a World Builder Save.

I'm unsure as to whether there is another limit coded somewhere or if the game cannot cope with registering more than 53 civilisations in a scenario, because loading a save file itself is fine. If anyone has had a similar experience and knows of a way to overcome this, I'd very much like to know.
 

PPQ_Purple

Purple Cube
Joined
Oct 11, 2008
Messages
5,013
It could be a bug or limitation in the game but it could also be related to your system.
 

Merkava120

Oberleutnant
Joined
Feb 2, 2013
Messages
450
Location
Socially distant
Interesting. I just downloaded my 100-civ DLL and I get the same problem: Start a game with 70 civs, save a WBS in the game, exit, load the WBS file (with the correct mod path), and it crashes. The crash dump gives error code 0xe06d7363 which is apparently an "unknown software exception". Edit WBS file to have 51 civs, load it, and no crash.

I don't have time to see where my exact "limit" is, if it's also 53, but some thoughts:
  • I have all of the debugging options turned on, do you? In the past I have had debugging options cause crashes when the normal game does not.
  • You said this happens with the 100 civ DLL and with the 77 civ DLL. I can't speak for the 77 civ one, but I made the 100 civ one, and I have noticed that leaving the DLL in debug mode when you compile it - instead of "release" - can result in weird crashes that don't happen in Release mode, which could be happening here. I don't remember if I uploaded this one in Release or not, but if you make a DLL in Release mode and try this all again, do you still have a 53 civ limit?
  • Interesting note: there are 52 leaders in Civ4 BTS. Could this be an error with the scenario "choose your civ" screen trying to display more leaders than exist?
Later tonight I will fiddle around some more and see if my limit is exactly 53.
 

f1rpo

plastics
Joined
May 22, 2014
Messages
1,318
Location
Germany
The Barbarians have a leader too, so there are technically 53. Are the WB saves that you two are experimenting with compatible with unmodified BtS – apart from having more than 18 players? I'd like to see the context of the crash in the debugger. (Though I rather doubt that the problem can be worked around.)
 

Merkava120

Oberleutnant
Joined
Feb 2, 2013
Messages
450
Location
Socially distant
EDIT: Removed my original reply because it was wrong...yeah, I'm still getting the problem. Created a WBSave with like 64 civs in a 100 civ DLL I just made, then tried to reload the save from within worldbuilder and got a crash.

I did narrow it down to 53 civs being the key number, just like above. I also eliminated the amount of leaders as a potential issue - having an extra leader, fully set up, does not stop the crash from happening. I also counted the player colors; there are 44, so those can't be the source of the problem.

The crash happens right when the screen is about to load the list of leaders for the player to choose from when they are opening a scenario. That and the suspicious number 53 make me think it still has something to do with leaders but I have no idea.
 
Last edited:

Merkava120

Oberleutnant
Joined
Feb 2, 2013
Messages
450
Location
Socially distant
The Barbarians have a leader too, so there are technically 53. Are the WB saves that you two are experimenting with compatible with unmodified BtS – apart from having more than 18 players? I'd like to see the context of the crash in the debugger. (Though I rather doubt that the problem can be worked around.)

Yeah, the ones I'm using I just edit the line that says "Mod=" to point to wherever the DLL is and that's all. Interestingly, if you actually go above the Civ limit (like loading a WBSave with 50 civs in vanilla BTS), you get a Python exception, not a crash.
 

f1rpo

plastics
Joined
May 22, 2014
Messages
1,318
Location
Germany
I was hoping you might upload such a WBSave. Well, I see that one can just give every player the same civ and leader type, so it's actually taken me just some 10 minutes to make my own WBSave (attached, requires a 60-civ DLL). However, I don't get a crash with this, so it would still be helpful if you could upload one of yours. I've tried it with the AI Auto Play mod because I happen to have a development environment set up for that. It's essentially the vanilla BtS DLL, so I don't think it's because of the mod that I'm not getting a crash. Haven't tried clicking "Launch", but I can select the 60th player without a problem.
Interestingly, if you actually go above the Civ limit (like loading a WBSave with 50 civs in vanilla BTS), you get a Python exception, not a crash.
That exception should come from the WB parser, which is written in Python and can be modded. I've changed it (in this Git commit) so that e.g. a 50-civ DLL can load scenarios written for an 18-civ DLL. (The other way around can't ultimately work, obviously.)
 

Attachments

  • GreaterThan53.zip
    33.5 KB · Views: 15

Merkava120

Oberleutnant
Joined
Feb 2, 2013
Messages
450
Location
Socially distant
All right did some more tests.

First, since I had to reinstall my main machine anyway (I've been using my laptop for modding until recently) I put a fresh install of Civ4 without any settings changed at all to see if that would affect anything. Nope, still crashes with the 100civ DLL and a WBSave with 60 civs (attached finally, meant to do this above lol)

Second, I downloaded your WBSave and tried it, and at first it just gave me 18 civs, but then when I changed the mod path to my mod with the 100 civ DLL it....works fine.

So it's gotta be the WBSave files somehow??

Edit: thinking about it, I got errors when trying to use my 60 civ WBSave in unmodded Civ on my main PC just now, but I did not get errors when trying to use your 60 civ WBSave in unmodded Civ on my main PC, it just gave me 18 civs. Wut?
 

Attachments

  • GreaterThan53Merkava.CivBeyondSwordWBSave
    473.5 KB · Views: 16

f1rpo

plastics
Joined
May 22, 2014
Messages
1,318
Location
Germany
After noticing that the LeaderName assignments can make a difference, I finally remembered an old post that had already established that this is the problem with large scenarios: link
(I had remembered some post all along but wasn't able to find it because I thought it was a recent one – and I didn't remember its conclusions.)

In your file, if I use automated search to replace
LeaderName=TXT_KEY_LEADER_
with
LeaderName=
and
CivDesc=TXT_KEY_CIV_
with
CivDesc=
the crash no longer occurs. Not specifying any names in the WBSave doesn't seem to help: still crashes when all LeaderName and CivDesc assignments are commented out (by replacing the same search strings as above with the comment character #).

This calculation might be accurate when the civ descriptions take up a lot of space:
In the 40 civ save I've posted, the names seem to be limited to an average of three characters (kind of makes it hard to know who's who).
And here's another post in that old thread calculating the number of valid characters. Looks like they just weren't aware that the civ descriptions count too. (I don't think any of the other string variables matter; can't avoid a crash by setting CivShortDesc or CivAdjective to shorter strings.)

I'm getting a crash when the total length of the LeaderName and CivDesc strings exceed 1239 characters. Perhaps the menu uses one of those 2048-character buffers that the DLL uses in a few places, e.g.
Code:
bool CvString::formatv(std::string & out, const char * fmt, va_list args)
{
   char buf[2048];
   // ...
Although 1240 ASCII(?) characters is nowhere near 2048, and putting all those names in a single buffer seems a little strange. Civ and leader name appear together only underneath the portrait: "Saladin - Arabian Empire"

1239 for 60 players isn't too bad. Actually just setting
CivDesc=TXT_KEY_CIV_AZTEC_SHORT_DESC
instead of TXT_KEY_CIV_AZTEC_DESC (for all civs) in your file is enough to get it to load, without touching the leader names. (Case-sensitive search and replace SHORT_DESC --> SHORT_DxESC, DESC --> SHORT_DESC, DxESC --> DESC)
The DLL could even restore the longer names once the game has been launched (or perhaps already once the human civ selection is through). In my own DLL, I've at least added an assertion that warns about this issue (edit: Git commit implementing this assertion):
assert-total-length.jpg

Oh, as for the context of the crash: Uncaught std::runtime_error in KernelBase.dll. Next up on the call stack, we have msvcr71.dll (VC++ standard library) and then a bunch of frames in the EXE. The disassembly there isn't illuminating (for me).
thinking about it, I got errors when trying to use my 60 civ WBSave in unmodded Civ on my main PC just now, but I did not get errors when trying to use your 60 civ WBSave in unmodded Civ on my main PC, it just gave me 18 civs. Wut?
Looks like it's normal that excess teams and players are ignored:
Spoiler CvWBDesc.py :
Code:
print "Reading teams desc"
filePos = f.tell()
self.teamsDesc = []
for i in range(gc.getMAX_CIV_TEAMS()):
   print ("reading team %d" %(i))
   teamsDesc = CvTeamDesc()
   if (teamsDesc.read(f)==false):       # read team info    
       f.seek(filePos)                   # abort and backup
       break
   self.teamsDesc.append(teamsDesc)
print "Reading players desc"
self.playersDesc = []
for i in range(gc.getMAX_CIV_PLAYERS()):
   playerDesc = CvPlayerDesc()
   playerDesc.read(f)                   # read player info    
   self.playersDesc.append(playerDesc)
The exception when loading your WBSave comes from here:
Code:
"read in a plot desc"
# ...
v = parser.findTokenValue(toks, "TeamReveal")
if v!=-1:
   for iTeamLoop in toks:
       iTeamLoop = iTeamLoop.lstrip('TeamReveal=')
       if len(iTeamLoop):
           # "IndexError: list assignment index out of range"
           self.abTeamPlotRevealed[int(iTeamLoop)] = true
   continue
That is, the plot descriptions refer to teams that aren't among the 18 that have been loaded. My test WBSave didn't have any team or player data in plot descriptions.

Edit: Added link to Git commit implementing the assertion
 
Last edited:

Merkava120

Oberleutnant
Joined
Feb 2, 2013
Messages
450
Location
Socially distant
Ahhh that makes so much sense. Thanks for spending time on this one. I did notice that your WBSave had LeaderName=32 and so on throughout the file. Honestly would not be hard to make a DLL that fixes the names after loading a WBSave (or before saving it, to prevent problems), but I'm going to deep dive later tonight/tomorrow and see if I can find a way to fix the source of the issue.
 

Nightinggale

Deity
Joined
Feb 2, 2009
Messages
4,982
Honestly would not be hard to make a DLL that fixes the names after loading a WBSave (or before saving it, to prevent problems)
WB has text passers for both reading and writing in python. This means it's technically possible to create a workaround using pure python. While I'm generally for doing as much as possible in the DLL, in this case python might be preferred as that allows using an unmodified 100 civ DLL file.
 
Top Bottom