Reseed!

More about crashes...

In post #35, I explained why crashes are unavoidable: they happen if I don't let civ5 breath, but civ5 kills your civ if it ever breathes (since your units suddenly drowned after the map script put an ocean hex under their feet). I tried to move as much things as possible in pre-generation steps that are spread across multiple frames but there is still a final burst made of three step: killing the last unit of each civ, generating a new map, placing new units.

However, I found a solution: replace the Map and Plot objects with custom implementations! This is tedious (it requires twice much work than Reseed altogether), boring, but it should work. That way the map script could run without actually modifying the map: it would only modify a dummy, pure LUA, map. Once I get a LUA representation of the new map, I will be able to first modify the starting plots, then move all units to those ones, then progressively update the rest of the map across multiple frames.

That should fix crashes for all. Also, it should allow me to prevent the "random instant game over" bug that occurs if you every founded a city. Indeed, I would always be able to add new units before I kill the other ones and I guess it will be enough to prevent civ5 to trigger a player's death. This would allow me to remove the limit I had to set up in v4.5.

you are a genius modder!!!!
cheers amigo!!

Thank you but I am just a developer. And developers can do magic.

The replacements for the Map and Plot objects just passed all the unit tests! Time to put them in action now, and see if all of that work was for something good.

So, some time ago, I saw that Perfectworld had a piece of code that used the IO functions from LUA to generate a file. Since then I always thought we could write files from mods. I thought that Firaxis had forgot to remove that or maybe they went as far as to sandbox it and put restrictions over the paths you could access.

Anyway, I started to investigate the file format of civ5 saves. It wasn't that complicated since I only wanted to write "turn 0" savegames and I was all fired up, very enthusiastic. Sure it would have taken some time but at least I would have been able to get rid of dirty hacks and all the other complications the civ5 engine introduces by bypassing it. And it would have provided me a full control over everything: I would have been able to write a true "turn 0" savegame with no religions and such, I would have been able to change the players, etc, etc, etc! I was already imagining how many cool things I could do with that, including new features for IGE.

So after two hours of work I had nice IO functions (since LUA can only write strings - oh yes, spank me hard baby), a piece of code able to write the beginning of a file and a test ready to compare my file with the one written by civ5. I hit ctrl+shift+B, alt-tabbed to Firetuner, fixed a couple of typos and... tada! No IO module. Doh! I guess Perfectworld's author was never able to run that little piece of code of him after all.

A few more hours of work to the trashbin. Civ5, I hate you.

Now this is unfair, you just gave me powerful reasons to play civ5 (when I really really shouldn't! so much to study, so little time...). Such awesome mods, Don
Can I translate this one too?

Hello.

I am glad you appreciate this mod too and I would love to see it translated. Still, it's a bit too early in my opinion: I am currently working intensively on this mod because I try not only to fix the crashes for the users who experiment them but also to figure out a way to overcome some limits of the game. As a result, the UI and the texts are likely to undergo major changes in the days to come and I want to push releases as fast as possible without waiting for localizations. The result is that if you were to translate it right now, the next versions would be released with one third of the text missing and displayed in English, and the other one inaccurate or outdated.

Now I think (and I hope for my sanity) things should quickly stabilize soon. Once we reach that step, I would welcome your translation.

Beta version with major changes

So here is the beta of v5. The engine has been totally revamped in order to try to prevent crashes for the users who experienced them (well, assuming I correctly guess what makes civ5 crash since I have about no data). This required a lot of work and the codebase probably inflated by 200% and it is almost entirely new, and unfortunately a lot more complex. This is a bad thing because it could add more subtle bugs, so I took special precautions by adding unit tests in critical pieces (40% of the codebase roughly).

Now the new code is also less hackish and civ5 can now breathe as much as it wants: the most important changes are that the map script works on a LUA structure rather than the actual map (so that I can know how the map will be modified, safely move the units accordingly, then apply the terrain changes), and that units are now moved according to a smart algorithm to prevent contacts between civilizations. At no point it is required to freeze civ5 until a whole set of changes is made, as it was in previous versions.

Unfortunately all of that also impacts speed: with the safe settings, the new algorithm is about 200% slower. But it is possible to tweak it to make it faster and reduce the overhead to 30% only (and I still have to do some optimizations). There is a balance to find depending on your computer between stability and speed so I will add a cursor to allow the user to choose between both. There are also UI improvements to be made but this beta version is only about the underlying engine.


This is a beta. Right now the settings are on the safe side, with a small batch size of 100. You can increase or decrease it by changing the value at the top of RSD_Process.lua (open it in a text editor). You will need to restart civ5 for the changes to take effect.

If you experienced crashes before, please tell me whether you still experience them with this version. Whatever the result is, I beg you, report it, along with your version of Windows and your CPU (or, if you don't know, how old is your computer). Also, if it crashes, please try a batch size of 10

Works fine for me on its own even with a batch-size of 50000, also works together with IGE. Causes Appcrash together with Civ V Unofficial Patch (G&K) v.21. Hope this helps! =)

Hello and tank you for the feedback xngen.
Can you tell me if you experienced crashes with the previous version?
Also, unfortunately, I did try civup v21 but I could not reproduce a crash. If you could provide a lua log the next time it happens, I would appreciate that a lot.

Pastebin of Lua.log

Crash seems to happen when world has been recreated and it is trying to reload the save. Only CivUp v21 and Reseed v5 is active. Using the standard Continents map with standard settings.

The crash happens in the dll of the expansion ("CvGameCore_Expansion1.dll"). And it seems to happen at any batchsize (tested 10,100,1000 & 50000). All DLC's are active including G&K.

I am using highest settings on all in DX11 but without antialiasing and have not tested at lower settings. App.log gets no info at all.

As another note, getting a "ChangeAIMapHint is null"-error on several maps that are not one of the 5 normal singleplayer maps. For example "Small Continents"-map.

Edit: Noticed CivUp had been updated to v22, same problem though. (Removed caches and old files) /Edit

Thanks again for the report Xngen.

Unfortunately, those are bad news : more of those non-sensical crashes that occur during the reloading... I will try to let more frames pass before the reloading start but I am quite desperate. I don't think it specifically happens with civup, at worst it may have an influence on the acceptable map size for your computer (not related to the CPU power in the end, related to Firaxis-only-knows-what). And, indeed, it happens when civ5 loads the game. And it's not a save corruption because you can safely load the savegame manually.

Regarding the other bug however, fixing it should not be a problem.

Finally, you forgot to tell me whether you experienced such bugs with the previous versions or not. I guess you did but I would like to be sure. Also, could you tell me you OS please?

PS: I will release a new beta within the next 24h, with the bugfix and that another desperate attempt regarding crashes.

Was trying reseed from the download-page for the first time 2 days ago, it was because I crashed I started browse the forum. I really like the idea of being able to reseed the map so I registered on forum only to be able to tell you about the problem. I could not see any error-messages in the lua-log either (been making lua-mods for MMO's before), and when I wrote my first post I believed it could have been a conflicting table or something else easily fixable.

I can reseed many times in a row when CivUp is not active, but as there are no lua-errors it seems it is a more subtle error in how the data being handled in the Core<->Lua adapter implementation. That might be why they left out the Load-option from the active game menu when using mods.

I rather have the choice of reseeding easily than a few minor tweaks on the UI so I'll skip the CivUp for now. Knowing about this error can be useful for other users though.

Using:
Windows 7 x64
12 GB Ram
Radeon HD 5670 1G RAM
+A few SATA-disks, keyboards, mice and stuff like any other computernerd.. =)


Edit: Was thinking it can have something to do with mod-reloading, as it loads mods that the save-game says is demanded. If there is a command for unloading other mods before loading the savegame it might do the trick. CivUp is a mod that is "saved", IGE is not, was just thinking it could be the difference. /Edit

Did some more testing, enabled "Load Game"-button for CivUp and started a few games with only CivUp activated. It seems that I crash every time I try to load during an active game dependant on CivUp, and I draw the conclusion that that must be the reason CivUp has disabled the "Load Game"-button. I can load the same save from the Mod-menu though.

Sorry for all the trouble, it was CivUps fault all along. =)

Also, forum-thread about it: No Load Game option while playing

@Xngen
Huh huh... I just noticed a comment on the topic you linked about a crash and I wasn't aware that some users were experiencing that. I was aware of the main problem with in-game reloads that causes bugs in a few mods but this a totally different matter. Besides, I am not convinced those crashes on reload only happen with civup (although it does on your computer), especially because I just studied civup code and there is absolutely nothing wrong or special with it. I am under the impression that for some users it will never happen, for some others it will happen without mods, and for other people it will only happen with mods X and Y, as it used to in previous versions.

So, no, this is not civup fault, it's civ5's fault and it has always been anyway (a mod is not supposed to make the game crash and it's certainly not supposed to make it crash when this mod only does regular operations). Or it's no one's fault if you consider that Firaxis' duty is only to ensure that their game works when unmodded, and not to ensure that their game, as a modding platform, is bug-free (which would be a strange twist since the game is advertised as moddable, but expectable given that modding did not get all the investements it needed and feels like a second rate citizen). Although I suspect that some users may experience that bug without mods.

And also it means I cannot just wash my hands with a "oh ! it only happens with civup" (even if it was) or a "well, it's a civ5 bug anyway", I still have to find a way to avoid that (although it will always be troublesome with a few mods like CivUp but for other reasons). I will release the next beta soon. I know you already did a lot, and I am grateful to you for that, but if you could still give a try to this next beta (no need for a lua log), I would appreciate.


PS: So it's not OS specific. I do have Win7-64 too. And if it matter an ATI card too, although a more recent model.

You're absolutely right! I meant it was CivUp causing it for me, not the CivUp-developers screwing up. There are alot more to find about the bug on the internetz.

Ofcourse I'll try the next beta if you think it will help! =)

Thank you for your cooperation, Xnview, this is really great to see people involving themselves in the debugging process.

So here is the beta2. Asides of small internal changes, it mostly adds a one second delay before saving, and another one before reloading. This is just a blind guess to be honest, to test one more unverifiable hypothesis about how civ5 works and what may causes the crash. Unfortunately, without the civ5 code and windows crash reports, this is the best I can do (well, technically, I could spend a few hundred hours trying to understand the civ5 code by looking at the asm code but it's not going to happen).

Yes, it still works for me in beta2, seemed somewhat quicker but sometimes the perception of time can fool you. =)

CivUp ofcourse still have the same problem with or without reseed, I think the only way to change that is resetting all globals to what they were set to in the mod-setup/load game screen. But that might be an impossible task if some references and/or hooks no longer exist while in an active game. The core dll might also have different codepaths ingame and inmenu that makes it a firaxis problem altogether.

English is not my native language (Swedish is), but I try to avoid misunderstandings. =)

I apologize if my problem is because of stupidity, but I can't seem to get the mod to work.
It's the only one I have installed.
I play GodsAndKings with a legit version.
I can load the mod through the MODselection and I get to chose SINGLEPLAYER and then SETUP GAME and LOAD game as the only options. From there it works as it usually does. And I have no reseed!-button in my diplomacy screen.
In the top right corner I have four buttons:
additional information (small)
Social policies (large)
Advisors (large)
Diplomacy (large)

Is the mod loaded at all?

I hope there's a simpel solution to my problem.
best regards

@Xnview
I am not sure how I have to understand your first sentence: do you mean that no longer get crashes with civup? That would be great news but I guess it's not what you meant. ^^

On the speed, this version should actually be slower. The next one will be faster, though, I added some optimizations.

Regarding your assumptions about global variables and reloading, you are correct. Mods like CivUp would need to serialize their variables before the reload occurs, and deserialize them after that. But not only civ5 exposes no "before save" event, but also mods can only serialize strings and that would make the serialization fairly heavy to implement for some mods.

Finally, no worries, I am not a native English speaker either and I have no difficulties to understand you, quite the contrary (probably because you may be yourself a developer or at least someone with good technical skills).


@Sakhmet
Hello.
You need to click the small button, additional informations. A menu will pop up and "Reseed" should be in that menu. There is an image on the mod page on the Steam Workshop that shows this menu with the "Reseed!" item.

Now, if you do not see this item on the menu I just mentioned, then Reseed was very likely not loaded. You could provide me a lua log if you want to be sure whether the mod was loaded or not (see post #2 to learn how to make a lua log) but I guess it wasn't and you just need to carefully read the installation/usage steps again. Did you manage to use other mods recently?