• Civilization 7 has been announced. For more info please check the forum here .

New Policy Idea for Bug Reports

Recursive

Already Looping
Moderator
Supporter
Joined
Dec 19, 2017
Messages
4,866
Location
Antarctica
Hi everyone,

Sorry for my disappearance, I had to deal with several personal crises and then got sick, but I did manage to close out the last VP Congress session. I'm feeling better, back in action and working on the next version.

I was looking over our large GitHub issue backlog (which would be significantly larger were it not for @axatin's considerable bugfixing, thanks again) and concluded that one of the reasons why I personally feel a lot of mental resistance to fixing some bugs is modmods. In general, I've noticed a pattern that issues from players who use a lot of mods tend to go unaddressed for much longer.

To be specific, here is the problem in more detail. When I encounter such a report, I have to:
- Go through every modmod listed.
- For each modmod listed, go to CivFanatics or Steam and download the version. Sometimes waste time trying to find the modmod only to find it has a different name, is unavailable, can only be found on page 22 of a forum thread, or only the latest version is available while the player was using an older version, sometimes one explicitly marked as incompatible.
- Activate all mods, in addition to the version of VP I'm using to debug, and wait for them to load.
- Try to load the savegame. About half the time, I discover that there's additional mods that weren't declared. Sometimes it'll be listed when trying to load the savegame, and Civ V won't let me load it. Other times, the undeclared mod causes a CTD when trying to load the savegame, wasting all of the previous effort.
- Sometimes because Civ V handled mods poorly, it randomly crashes on one loading attempt and succeeds on another one.
- Then go through all the work required to solve the bug normally.

I can't speak for the other developers, but I find this process exhausting and it reduces my speed of bugfixing greatly.

As a result, my idea is as follows: if an issue needs a savegame to debug, the player must upload the folder of all the other mods they're using to their bug report. This folder is located at My Games\Sid Meier's Civilization 5/MODS.

This idea really personally appeals to me because the purpose of the GitHub issues tracker is to debug problems with core VP, not modmods.

Potential challenges:
- Players may not know when an issue needs a savegame to debug. In general, anything which is simply a text or database issue doesn't need a savegame (along with some Lua files), but it's never unhelpful to have as a resource. By default, we ask for a savegame, because players are not developers and they may not know in what cases a savegame is required.
- Players with bad internet connections or who find such a requirement overly arduous may simply not report issues if this requirement is in place.

I feel like if this policy were implemented, it'd be a lot easier for developers to address issues, because the process becomes:
- Download the uploaded MODS folder.
- Activate all mods in the folder, in addition to the version of VP I'm using to debug, and wait for them to load.
- Try to load the savegame. If it fails, slap a "Cannot reproduce" label on it and close it. Should probably try two or three times because of Civ V's glitchy loading.
- If it succeeds, go through all the work required to solve the bug normally.
- Delete the downloaded MODS folder and move on to the next issue.

I would like to hear the thoughts of the community on this idea (which is not a policy yet), particularly our other developers. @Gazebo @ilteroi @Rekk @axatin @pineappledan

I'm not putting this to a community vote as it should really be a developer decision, but feedback/discussion is welcome.
 
Last edited:
I think this is a good idea, it would really save a lot of time and make the bugfixing less tedious. Asking for the mods folder is usually what I already did when I wanted to fix a bug and a lot of modmods had been used.

Apparently some mods are too large to upload on github, for those it would be required to provide a link to steam/cfc instead.

On a related note, if any modfiles have been changed manually (CustomModOptions.xml for example), it should also be required to upload those.
 
As I've hinted on Discord: I'm ok with it but will take a long, long while for me to upload the current version of 3/4 UC. As axatin said above - I'll probably just upload all other mods (small file sizes so my bandwidth can handle it) and will just link to the version of 3/4 UC I'm using (very big file!)
 
How about a requirement to reproduce the bug without mod mods? If you can't then maybe it's a problem with mod mods, not VP itself.
That'd be hard on the players and unfun, I feel like. Not everyone has the skill set for reproducing bugs, and it's possible that the bug is created by unusual circumstances which would be difficult to replicate in an unmodded game. This is a compromise.

As I've hinted on Discord: I'm ok with it but will take a long, long while for me to upload the current version of 3/4 UC. As axatin said above - I'll probably just upload all other mods (small file sizes so my bandwidth can handle it) and will just link to the version of 3/4 UC I'm using (very big file!)
I think linking to the download link for a modmod is acceptable as an alternative if it's really large.

On a related note, if any modfiles have been changed manually (CustomModOptions.xml for example), it should also be required to upload those.
Agreed.
 
i feel like this is inefficient, uploading gigabytes of junk to github.

the minidump should be good enough analyze most crashes?

also, a little effort from players is not expecting too much? if somebody has a huge mod list, they should try and do binary search to find the offending one which triggers the issue.
 
It is absolutely required if (and only if) you want also to fix the modmods, which I personally think is not the task of VP's devs. If any modmod causes issues with VP it should be up to the creators to fix.

If on the other hand you will take that load on your shoulders this change is indeed necessary. In the bugs submitted by myself I'll already include my modmods, however I am not using large ones or that many so it is not a hassle.

re:junk uploaded to github
the owner of the rep (gazebo?) should be able to set at timeframe after which solved issues are outomatically deleted if space in the rep would become an issue

Regards
XSamatan
 
Does discord have a storage solution? Or another service? I guess GitHub could but it’s a bit trickier. My thought on this is that those of us blessed with fatter bandwidth pipes could work to keep a working zip file repository of the most common mod mods with easy access to devs for this purpose.
 
this is what I do right now (though I have missed a few from time to time which I know has urked recursive).

Frankly its even easier for me as the bug reporter. All you do is zip up your mods folder, and then just upload it into every report (I just keep the zip around for whenever I need it). Its just plain easier than having to list out your mods, and if its easier on the developers, its a complete win win to me
 
also, a little effort from players is not expecting too much? if somebody has a huge mod list, they should try and do binary search to find the offending one which triggers the issue
I think it is, in most cases this is not possible as the save won't load with different mods. And most people won't even try.

I don't see why not, if someone really really can't do it, they can just say so and we do it the old way.
I'm sure github can handle it, especially if old issue files are deleted.
 
Top Bottom