Civ 5 mod problems -- LONG POST!

Not sure if this is the right place to post these kinds of questions, so I have no objection if a Mod(erator) moves it to where it belongs.

To begin, I've washed my hands of CBE until a couple patches, DLC and maybe an expansion gets the game to where it should have been in the first place. Saying that, I'm going back to my heavily-modded Civ5. Flipped through Steamworks to see what was new in the last 6 months and added a few more. Fired up the game, did my Advanced Setup with 41 CSs and 10 AI opponents on a Huge Pangaea map with Raging Barbarians for an Emperor Marathon game. Started playing and noticed something right off: No ancient ruins, anywhere. 150 turns later with probably 1/8-1/6th of the map revealed and I had yet to see a single CS. Definitely something wrong. Then too, I've played Raging Barbarians before but never experienced anything like the tsunami that I was forced to contend with. Like a new Barb camp every turn -- and that was just in the area that I had uncovered. Lord knows how many more were being spawned in the 7/8-5/6ths I hadn't uncovered yet. Worse still, they were spawning additional Barb units every 5 turns. (I'll come back to this later to show why it's significant.)

Thinking that maybe I had just caught a particularly nasty RNG, I started over. Used the same game settings, but unchecked Raging Barbarians. 150 turns later I conclude that I'm stuck in the same situation, right down to the rampaging Barbs, even though Raging Barbs had been unchecked. And paying attention to the many, many combats, I also discover that my units never inflicted as much damage as the combat odds predicted, and the Barbs always inflicted more than was predicted.

Something is probably amiss here, and I'm guessing it has something to do with the mix of mods that I've incorporated that are conflicting or mutating each other. (However, if there are conflicts, they aren't causing CTDs or freezes.) So I start looking at my mod selection and discover something peculiar: In the game's Main Menu/Mods selection section, it shows that I've got 44 active mods (and about 6 DLC scenarios that are NOT active). (And, yes, I _did_ make sure the activate button was illuminated.) In contrast, going to Steamworks and clicking "Subscribed items", it shows that I've subscribed to 109 mods. Taking a close look and making comparisons it looks like all 44 game-acknowledged mods are on the Steamworks list. That leaves me wondering: What happened to the other 65 mods?

Is there a game-imposed limit of 50 Steamwork mods + DLC items that I've never noticed before? If so, how does the game decide which mods will be used? I ask because in going over the two lists, the 44 actually activated mods are scattered all over the Steamworks list. Further, of the @half-dozen mods that I just added, those _are_ activated. This makes me think that game-used mods are the 50 that have been most recently selected.

So, anyone know what might be happening here? I'm most concerned about the disappearing CSs and ancient ruins. I'm also worried about non-Enraged Barbs spawning faster than rabbits. But it would be certainly helpful if anyone could explain why there is a disparity between the Game-acknowledged mod selections and the Steamworks list of subscribed items.

[If requested, I will list out the two mods lists and my dxdiag.txt as spoilers. I won't relish all that typing, but if it's useful info, I'll do it.]

100+ views. Doesn't anyone have an idea why the game shows only 50, but Steamworks shows 100+?

Steam is unreliable.

If you passed the mod selection screen too quickly, it's possible that the game was not able to completely finish downloading everything, and gets "stuck."

You may want to try deleting your Mod cache database file from the civ5 cache folder and restarting the game, and then sitting at the mod selection screen until everything is fully loaded.

Well, that was interesting. I emptied out the mods folder and started Civ 5. When I got to the Mods section, all that was there showing were the DLC scenarios, so I turned off showing DLC content. When mods started to load up, it stopped at _39_, out of the 109 that Steamworks is still showing as subscribed. That's 70 mods NOT being imported.

To say I am puzzled is an understatement.

As I said, Steam is unreliable, especially with a large number of mods.

The only solution I know of is to keep deleting that database file and keep trying again until it is able to successfully load all the mods. Alternatively, try to find alternate download links and unzip the mods into your mods folder manually; At least then the game only has to build the database.

BTW, Thank you! You've been most helpful.

I'm thinking of just Unsubscribing everything, then prioritize 50 to see how well they cross over after I subscribe those 50 again.

The problem with deleting the cache/Civ5ModsDatabase.db file is that it will force the game to redownload every mod - even the ones it has successfully retrieved. And as the game attempts to download mods in parallel, depending on how many mods you have subscribed to and how big they are, the large amount of data being requested from one source could appear as a denial-of-service attack to the server and it will either cut you off, or seriously throttle the connection transfer rate - either of which will cause some of the threads downloading the mods to time out and leave the mods DB in a "stuck" state.

One approach is as you suggest, to un-subscribe and then re-subscribe in batches. However, if you are happy using an SQL utility (such as SQLiteSpy) see post #8 in this thread for an alternative

Using the SQL utility is the best approach, as it allows you to target specifically the ones that failed.

I didn't suggest it, since not everyone is comfortable mucking about in the database files, but even with my meager 20-30 mods I've run into issues with Steam not wanting to download all of them and causing similar 'stuck' issues. I've been using the SQL-editing solution ever since I ran across whoward's post about it myself, and have been very successful with it.

whoward69 said:

One approach is as you suggest, to un-subscribe and then re-subscribe in batches. However, if you are happy using an SQL utility (such as SQLiteSpy) see post #8 in this thread for an alternative

Click to expand...

Hmm. Got me thinking. (Always a scary proposition.) I consider myself only one-quarter computer literate, so I don't even know what SQL stands for much less how to use it. Buuuuttttt I'm wondering.....

So I'm pondering on the synching process.
1. I make a selection of Steamworks mods by Subscribing to them.
2. I start the game and go to the Mods screen
3. The game imports those mods from Steamworks by placing the mod files in ..../MODS

The necessary files are now all resident in MODS. What happens if I then go to Steamworks and Unsubscribe those mods? What happens to the files in MODS?

What I am thinking of doing now is to create a ModsTemp folder, moving the mods files from MODS.
Then Unsubscribe those mods. Now both MODS and Steamworks are empty (nothing subscribed).
Then I Subscribe ten more mods, then start the game. Those new mods get added to MODS.
Transfer those mods files to ModsTemp and Unsubscribe them at Steamworks. Subscribe another ten. Etc. Rinse and repeat.

Eventually, I will have nothing Subscribed at Steamworks, nothing in the MODS folder. Once I have ALL of the mods that I want in ModsTemp, I then transfer all of them back into MODS. I now have the game seeing however many mods in the MODS folder, but no corresponding Subscribed items at Steamworks.

What does the game do then? Does it dump everything in the MODS folder? If that's the case, all I would have to do is to reSubscribe to those specific mods. Then both MODS and Steamworks would be in agreement -- for potentially an insane number of mods.

Now this is where I get real hazy. Starting up the game, it makes a "handshake" with Steamworks to verify the Subscription status of those many, many, many mods. That, I imagine could take quite awhile. (Especially if there are updates involved.) Is that lengthy process likely to be viewed as a Denial Of Service attack? Yes, I have a problem. (Actually ALL mod-users have a problem.) No, and I have ALL the mods I ever wanted. (Provided I am willing to invest all of that back-and-forth time necessary to gather all those mods.)

So, what's the verdict? Does this approach sound feasible?

CaptainPatch said:

The necessary files are now all resident in MODS. What happens if I then go to Steamworks and Unsubscribe those mods? What happens to the files in MODS?

Click to expand...

All one Steamy system - the game will lovingly delete all the mods from the MODS sub-dir.

But you're on the right lines

• Subscribe to mod(s)
• Let game download mod(s)
• Exit game
• Take a COPY of the mod(s) from the MODS sub-dir
• Restart game
• Unsubscribe from mod(s)
• Watch game delete mod(s) from the MODS sub-dir
• Exit game
• Move/Copy the mod(s) back into the MODS sub-dir
• Restart game
• Game will find mod(s) and allow you to play with them BUT will have no idea that they came from Steam so won't bother checking in with Steam about them - this is the same situation as if you'd created your own mod(s) via the SDK

This basically makes all your downloaded mods local to you. Which has the advantage that if Steam throws a wobbly they won't be auto-deleted, or if the author updates their mod on Steam, your copy won't auto-update (potentially breaking saved games). But has the disadvantage that if the author updates their mod it won't auto-update (you'll need to do the subscribe-copy-unsubscribe-move thing again if you want the latest version)


Checking for a change to a mod - even for 100's of mods - is a very quick/small transaction compared to downloading a mod, so doesn't appear as an attack on the server.

To add, I've also not had much success with getting past the "stuck" issue even with repeated unsubscribing and resubscribing whenever I had a mod that stubbornly refused to download.

The only sure-fire method was the SQL-editing.

If I run into the issue again, I can probably try taking some screenshots and put together a visual guide, or something. I don't use whoward's SQL statement, but rather manually go through the list and delete the affected rows, though they do the same thing.

Some new weirdness. I emptied the current MODS folder into a temp folder and left MODS completely empty. I then went to Steamworks and opened my Subscribed list. I Unsubscribed all of the mods that were now in the temp folder, along with a bunch of low-priority mods. That left about 40-50 on the list and another @40 in the temp folder. Started the game and went to MODS. Here's where the weirdness comes in: Considering the 40-50 remaining mods on the Subscribe list, the game only imported _three_ mods -- and they were for three mods that I had Unsubscribed and were no longer on the Subscribed list.

1) An empty MODS folder, @40-50 mods on the Steamworks Subscribed list, and the game won't import any of them?

2) Why does the game insist on importing 3 specific mods that are NOT Subscribed?

Just to make sure it wasn't data lingering in the RAM cache, I powered down the PC and gave it five minutes to clear its head. Didn't change anything. The game insists on pulling in those three mods, no matter what.

You'll also need to delete the C:\Users\{username}\Documents\My Games\Sid Meier's Civilization 5\cache\Civ5ModsDatabase.db file - as that's where the game stores info about what it's downloading / needs to download

Woot! That seems to have been the culprit. Discovered that upon starting the game, a new Civ5Modsdatabase.db is created, even before going to MODS. The current Subscribed list came up, and I pulled back the first 39 from the ModsTemp folder. All of the combined batch listed in MODS, so that's good. Now all I have to do is sort which specific mods are causing the initial world creation to freeze. (Doable.)

Thanks a bunch!

And if anyone ever wonders why I walked away from hosting my mods on Steam a couple of years back ... just read this thread

whoward69 said:

And if anyone ever wonders why I walked away from hosting my mods on Steam a couple of years back ... just read this thread

Click to expand...

It IS convenient, though... so long as you don't have too many.

Glad you got everything working though, OP.
This is mostly why I try not to go insane with mods.

whoward69 said:

And if anyone ever wonders why I walked away from hosting my mods on Steam a couple of years back ... just read this thread

Click to expand...

Is it really fair to heap the blame entirely on Steam? The glitch that caused the problem in the first place was a result of the game's program coding. That makes it a Firaxis problem. It theoretically encourages Steam modding, but then makes it awkward to fully utilize Steamworks.
**********************************
NEW QUESTION!
Anyone familiar with a mod that interferes with being able to select a city (the first city) build project? Everything else is working fine (so far), but I can't activate "Choose Production". That means I can't even finish Turn One. The only things that I have that have anything to do with the City screen are a whole assortment of buildings packs. For the Ancient era I have Ancient Building Pack and Ancient Building Pack 2. Hmm. I think I'll turn those off and see if that affects anything.
_____________________________________
Turns out the mods causing problems were "Communitas Interface" and "Communitas AI & Tools". Given that those mods are so highly rated, with so many subscribers (20K+ and 60K+ respectively), I'm surprised they were the ones causing problems.

Didn't say I did blame (only) Steam, just that I walked away from it. The problem is both with Steam (complete lack of concern for the number and size of updates they send out, inability to keep a stable platform - such that mods appear to be "gone" and the game deletes them, inability to provide basic feedback mgmt tools for mod authors, etc, etc, etc) and Firaxis (poor integration of Steam into the game, incomplete mod uploading tool, etc, etc, etc) Steam may be convenient from a mod users point-of-view (so long as you can forget all the problems it causes when it goes horribly wrong) but from a mod authors point-of-view it truly sucks big time.

I agree that Steamworks is rather poorly done (compared to how good it could be). A better approach can be found at Nexus forums: http://www.nexusmods.com/games/?

I really wish they included the Civ games. Nexus not only gives you the mods, they also give you the tools to improve game performance. (Unfortunately, a rather steep learning curve to use those tools.) But worth it once you use those tools to clean up the garbage code that the developers left in the program just because it was too much bother to go back and tidy up the coding.