Identifying Lost Mods

Quintillus

Archiving Civ3 Content
Moderator
Supporter
Joined
Mar 17, 2007
Messages
7,489
Location
Ohio
As you've likely noticed, there has been a relative flurry of activity around restoring links to old scenarios and modpacks recently. But this isn't something that has never happened before, one only needs to look at stickies by Plotinus and Ozymandias to see the efforts of the past. What's different now?

In addition to the re-uploads by @Virote_Considon , and the return of @KingArthur with a list of scenarios on his hard drive, the two new moderators in the Civ3 forums (myself and @Blake00 ) are both motivated to restore as many scenarios as possible. Blake00 has already done an extensive job of this in the Civ2 forums. This means that whenever new files are reuploaded, we can fairly quickly link to them in the original threads, solving the problem of far too many scenarios requiring you to search the last few pages looking to see if anyone has re-uploaded it. I've already done this for Virote_Considon's re-uploads, and it's quicker than last year when we were working with The_J and Thunderfall, who already have a lot on their plates (but who also support these efforts). If you haven't checked out the main page of civfanatics.com recently, you should, The_J has done a fantastic job of making it active and interesting again.

The other change we have is that I've been able to create a list of all the Scenarios and Modpacks threads that have broken links on their first page, grouped by the type of broken link - HTTP only (but still working - these work fine in Firefox/IE but not Chrome), AtomicGamer (these may be archived), and all other sites. This makes it much easier to identify what's missing than manually checking every single thread. It still requires work, but directed work.

And that's where this thread, and your participation, comes in. We are requesting help identifying which threads with broken links have re-uploads later and just need the link updated, and which ones do not and will require finding someone who downloaded the scenario years ago.

We plan to start with the broken links that are not to AtomicGamer. There are 87 of these for modpacks, and 77 for scenarios. Eventually we can do all these ourselves, but it will go quicker, potentially much quicker, with help.

I've started an online spreadsheet to keep track of those 87 modpacks, which should be publicly viewable. I've screenshotted a section of it below:

1658176165491.png


The Title is the CFC thread title, the URL is the link to the thread. Farther to the right, there is the label and URL of the broken link, some scenarios such as Test IV Time - The Alternative World have multiple broken URLs, perhaps a mainfile and a patch, or mainfile and optional extra artwork.

This is sorted by how many views the thread has, as a rough proxy of popularity. And the "Result" column is color-coded.

Green = False alarm, things are okay. The existing example was missing a patch from the prior version to the current, but the current version download is working, so it's all accessible.
Blue = Restored.
Yellow = Restored, but the current download isn't ideal. In the existing example, the download requires knowing which button says "Download" and which says "Cancel" when the site warns you that the file may harm your computer. All the text is in Russian, so for most users it won't necessarily be clear what to do. But the files are all there, so for now that's good enough (we'll revisit that later).
Orange = Missing, but there's an AtomicGamer link that we haven't checked yet that may contain the file.
Red = Missing, no sign of it

I propose a few additional colors

Gray = The username in the column is actively checking whether it is available (within the next couple hours).
Lavendar = Re-upload found, needs to be updated in the first post by a moderator. Please add comments to the "Result" column with relevant information!

It may sound easy, but it takes a few minutes for each one, and with 87 of them for modpacks (plus more for scenarios, plus more for AtomicGamer, etc.), it takes time. Being able to divide this up among, say, six people instead of two would speed things up.

I've also now uploaded the spreadsheet of 77 potentially missing scenarios, which hasn't been started yet.

If you are interested, respond here, and send me (via PM if you prefer) an e-mail to which I can send an edit invitation to the spreadsheet. If we get some momentum around this section, I'll post links for the scenarios too, and then the AtomicGamer ones (which require even more time for each one, but have a higher expected chance of success).

FAQ

Q:
Will this extend to other categories (Maps, Unit Graphics, etc.)?
A: If this is successful, eventually, yes.

Q: What about previous restoration efforts?
A: We plan to refer to the threads from previous years' efforts as part of trying to identify missing resources once we know they're really missing. But the first task is figuring out what's really missing. I also expect we'll be bumping Virote_Considon's thread when we find missing mods/scenarios that he has locally. He deserves a lot of credit for helping motivate us with that thread, too!

Q: What is the overall vision?
A: We'd like to get to the place that every modpack/scenario has more than one source available, to prevent their disappearance if one site goes offline. Blake00 has already done a lot of this for Civ2. If we get the ball rolling on this initial effort to make sure all the modpacks/scenarios have one link, I'll be posting updates on plans to get to two or three sources for each scenario. That would let us get to the point of not being dependent on any one Great Library.
 
Last edited:

Blake00

CFC Archivist & Social Media Helper
Moderator
Supporter
Joined
Sep 24, 2016
Messages
1,401
Location
Australia
Sounds good mate. You know I'm in to help where I can (usually on the weekends) in between all my other crazy projects.

Had a peak at your spreadsheet.. I see it's just the raw results right now. What sort of fields are you thinking of sticking in for volunteers to report their findings in?
Oh and I noticed some of those dead links relate to @Vuldacon 's EFZI2 which was a surprise as I thought all his stuff was current and good. No doubt he can help relocate those ones and plenty of others will have copies too. :)

.
 

Civinator

Blue Lion
Supporter
Joined
May 5, 2005
Messages
7,890
Here is the mod "Bigger, Better Worlds" (2006) from a very old archiv HD that I own. If you have finished your operations I can delete the file attached.

Edit: Attachment removed, as it must not exist twice on this server.
 
Last edited:

Aiken_Drumn

Deity
Joined
Oct 16, 2021
Messages
2,209
Location
NES/FG/SF Activity:Arguing the toss
Mars Attacks!


While dead.. also only ever had 7 comments.. I don't know if anyone actually downloaded it, or if it was any good. How are you scraping the data for what we are trying to find?
 

Quintillus

Archiving Civ3 Content
Moderator
Supporter
Joined
Mar 17, 2007
Messages
7,489
Location
Ohio
I ran through another 15 at the coffee shop yesterday. The success rate was much lower than the first 10. If you want to have a good feeling about things being available, you might want to start with the scenarios!

Had a peak at your spreadsheet.. I see it's just the raw results right now. What sort of fields are you thinking of sticking in for volunteers to report their findings in?

My thinking had been that the "Result" field could carry any findings, as it's free-form text. If there is a need for additional columns, however, they can be added. With this thread as an option too if something longer-form is needed.

Here is the mod "Bigger, Better Worlds" (2006) from a very old archiv HD that I own. If you have finished your operations I can delete the file attached.

Awesome! I have downloaded it and restored it to the thread. I didn't expect this thread to result in a modpack being found so quickly! :thumbsup:

Mars Attacks!


While dead.. also only ever had 7 comments.. I don't know if anyone actually downloaded it, or if it was any good. How are you scraping the data for what we are trying to find?

A bot friend of mine looked at the first page of each thread in the modpacks and scenarios sections of the forums, and looked for broken links (404 not found, timeouts, internal server errors, gateway timeouts, that sort of thing). Thus, it's an exhaustive list of broken links on the first page (excluding signatures).

So, yes, the ones towards the bottom of the list (which is sorted by view count for the thread) were likely not super popular, and are the most likely to be lost, but least likely to be in demand. Or, potentially, they have the fewest views because they are newer.

I don't know yet where the threshold should be for "is this worth checking?" But I'd recommend starting at the top of the list. If we cover the top half of them and recover half of what's missing, that will be a pretty good result.
 

Quintillus

Archiving Civ3 Content
Moderator
Supporter
Joined
Mar 17, 2007
Messages
7,489
Location
Ohio
I personally don't have a problem with bumping old threads to make it visible that re-uploads are available for them. We don't have a ton of traffic in the completed modpacks/scenarios forums, and there's a side benefit that it might make someone aware of a scenario they'd like to play that they hadn't heard of, most likely due to it predating the join date. If it's just to note that they aren't available, it makes sense to post here/update the spreadsheet (if you PM me an e-mail address, I can give you write access).
 

Quintillus

Archiving Civ3 Content
Moderator
Supporter
Joined
Mar 17, 2007
Messages
7,489
Location
Ohio
True, but I was thinking a comment might trigger an email to someone who could still have it?

After thinking about this for a while, I'm favoring figuring out what's missing before bumping threads because it's missing. It kind of goes back to the "Mars Attacks!" example. It's a low priority, unlikely that someone has it, and unlikely that someone's going to play it even if we find it.

Once we know what's missing, we can decide if it's worth bumping certain threads. My plan had been to initially make posts in the main C&C in the hopes of exactly what happened with Civinator's copy of Bigger, Better Worlds would happen. There are enough long-term-players here that we might recover a fair amount that way, and if needed can later bump threads of those that are still missing, particularly if there's evidence that someone would play them.

I think you've found a decent compromise in the profile posts left for the creators of a couple scenarios (who knew The_Omega had and old Desert Earth scenario from before the more popular ones?). The creators are the most likely people to still have a copy laying around somewhere, so if they swing back by CFC some day, that might lead to their restoration. Or if they're otherwise restored in the interim, that might allow them to revisit their old work that had been lost locally.

I can also tell that you've checked at least a few of the scenarios. If you'd prefer to post a list of scenarios checked/results here, that's great, or I can add you to the spreadsheet... but only with an e-mail, sadly CFC doesn't have native spreadsheets yet.
 

Blake00

CFC Archivist & Social Media Helper
Moderator
Supporter
Joined
Sep 24, 2016
Messages
1,401
Location
Australia
On row 4 concerning Conquests of Might and Magic III..
https://forums.civfanatics.com/threads/conquests-of-might-and-magic-iii.370434/
that's a duplicate thread of this one where the downloads have been restored by you to my upload and node60s.
https://forums.civfanatics.com/threads/conquests-of-might-and-magic-iii-comm3-epic.619720/
Now there's different restored versions of the same mod to both threads.. so it's a bit a mess. Tom says at the top of the first linked thread that it's been superseded by the second thread but he doesn't make it 'that' obvious. Perhaps we should remove the download and just direct people to the newer thread?

As mentioned over PM I've done a bit of work in both files. Mostly in the scenario file though.
 
Last edited:

Blake00

CFC Archivist & Social Media Helper
Moderator
Supporter
Joined
Sep 24, 2016
Messages
1,401
Location
Australia
Smashed about 20 more in the spreadsheets over the weekend, was only meant to spend one day on it but got in the groove and spent more than 2 on it lol. Work included some new rescue uploads, also updated lots of info on my older rescues, and I did another 5 or so rescues that weren't in the spreadsheets (I guess there's more spreadsheets coming lol). In some cases I noticed a mod also had dead links over in the download resource pages so I went and fixed them too while fixing the thread to save us the trouble later when we do that whole other audit lol!
 
Top Bottom