Identifying Lost Mods

Quintillus

Archiving Civ3 Content
Moderator
Supporter
Joined
Mar 17, 2007
Messages
8,393
Location
Ohio
As you've likely noticed, there has been a relative flurry of activity around restoring links to old scenarios and modpacks recently. But this isn't something that has never happened before, one only needs to look at stickies by Plotinus and Ozymandias to see the efforts of the past. What's different now?

In addition to the re-uploads by @Virote_Considon , and the return of @KingArthur with a list of scenarios on his hard drive, the two new moderators in the Civ3 forums (myself and @Blake00 ) are both motivated to restore as many scenarios as possible. Blake00 has already done an extensive job of this in the Civ2 forums. This means that whenever new files are reuploaded, we can fairly quickly link to them in the original threads, solving the problem of far too many scenarios requiring you to search the last few pages looking to see if anyone has re-uploaded it. I've already done this for Virote_Considon's re-uploads, and it's quicker than last year when we were working with The_J and Thunderfall, who already have a lot on their plates (but who also support these efforts). If you haven't checked out the main page of civfanatics.com recently, you should, The_J has done a fantastic job of making it active and interesting again.

The other change we have is that I've been able to create a list of all the Scenarios and Modpacks threads that have broken links on their first page, grouped by the type of broken link - HTTP only (but still working - these work fine in Firefox/IE but not Chrome), AtomicGamer (these may be archived), and all other sites. This makes it much easier to identify what's missing than manually checking every single thread. It still requires work, but directed work.

And that's where this thread, and your participation, comes in. We are requesting help identifying which threads with broken links have re-uploads later and just need the link updated, and which ones do not and will require finding someone who downloaded the scenario years ago.

We plan to start with the broken links that are not to AtomicGamer. There are 87 of these for modpacks, and 77 for scenarios. Eventually we can do all these ourselves, but it will go quicker, potentially much quicker, with help.

I've started an online spreadsheet to keep track of those 87 modpacks, which should be publicly viewable. I've screenshotted a section of it below:

1658176165491.png


The Title is the CFC thread title, the URL is the link to the thread. Farther to the right, there is the label and URL of the broken link, some scenarios such as Test IV Time - The Alternative World have multiple broken URLs, perhaps a mainfile and a patch, or mainfile and optional extra artwork.

This is sorted by how many views the thread has, as a rough proxy of popularity. And the "Result" column is color-coded.

Green = False alarm, things are okay. The existing example was missing a patch from the prior version to the current, but the current version download is working, so it's all accessible.
Blue = Restored.
Yellow = Restored, but the current download isn't ideal. In the existing example, the download requires knowing which button says "Download" and which says "Cancel" when the site warns you that the file may harm your computer. All the text is in Russian, so for most users it won't necessarily be clear what to do. But the files are all there, so for now that's good enough (we'll revisit that later).
Orange = Missing, but there's an AtomicGamer link that we haven't checked yet that may contain the file.
Red = Missing, no sign of it

I propose a few additional colors

Gray = The username in the column is actively checking whether it is available (within the next couple hours).
Lavendar = Re-upload found, needs to be updated in the first post by a moderator. Please add comments to the "Result" column with relevant information!

It may sound easy, but it takes a few minutes for each one, and with 87 of them for modpacks (plus more for scenarios, plus more for AtomicGamer, etc.), it takes time. Being able to divide this up among, say, six people instead of two would speed things up.

I've also now uploaded the spreadsheet of 77 potentially missing scenarios, which hasn't been started yet.

If you are interested, respond here, and send me (via PM if you prefer) an e-mail to which I can send an edit invitation to the spreadsheet. If we get some momentum around this section, I'll post links for the scenarios too, and then the AtomicGamer ones (which require even more time for each one, but have a higher expected chance of success).

FAQ

Q:
Will this extend to other categories (Maps, Unit Graphics, etc.)?
A: If this is successful, eventually, yes.

Q: What about previous restoration efforts?
A: We plan to refer to the threads from previous years' efforts as part of trying to identify missing resources once we know they're really missing. But the first task is figuring out what's really missing. I also expect we'll be bumping Virote_Considon's thread when we find missing mods/scenarios that he has locally. He deserves a lot of credit for helping motivate us with that thread, too!

Q: What is the overall vision?
A: We'd like to get to the place that every modpack/scenario has more than one source available, to prevent their disappearance if one site goes offline. Blake00 has already done a lot of this for Civ2. If we get the ball rolling on this initial effort to make sure all the modpacks/scenarios have one link, I'll be posting updates on plans to get to two or three sources for each scenario. That would let us get to the point of not being dependent on any one Great Library.
 
Last edited:
Sounds good mate. You know I'm in to help where I can (usually on the weekends) in between all my other crazy projects.

Had a peak at your spreadsheet.. I see it's just the raw results right now. What sort of fields are you thinking of sticking in for volunteers to report their findings in?
Oh and I noticed some of those dead links relate to @Vuldacon 's EFZI2 which was a surprise as I thought all his stuff was current and good. No doubt he can help relocate those ones and plenty of others will have copies too. :)

.
 
Here is the mod "Bigger, Better Worlds" (2006) from a very old archiv HD that I own. If you have finished your operations I can delete the file attached.

Edit: Attachment removed, as it must not exist twice on this server.
 
Last edited:
Mars Attacks!


While dead.. also only ever had 7 comments.. I don't know if anyone actually downloaded it, or if it was any good. How are you scraping the data for what we are trying to find?
 
I ran through another 15 at the coffee shop yesterday. The success rate was much lower than the first 10. If you want to have a good feeling about things being available, you might want to start with the scenarios!

Had a peak at your spreadsheet.. I see it's just the raw results right now. What sort of fields are you thinking of sticking in for volunteers to report their findings in?

My thinking had been that the "Result" field could carry any findings, as it's free-form text. If there is a need for additional columns, however, they can be added. With this thread as an option too if something longer-form is needed.

Here is the mod "Bigger, Better Worlds" (2006) from a very old archiv HD that I own. If you have finished your operations I can delete the file attached.

Awesome! I have downloaded it and restored it to the thread. I didn't expect this thread to result in a modpack being found so quickly! :thumbsup:

Mars Attacks!


While dead.. also only ever had 7 comments.. I don't know if anyone actually downloaded it, or if it was any good. How are you scraping the data for what we are trying to find?

A bot friend of mine looked at the first page of each thread in the modpacks and scenarios sections of the forums, and looked for broken links (404 not found, timeouts, internal server errors, gateway timeouts, that sort of thing). Thus, it's an exhaustive list of broken links on the first page (excluding signatures).

So, yes, the ones towards the bottom of the list (which is sorted by view count for the thread) were likely not super popular, and are the most likely to be lost, but least likely to be in demand. Or, potentially, they have the fewest views because they are newer.

I don't know yet where the threshold should be for "is this worth checking?" But I'd recommend starting at the top of the list. If we cover the top half of them and recover half of what's missing, that will be a pretty good result.
 
Roger that. Finally, do I have mod approval to bump such old threads as needed please?
 
I personally don't have a problem with bumping old threads to make it visible that re-uploads are available for them. We don't have a ton of traffic in the completed modpacks/scenarios forums, and there's a side benefit that it might make someone aware of a scenario they'd like to play that they hadn't heard of, most likely due to it predating the join date. If it's just to note that they aren't available, it makes sense to post here/update the spreadsheet (if you PM me an e-mail address, I can give you write access).
 
True, but I was thinking a comment might trigger an email to someone who could still have it?
 
True, but I was thinking a comment might trigger an email to someone who could still have it?

After thinking about this for a while, I'm favoring figuring out what's missing before bumping threads because it's missing. It kind of goes back to the "Mars Attacks!" example. It's a low priority, unlikely that someone has it, and unlikely that someone's going to play it even if we find it.

Once we know what's missing, we can decide if it's worth bumping certain threads. My plan had been to initially make posts in the main C&C in the hopes of exactly what happened with Civinator's copy of Bigger, Better Worlds would happen. There are enough long-term-players here that we might recover a fair amount that way, and if needed can later bump threads of those that are still missing, particularly if there's evidence that someone would play them.

I think you've found a decent compromise in the profile posts left for the creators of a couple scenarios (who knew The_Omega had and old Desert Earth scenario from before the more popular ones?). The creators are the most likely people to still have a copy laying around somewhere, so if they swing back by CFC some day, that might lead to their restoration. Or if they're otherwise restored in the interim, that might allow them to revisit their old work that had been lost locally.

I can also tell that you've checked at least a few of the scenarios. If you'd prefer to post a list of scenarios checked/results here, that's great, or I can add you to the spreadsheet... but only with an e-mail, sadly CFC doesn't have native spreadsheets yet.
 
On row 4 concerning Conquests of Might and Magic III..
https://forums.civfanatics.com/threads/conquests-of-might-and-magic-iii.370434/
that's a duplicate thread of this one where the downloads have been restored by you to my upload and node60s.
https://forums.civfanatics.com/threads/conquests-of-might-and-magic-iii-comm3-epic.619720/
Now there's different restored versions of the same mod to both threads.. so it's a bit a mess. Tom says at the top of the first linked thread that it's been superseded by the second thread but he doesn't make it 'that' obvious. Perhaps we should remove the download and just direct people to the newer thread?

As mentioned over PM I've done a bit of work in both files. Mostly in the scenario file though.
 
Last edited:
Smashed about 20 more in the spreadsheets over the weekend, was only meant to spend one day on it but got in the groove and spent more than 2 on it lol. Work included some new rescue uploads, also updated lots of info on my older rescues, and I did another 5 or so rescues that weren't in the spreadsheets (I guess there's more spreadsheets coming lol). In some cases I noticed a mod also had dead links over in the download resource pages so I went and fixed them too while fixing the thread to save us the trouble later when we do that whole other audit lol!
 
A few updates..

Haven't done any more work on this for a while although I notice no one else has either so at least it's not just me haha! ;) Hopefully I can get back to all this again probably early next year as I really want to finish up releasing my Civ2 mods before getting back into more preservation work.

the two new moderators in the Civ3 forums (myself and @Blake00 ) are both motivated to restore as many scenarios as possible
A few people have contacted me about Civ3 forum issues since reading this line so I thought I'd better clarify things. While Quintillus was very kind and gracious to put me up there with him however technically at this point in time I'm not really a full Civ3 moderator as The_J only gave me mod access to the Civ3 creation sub forum to help with preservation work. So I'm unfortunately I'm powerless to help with issues (eg duplicate posts & broken links etc) in the main Civ3 forums. Quintillus is your main one for that stuff (EDIT: & Laurana Kanan is now back too!), however if he's busy/absent then yeah sure message me and I'll relay it to the admins. :) Or just hit the report button and they'll deal with it directly.

And lastly I already posted this in another Civ3 thread but since this is now the official main broken links fixing thread I figured I'd post the Atomic Gamer backup mod rescue trick here below too for anyone wishing to help...



Thanks to help from Stromling over on the Civ4 forums where The_J is also pushing some great preservation work I have finally figured out the system used by the old Atomic Gamer file hosts used by many Civ3 modders and figured I'd post a copy of it here for the Civ3 community too!

I've got a bunch of lost Civ 3 and 4 scenarios that were not part of the Atomic Gamer CFC backup due to being personal uploads on their site instead.. I and others have been trying to figure out how to search for download codes in the MASSIVE backup collection for years and hadn't had any luck!
https://archive.org/details/atomicgamer

The codes in the archive names and the download codes were 2 different things and I couldn't work out a connection as I had not realised that the searchable codes on the archives were actually shortened reversed dates!

Stromling already explained this to me over PM but I'll stick these step by step examples here for others so we can spread the word and solve a lot of problems across this site (mostly in the civ3 & 4 forums). Basically if we put any broken atomic gamer download into web archive we can hopefully see the extra info we need..

Example 1 - the missing Civ4 Age of Discovery Mod (no not Dale's, this is another one) I was trying to find yesterday:
https://forums.civfanatics.com/threads/age-of-discovery.415830/
I put it's dead download into webarchive and find this backup:
https://web.archive.org/web/20130531074316/http://www.atomicgamer.com/files/90780/age-of-discovery
From this link I can see the download code of 90780 and the page tells me the file name, it's size, and the date it was uploaded to Atomic gamer:
AgeOfDiscovery.7z - 32.7 MB - 03/19/2011 (19/03/2011 outside of America)
That date means it should be in the 110319 archive or archive days close to that. So I search for that number in the collection and up it comes:
https://archive.org/details/AtomicGamer_110319
I click show all files:
https://archive.org/download/AtomicGamer_110319
I click view contents on the main tar file:
https://ia801306.us.archive.org/vie...ems/AtomicGamer_110319/AtomicGamer_110319.tar
And bingo.. there's a archive with the download code 90780 and it's a similar size! :) Filename is not there sadly so ya gotta download, check if its the right file and then rename to it's old name.

Example 2 - the missing Civ 3 Star Trek mod bonus music files I've been trying to find for 2 friggin years (link briefly mentioned in the description text)
https://forums.civfanatics.com/resources/star-trek-mod-v-0-9-0-beta-7z.16249/
I put it's dead download link into webarchive and it FAILED:
https://web.archive.org/web/*/http://www.atomicgamer.com/file.php?id=86966
However I can see the failed link is in an old format (file php id) compared to the discovery one above (files/xxxxx/) which had the download code in some slashes, so I try the 86966 code in that format with a * on the end which will show all pages listed under that download code that were hopefully backed up.
https://web.archive.org/web/*/http://www.atomicgamer.com/files/86966/*
Bingo!
https://web.archive.org/web/20120617234339/http://www.atomicgamer.com:80/files/86966/mussfx_civ3stm
From this link I can see the download code of 86966 and the page tells me the file name, it's size, and the date it was uploaded to Atomic gamer:
Sounds.zip - 724.3 MB - 07/03/2010 (03/07/2010 outside of America)
That date means it should be in the 100703 archive or archive days close to that. So I search for that number in the collection and up it comes:
https://archive.org/details/AtomicGamer_100703
I click show all files:
https://archive.org/download/AtomicGamer_100703
I click view contents on the main tar file:
https://ia801300.us.archive.org/vie...ems/AtomicGamer_100703/AtomicGamer_100703.tar
And bingo.. there's a archive with the download code 86966 and it's a similar size! :) Filename is not there sadly so ya gotta download, check if its the right file and then rename to it's old name.

Sometimes there won't be a web archive backup of the old atomic gamer download page but as you can see above it could just be a matter of the old format atomic download link needing to be adjusted to their final one, and if all that fails then Stromling points out that you can usually work out the date of upload by the day it was posted on the CFC forums, sometimes file names and sizes are mentioned in the posts too.

. . .
 
Last edited:
A few people have contacted me about Civ3 forum issues since reading this line so I thought I'd better clarify things. While Quintillus was very kind and gracious to put me up there with him however technically at this point in time I'm not really a full Civ3 moderator as The_J only gave me mod access to the Civ3 creation sub forum to help with preservation work. So I'm unfortunately I'm powerless to help with issues (eg duplicate posts & broken links etc) in the main Civ3 forums. Quintillus is your main man for that stuff, however if he's busy/absent then yeah sure message me and I'll relay it to the admins. :) Or just hit the report button and they'll deal with it directly.
Oh, haha, I did not realize you weren't a Civ3-wide mod! Back in the vBulletin days it was easier to tell who was a moderator of which forum. I don't know if it's a bug or feature that it isn't very visible on XenForo.

And indeed, I got distracted by a different (non-Civ-related) side project, then traveled some, then started a new job, plus now it's the one season where I follow sports, and I'm trying to enjoy the last few nice weekends in the northern hemisphere before it gets cold for several months (this weekend looks quite promising in terms of weather, TBD if it'll be the last nice and warm one). I expect that some time this winter when it's cold and I don't want to venture outside I'll dive back into this again, and I can still fix up links as they're reported* in the meantime.

(* - Technically "within a couple days of when they are reported" is more accurate, I swing by here most nights but not every night. Feel free to tag me for things that need to be fixed at some point, but report spambots or other items requiring immediate attention. And thanks for continuing to represent the "civil" in Civilization since I've become a moderator!)
 
FWIW (I'm literally just scrolling through my bookmarks here; most of the BOLDING. should be for obvious reasons ... and I did find some things I had no idea existed :hmm:) -
Misc:
! https://polycast.civfanatics.com/modcast/season6/focus_civ3.mp3 - August 16th, 2014 podcast with Blue Monkey, Rob (R8XFT), Plotinus, El Justo, Balthasar (!!!)

... more to follow. :coffee:


:D


- See: Already :hammer2:
  1. All Google searches for <CIV 3 "North America" ".bix"> redirect to TET.
  2. All links for "The World 1948-2030"link to ".ru" d/ls ... :undecide:
 
Last edited:
There seems to be a German equivalent of Quint’s DVD compilation ("Die grosse Civ3 DvD") at https://www.civforum.de/showthread.php?44660-Die-grosse-Civ3-DvD. The page readily translates with Google, and the list is most extensive - @Civinator, my Comrade?
I did come across that at some point, I can't remember if it was before or after my compilation was finished. Unfortunately it seems it went dormant by 2014.

More broadly though, that's quite a list! And it inspired me to upload the needs-HTTPs spreadsheets (scenarios, modpacks), and to make a few updates among those, including for at least a couple in your list (Native America Mod, Fantasy Empires).

To make sure I'm following correctly - the list is your list of bookmarks of scenarios that you deemed worth bookmarking at some point? Some of them, e.g. Twillight of Byzantium, seem to be working as expected from a downloads standpoint. Still useful in terms of having a list of things that piqued a current member's interest though, and perhaps especially the ones such as Rob Cheng's scenarios that wouldn't have shown up on my list of mods at CivFanatics.
 
Cheers, hopefully some of these might solve some of the dead links in Quin's excel lists. :)

Mod "Mars" WARNING: ".RU" SITE (... @Laurana Kanan ?? :help: ; :thanx: ; :D ) I truly do feel sorry for "civfanatics.ru" :undecide:

I was going to say I wouldn't worry about that place, however unfortunately all I see is this window below when I visit that thread. Presumably due to recent real world events they've done a purge of foreign members with recently active accounts as there was absolutely no warning and no other reason I can think of for the ban. I hadn't posted there in a while nor had I directly interacted or spoken with anyone there either. My last post there was way back near the start of the year updating one of my Civ2 preservation project threads in a quiet corner of their forum where no one else ever seemed to post (the civ2 section). I know some staff didn't mind me posting there as some of my Civ2 threads even got pinned over the years but yeah.. clearly times have changed and one of the admins decided to 'take out the trash' lol. However if you can still see into that thread you'll probably see that I replied to it ages ago hoping to speak to Michael about his Mars mod as while I successfully rescued his original Russian version, the download for his English release is dead sadly & I've been unable to find a copy of it. He's been MIA for years but if he ever does return I won't be able to answer him sadly! Wish I could find the file! :(
1666449995845.png
 
Top Bottom