Civilization 2 downloads area issues and updates

[Blake00]

Hey guys,

A while back I came here to request the creation of some Call to Power 1 & 2 download areas, which you guys kindly granted and I filled them with downloads that I see have been getting plenty of visitors to in the years since, which is good as after the fall of Poly downloads CTP needed a new home for it's patches & mods!

Anyway these days my attention has been on Civilization 2 and as some of the admins may already know I've been doing Civ2 preservation work over the last few years here at CFC building giant lists cataloging archives of lost Civ2 websites, huge lists cataloging and rescuing scenarios, which expanded to collecting lots of rare lost scenarios from other collectors and dead sites.

If you look in the Civilization 2 downloads area you can see I'm just getting started on the big process of uploading hundreds of these old collected lost scenarios here to CFC to ensure the communities have access to all the lost stuff again. I'm doing my best to ensure they have consistent and easily identifiable titles along with good descriptions and a nice picture to show off the authors work.

As part of this process I'm also going through all the old downloads in the Civ2 section to make sure I don't accidentally duplicate anything. While doing that I've noticed a number of issues and that things are in a bit of a mess in there...

One of the big problems is Test of Time is obviously the same engine but 'kind of' a different game (eg like what CTP2 was to CTP1) and because of that ToT scenarios are not compatible with MGE while visa versa MGE scenarios are not compatible with ToT either, yet they're all mixed in together in the same download section and often people were lazy with their titling and descriptions making it hard for the casual viewer to make the right download choice. I don't really know how to fix that problem though as identifying and separating all of the content into 2 separate ToT and MGE/preToT sections would be a huge job for an admin and I'm sure you guys would rather being doing a thousand other things that cleaning up a download section for a 25 year old game lol!!

However some of the other problems are silly little things that might be more solvable however I can't figure out how to fix them. I'm guessing I probably can't but thought I'd clarify with you guys in case I've missed something.

Eg on a few occasions I've accidentally uploaded a file under the wrong sub category (eg a modpack in the scenario area or visa versa, or a scifi scenario in the regular scenario area outside of the scifi area etc) and I can see others have accidentally done this too over the years. I looked for options to move my accidents to their correct sub category or to even fully delete their resource page so I could start over again in the right section but had no luck finding any options for either.

There's also quite a number of older downloads with dead external links posted by users long gone from CFC that I actually have a working file for but I can't find any options to fix or update them. Although I'm guessing that's not possible since that feature could be abused.

So if other peoples dead items can't be updated then that begs the question of is it okay to create duplicate content in there? Especially when it was the scenarios creator that make the original dead entry? I can see some other duplicate content in there (same scenario listed twice by different people) so I kinda feel like I'm adding to the problem if I do that, yet a dead download is a dead download and people can't get the scenario unless its added again or the original can be modified.

What's the standard procedure around here for problems like these? Wanna make sure I do the right thing by this site and it's rules.

.

 

[leif erikson]

Thank you for your efforts for our community.

Have not worked much in the downloads section of the forum. I will bring this issue up in staff and see what we can come up with.

 

[The_J]

I think there are 2 separate issues here:
#1 is it possible to identify if it's a ToT or MGE scenario within the file? (no matter how obscure the marker would be)
#2 is it possible to mass-reassign items in the download area?

If there is a marker, then it'd not be very complicated to write a script to identify which is which, but I don't know how the download DB works on the moderators site in Xenforo.

 

[unartis]

Hi All!

I'm struggling with a similar problem, I want to share my own old mods for Civilization 3 again because the old link doesn't work. How can I do this without having to introduce it as a new thread?

My problem is that when editing the download menu I can't solve the change to the link assigned to the download button, I can't find it anywhere(I marked it with the number 1)
It is possible to insert a link into the text file, but it does not apply to the main download button. (I marked it with the number 2).
It’s my own mode, so I can enter the editing interface.


If anyone knows the answer I would thank you.

 

[Blake00]

I think there are 2 separate issues here:
#1 is it possible to identify if it's a ToT or MGE scenario within the file? (no matter how obscure the marker would be)
#2 is it possible to mass-reassign items in the download area?

If there is a marker, then it'd not be very complicated to write a script to identify which is which, but I don't know how the download DB works on the moderators site in Xenforo.

1. I doubt it as ToT uses the exact same file extensions as MGE for scenarios. Humans can tell the difference as ToT art assets are in 24bit bmp's instead of 8bit ones but that's about it. So even if you developed something to scan all the zips I'm not sure if there's an effective way to auto separate. Only a human could really do it properly.
2. Good question.

Hi All!

I'm struggling with a similar problem, I want to share my own old mods for Civilization 3 again because the old link doesn't work. How can I do this without having to introduce it as a new thread?

My problem is that when editing the download menu I can't solve the change to the link assigned to the download button, I can't find it anywhere(I marked it with the number 1)
It is possible to insert a link into the text file, but it does not apply to the main download button. (I marked it with the number 2).
It’s my own mode, so I can enter the editing interface.


If anyone knows the answer I would thank you.

View attachment 600711

View attachment 600712

If you look at your first screenshot literally right above where you put a 1 and circled the "Download Now via External Site" button you'll see a "Post Resource Update" button. That's the one you want mate.

.

 

[The_J]

1. I doubt it as ToT uses the exact same file extensions as MGE for scenarios. Humans can tell the difference as ToT art assets are in 24bit bmp's instead of 8bit ones but that's about it. So even if you developed something to scan all the zips I'm not sure if there's an effective way to auto separate. Only a human could really do it properly.

mmhh... can you give me 2 links, one to a ToT and one to a MGE scenario?
Automatically determining the type of bmp should be possible, I think.

 

[unartis]

If you look at your first screenshot literally right above where you put a 1 and circled the "Download Now via External Site" button you'll see a "Post Resource Update" button. That's the one you want mate.

@Blake00,
Thank you, it was there before my eyes yet i didn't see it

 

[Blake00]

mmhh... can you give me 2 links, one to a ToT and one to a MGE scenario?
Automatically determining the type of bmp should be possible, I think.

Here's a ToT modpack I recently posted:
https://forums.civfanatics.com/resources/star-trek-the-final-frontier-modpack-tot.28242/

And here's an MGE scenario:
https://forums.civfanatics.com/resources/alpha-centauri-scenario-mge.28910/

No all scenarios have custom graphics meaning there may be no graphics files in them so that idea wouldn't sort everything out but it might take care of most of the work. @Prof. Garfield might have some ideas about other possible identifiable differences between ToT & MGE Modpacks and scenarios I haven't thought of.

.

 

[The_J]

Okay, I see that the ToT scenario has some .bmp files in it, but right now I haven't figured out if it's a 24 bit .bmp, or a 8 bit indexed colour bmp. (I know you say 24 bit, but my script can't tell yet, because I can't tell from the file itself yet).
Your MGE scenario has gif files though.
So 3 questions:
- Does this difference of 24 to 8 bit apply to any type of image file?
- what image types are accepted by either MGE or ToT?
- could these be mixed?

 

[Blake00]

Opps I may be wrong about the 24bit sorry. When my old Paint Shop Pro loads ToT Bmp images it says they're 24bit true colour which is why I thought that. However in Windows File Explorer when I look at the Bmp file property details it says 16bit.

Also on Catfish's differences between Civ2 versions page he says ToT is 15bit so yeah the files probably are 16bit.
http://users.tpg.com.au/jpwbeest/jp_versions.htm

8bit Gif and Bmp are accepted by MGE. I'm not sure if ToT retained the ability to load 8bit Gif but probably. Every ToT scenario and mod I see with some custom graphics seems to have the 16bit Bmps whether it be units, cities, icons, people, terrain etc so maybe it doesn't accept the old 8bit gifs anymore, although I still see 8bit Title.gif files occasionally in ToT stuff.

 

[Prof. Garfield]

I think there are 2 separate issues here:
#1 is it possible to identify if it's a ToT or MGE scenario within the file? (no matter how obscure the marker would be)
#2 is it possible to mass-reassign items in the download area?

If there is a marker, then it'd not be very complicated to write a script to identify which is which, but I don't know how the download DB works on the moderators site in Xenforo.

I don't know of a way to determine the game version just from the file names. However, if your script can look at the first dozen bytes in the .scn file, you can determine what version of the game it is. From @Catfish 's website

I'm pretty sure bytes 10 and 11 are used for the same purpose in Classic/Fantastic Worlds/Multiplayer Gold Edition/Test of Time. We'd have to look up the numbers for older versions of the game to make the classification, and also note that Test of Time Patch Project changes the version number too, I believe.

 

[Blake00]

won't work with the ToT modpack I linked previously as there's no scenario file in it.. so here's another ToT download for you The_J that does have one if you want to experiment with Garfield's info:
https://forums.civfanatics.com/resources/star-wars-insurrection-for-tot.24173/

 

[The_J]

I was just about to ask for that.
How common is it that there is a scenario file?

I'm also having trouble with the fact that neither your ToT scenario nor your MGE scenario has a version number which fits to the one listed on the website, although they are different.
Maybe I should just crawl through a bunch and see what version numbers there are.

Opps I may be wrong about the 24bit sorry. When my old Paint Shop Pro loads ToT Bmp images it says they're 24bit true colour which is why I thought that. However in Windows File Explorer when I look at the Bmp file property details it says 16bit.

Also on Catfish's differences between Civ2 versions page he says ToT is 15bit so yeah the files probably are 16bit.
http://users.tpg.com.au/jpwbeest/jp_versions.htm

8bit Gif and Bmp are accepted by MGE. I'm not sure if ToT retained the ability to load 8bit Gif but probably. Every ToT scenario and mod I see with some custom graphics seems to have the 16bit Bmps whether it be units, cities, icons, people, terrain etc so maybe it doesn't accept the old 8bit gifs anymore, although I still see 8bit Title.gif files occasionally in ToT stuff.

Okay, that would be a bit complicated .

 

[Prof. Garfield]

I'm also having trouble with the fact that neither your ToT scenario nor your MGE scenario has a version number which fits to the one listed on the website, although they are different.
Maybe I should just crawl through a bunch and see what version numbers there are.

Oh, I should have explicitly mentioned that the document above was for two test of time versions, original, and patched to "1.1".

The attached document (apparently written in year 2000) yields this information.

(I uploaded the image to preserve formatting).

I checked a Test of Time Patch Project saved file, and the byte in question has a 33. I don't know when the Test of Time Patch project started differentiating saved files, but even a game saved with the most recent version still uses 33. Since 32 was the official version 1.1 patch, I think it is safe to say that all TOTPP saved games have 33 as the version byte. If @TheNamelessOne happens to be around, he can weigh in.

As far as scenario building goes, I only really know about programming with Lua, and I don't know anything about modpacks. If the folder has a .scn or a .sav file, you can check the byte. Otherwise I don't know what to suggest.

Maybe @JPetroski , @CurtSibling , @Knighttime , @techumseh , @tootall_2012 , @Fairline or @Civinator have some idea.

 

[The_J]

so... the last scn file from blake for ToT seems to have as the 10th byte just a 2... which I guess it also shouldn't be... so I'm still confused lol.
I'll crawl the download db, let's see what exactly turns up in it.

 

[Prof. Garfield]

so... the last scn file from blake for ToT seems to have as the 10th byte just a 2... which I guess it also shouldn't be... so I'm still confused lol.
I'll crawl the download db, let's see what exactly turns up in it.

In my hex editor, a byte with hex value 32 shows up as the character "2". This makes sense based on the ASCII character table. That means that the hex value 2C is the character "," (Multiplayer Gold Edition), 28 is "(" (Fantastic Worlds), and 27 is "'" (single quote) (Conflicts in Civilization) and earlier). 31 is character "1" (unpatched TOT), and 33 is character "3" (Test of Time Patch Project).

 

[The_J]

Uh man, I knew I was missing something .
Okay... I've got now all the urls of the civ2 scenarios (only main download section), now I'll need to see how to get behind the damn button with Python (let's see if I do that today or maybe on Tuesday; seems I can do that with Selenium, but that'll be ugly), then I can start screening.

EDIT: Okay, figured it.
As expected, very ugly. Need to change the browser settings to allow automatic download, then need to go to the download page, change the frame, accept the cookies, switch back, and download it. There's no connection between the download link and the name of the final file, but that I can relate to each other via the time of downloading.

At the beginning this seemed faster, because I was directly calling the download button, which also lead me to skipping the cookie dialog, but since the browser then actually didn't open a webpage, the underlying tool would stall o_O.

Well... anyways... tomorrow I'll do the automatic download, and then probably on Tuesday I'll screen through what we have here in the database.

 

[Blake00]

Thanks @The_J , I didn't think this would all lead to much other than "yup, too hard basket" so this is great!

And cheers @Prof. Garfield , I knew you'd think of a better idea, and if not I was gonna tag the rest of the SL to get their thoughts next but you've taken care of that too haha.

Yeah most modpacks don't come with a scenario file as they're intended to modify the existing sandbox game or an existing scenario that comes with the base game. However they are in the minority and most of the downloads would be scenarios or modpacks with scenario files in them so in theory if most can be automatically separated then the leftovers would be a smaller more achievable amount that a human could check and figure out.

Of course then there's the other miscellaneous non-scenario/modpack stuff in the downloads section such as maps (which I think are cross-compatible) and lone graphic files which is a different headache. May be easier to leave them in a generic Civ2 area and just split the scenarios and modpacks up into ToT and MGE sections.

I should also say that many people thankfully DID use good terminology in their titles or posts so some sort of script that searches for "ToT" and "Test of Time" across all the Civ2 downloads would also allow a lot of stuff to be separated too.

.

 

[The_J]

Thanks @The_J , I didn't think this would all lead to much other than "yup, too hard basket" so this is great!

Well, let's see how it goes.
I've already run into a first problem, where some zip files do not download automatically (in contrast to others, which do), and that must be dependent on some arcane browser setting, which I haven't figured out yet...so....yeah, let's not predict anything

And cheers @Prof. Garfield , I knew you'd think of a better idea, and if not I was gonna tag the rest of the SL to get their thoughts next but you've taken care of that too haha.

Yeah most modpacks don't come with a scenario file as they're intended to modify the existing sandbox game or an existing scenario that comes with the base game. However they are in the minority and most of the downloads would be scenarios or modpacks with scenario files in them so in theory if most can be automatically separated then the leftovers would be a smaller more achievable amount that a human could check and figure out.

Of course then there's the other miscellaneous non-scenario/modpack stuff in the downloads section such as maps (which I think are cross-compatible) and lone graphic files which is a different headache. May be easier to leave them in a generic Civ2 area and just split the scenarios and modpacks up into ToT and MGE sections.

I should also say that many people thankfully DID use good terminology in their titles or posts so some sort of script that searches for "ToT" and "Test of Time" across all the Civ2 downloads would also allow a lot of stuff to be separated too.

.

I feel confident that I'll be possible to organize most things .

 

[Blake00]

I've already run into a first problem, where some zip files do not download automatically (in contrast to others, which do), and that must be dependent on some arcane browser setting, which I haven't figured out yet...so....yeah, let's not predict anything

That may not be your scripts fault, as people may not be able to manually download those ones either. Like I said up in my original top post the whole MGE and ToT being mixed in together is just one if the problems in the Civ2 downloads area, as the others are duplicates and dead downloads where instead of uploading the file directly to the download page the user instead setup an external link to either another part of CFC that's no longer there or another site entirely that's no longer there. I don't know if this would affect what you're doing but there's also an additional problem where some old downloads by @Ansar (who's MIA now) that appear to link to file downloads at the old CFC Civ2 downloads area which don't work unless you open the download link in a new window as for some weird reason Chrome is blocking the the file downloads from opening in a new tab.

.