Current and Future Developments

Howdy all, its been really quite here for a while. I've been all kinds of busy of late as well as acquired a new pc so I'll have to reset up everything on it eventually. I fired up M:C the other day to check out something and it got me itching to work on it some more. Anyway, all in do time. If you read this drop a line and say hi and what you've been up too as well.
 
Kail!

Good to see you around! :)

I have been up to very little of late, after finding a couple of bugs in FTTW that I couldn't fix I decided to take a break from modding, I just wasn't enjoying myself.

So what little productive energy I have had has mostly been focussed on my writing, as well as a brief sojourn into Space Engineers.

I am trying to rebuild my motivation and energy to get back to doing.. something.

Look forward to seeing the next step with M:C!
 
fhew glad yer still here :):king: I was afraid civfanatics being offline had driven off all the Elder Statesmen of Civ4Col modding :eek::crazyeye:
I'm absorbed in work stress etc atm, but still alive/around & hope Night is too!
 
Lib, orlanth, great to hear from both of you. I'm sure Night is out there in cyber space somewhere also. Yeah, I jumped on to make some posts a while back then got the news that atomicgamer was going down. Seems strange that they would just take it down, why not sell it? Perhaps no one would buy. Anyway, that reminds me I need to upload some of my files to a new server as I was using atomicgamer for a few of them (which I redownloaded soon as I heard the news).

I have a big project I have to complete this month so I want have any time for modding, but anyway, I have this cool dream that if we ever get M:C to a point to where we can say "we're gonna do no more" then it would be cool to create a document that detailed M:C's development We could take quotes from the forum here, the arguments, the funny things said and interesting conversations that came up, adding screen shots and such as well. We've all contributed in writing so much here that it would be awesome to immortalize some of it. I'm sure many big mods could have such a document made. Hmm, makes me think maybe you could hire out to create such documents :)

Anyway, again, super great to hear from you guys! Still Alive!
 
I’m also glad to hear from you all.
I’m also still here, autumn will be busy for me but I continue playing to find issues that are worth commenting to prepare my next report.
 
I'm sure Night is out there in cyber space somewhere also.
I was about to post something asking if everybody were abducted by aliens or something and then it turned out that it was actually the forum deciding not to inform me of new posts.... again :(

The status for CivEffects is that I finished it (more or less), then I started playing and ran into a few issues. I then sort of ran away screaming and needed a break. It was so hard to make and now it appears that there are a few bugs, which triggers in the background and that's the hardest kind of bugs to debug. Nothing will come of postponing looking at it though :badcomp:

Yeah, I jumped on to make some posts a while back then got the news that atomicgamer was going down. Seems strange that they would just take it down, why not sell it? Perhaps no one would buy. Anyway, that reminds me I need to upload some of my files to a new server as I was using atomicgamer for a few of them (which I redownloaded soon as I heard the news).
Remember that we have free server space on SF.

I have this cool dream that if we ever get M:C to a point to where we can say "we're gonna do no more"
I don't think we will ever get to that point. If development stops, it's certainly not due to lack of ideas of stuff, which would be awesome to add to the game.
 
The status for CivEffects is that I finished it (more or less), then I started playing and ran into a few issues. I then sort of ran away screaming and needed a break. It was so hard to make and now it appears that there are a few bugs, which triggers in the background and that's the hardest kind of bugs to debug. Nothing will come of postponing looking at it though :badcomp:

Swell to hear from you Night, hope all is well!

Did you even finish the grunt work that I started working on with the help text stuff? Sorry I've been lazy on top of preoccupied with other things and didn't finish that.

Remember that we have free server space on SF.

Ok, so we can up load what ever we want? The links to some of my mods are still broken because of atomicgamer going down so I still need to fix those.
 
Did you even finish the grunt work that I started working on with the help text stuff? Sorry I've been lazy on top of preoccupied with other things and didn't finish that.
I can't even remember that we were about to do anything about that, though it sounds like a good idea. I'm pretty sure I didn't do anything other than CivEffects and then got into a bad mood when it didn't work as expected. It was something about working most of the time and then suddenly it did something wrong and at that time I totally didn't feel like debugging for a semi random event. I'm still not happy about doing so, but the only alternative is discarding everything, which is even worse.

Ok, so we can up load what ever we want? The links to some of my mods are still broken because of atomicgamer going down so I still need to fix those.
It has to be project related or at least semi related. A subfolder in the download section with mods and modcomps you made, which is used in M:C could fit that description... I think.

I added a new git repository at some point for fastdep. It couldn't handle relative paths and absolute paths broke the line length limit. I forked the source and fixed that and then a few other issues. Being a fastdep custom made for M:C, I decided that I it could be part of the project rather than starting a new project for less than 20 commits.
 
Debugging CivEffects is somewhat tricky and I decided to help myself debugging before really giving it a try. I decided to add all XML types to enums as this will give the XML Type rather than the int index during debugging. I'm working on a perl script, which reads all XML files in a list and then generate a header file based on that. Example output:
PHP:
enum FeatureTypes
{
	NO_FEATURE = -1,

#ifndef WITHOUT_XML_HARDCODING
	FEATURE_ICE,
	FEATURE_JUNGLE,
	FEATURE_FOREST,
	FEATURE_LIGHT_FOREST,
#endif

	NUM_FEATURE_TYPES = 4,
	FIRST_FEATURE = 0,
};
I added a few clever abilities to this on top of just being a regular enum.

WITHOUT_XML_HARDCODING isn't set in the project. This allows the debugger to read all the numbers like the #ifndef isn't present. However it is set in the makefile, which mean if any of the individual features (or whatever) is used in the code, the compiler will cause an error. That should prevent accidental hardcoding to just a single mod.

FIRST_FEATURE or FIRST_* is something I started adding to the enum. It allows code like
PHP:
for (FeatureTypes eFeature = FIRST_FEATURE; eFeature < NUM_FEATURE_TYPES; ++eFeature)
That will result in cleaner and more readable code as and less typecasts. The last is important here because typecasts can hide bugs. Vanilla uses typecasts a lot and it can easily typecase UnitTypes to UnitClassTypes without anybody noticing. Without the typecasts, incorrect assignments like that will cause a compiler error.

The new header file is written in memory and once it's complete, it is compared with the existing one. It's then only written to the file if they aren't identical. This prevents the compiler from detecting the header as modified, which in turn prevents unneeded cpp recompilations.

I filled the list of xml files and it consistently use around 0.50-0.51 seconds to run through all xml files. That's not a lot and that got me thinking. If this script is called from the same location as fastdep is called, then it is called once every time the compiler is used. If MSVC is asked to start the game, it will compile and then start. The compilation is near instant if the DLL will not have to be recompiled. This mean if modders always start the game from MSVC, we will know XML and DLL is in sync, regardless of the modder working on DLL or XML edits.

In other words, if XML and DLL modders alike opens the project file and hit F5 to start the game each time he/she edited anything, then the compiler will know XML length and optimize for those, which in turn will make the code run faster. Also it makes DLL coding easier and helps a great deal regarding memory management. In fact I would be able to store XML info sequentially, which also would make the game run faster. Say we loop all units to look at one specific number. The CPU cache (hardware) will notice this pattern and start requesting info on the next units before the CPU reach the point where it needs those info. This does make looping all infos significantly faster than it is right now where each unit is stored in a random location and as such there will not be a pattern when looping.

What do you say? It it feasible to start the game from the project or should this enum setup stay for debugging only?
 
Have you even tried to start the game from MSVC?
Sure. I wrote that perl script to set up the paths to do that. I uploaded it to one of the M:C threads, though I don't recall which one. I plan to add it to git too, but I wanted to finetune it or something first.

Naturally I made sure I can start the game from MSVC before proposing making it a standard. If that part fails, then the whole concept fails and I usually try to think things through enough not to overlook something that essential before posting about anything.

I am still on my new system and haven't installed anything to be able to mod yet.
Install BTS and colo in non-default locations, like C:\games.
Using program files will cause git to get write permission issues and by using the game MOD folder rather than the my documents one, the perl script will automatically find the game exe and set up paths correctly. Also it seems that some people get odd issues with the my documents MOD folder, particularly with BTS, but some also with RaR. Using the game MOD folder has no known issues as long as it is installed outside program files. A short path with only standard ASCII (read: English) characters should be used. Nothing weird like ö as that too can cause issues.

Just some advice to all installing anything using the civ4 engine. It can really prevent problems later on ;)
 
Yes the my documents issue with bts was when changing resource files (HUD colours/appearance and such) it would blow up in my documents.

Apparently Archid fixed this some how.. I have no idea how he did it!?! but yes that is good advice for all the mentioned reasons!

I am having fun with crashes at the moment... FTTW works fine on two of my computers, put it on someone elses out there on the interweb and it crashes!! with the exact same build... I am so utterly confused with everything at the minute!! :wallbash:

I want a new version of M:C to come out so I can be the one to report headaches, not the one who has to figure out why! :D
 
I am having fun with crashes at the moment... FTTW works fine on two of my computers, put it on someone elses out there on the interweb and it crashes!! with the exact same build... I am so utterly confused with everything at the minute!! :wallbash:
Those bug reports are fun. It can be anything from install location, failure to update the exe to OS specific issues. M:C is 64 bit only even though it is a 32 bit application. However if all the graphics are unpacked, it will work on 32 bit.

Sure reporting is a whole lot more fun than hunting the bugs. However making the game good enough to allow nothing to be worth reporting is even more fun :king:
 
"Setting up everything" is not as tricky as you make it sound like. Just follow the guide https://sourceforge.net/p/colonizationmodcollection/wiki/GIT/
I'm going to add perl to that guide at some point, but for now, just remember to install this: http://strawberryperl.com/

That should do it and since you already used git and stuff before, it shouldn't take long. It might take a while to download everything from git, but you can copy your checkout from your old computer. That way it will keep your SF account settings and stuff as well as save you from downloading all the old revisions.

Notepad++ is also a good thing to have installed.
 
More or less done with the script to generate enums to match the xml files. I'm not going to commit it just yet as I still require some testing to ensure that it works as intended.

Now the DLL will not go out of sync with the xml files anymore. However it can go out of sync with the exe as the exe has some of the enums hardcoded (bad, but there is nothing we can do about that). I plan to add compile_asserts. That's a boost function where the precompiler fails to compile if the argument is false. Quite useful for ENUM_VALUE == 5 and similar. It would likely have to be in _xml_check.cpp or similar as we would like to check ASAP. I'm not sure if it is a good or bad idea to place it in _precompile.cpp.

We need to clean up both the code and the xml files. We currently have both EuropeTypes and TradeScreenTypes and they appear to be somewhat interchangeable. EuropeTypes should die.

The XML files are inconsistently named. Some are singular (ends in Info.xml) and some are plural (ends with Infos.xml). Some are named the same as the enum type, some nearly the same, but not 100% and some have totally different names like TradeScreenTypes is defined by CIV4EuropeInfo.xml. Once all branches have been merged into one, I propose renaming all XML files into a consistent format. Removing the CIV4 prefix would be good because of the ability to go to a file by pressing the first letter in the name. Starting all of them with C is no good for that.

GlobalTypes.xml needs cleanup as well. It requires sync with enums, which is bad by design. Right now TradeScreen in that file is out of sync with both EuropeInfo.xml and the enum and the enum doesn't agree 100% with the xml either. I propose removing GlobalTypes.xml and store the types as Type in regular xml files, possibly BasicInfos. That will allow my new script to keep the enums in sync and mods will then only have to set types in one location meaning no desync bugs in setup.

Edit:
It seems that CvMapGeneratorUtil.py is the only one using the world size enum. Here it is used to set 4 iGrain. Move those to WorldInfos.xml and make the DLL not use the WorldSize enum. It might also be good to figure out precisely what the grain does to the map making scripts. Something about adding plots in regions.
 
I created a new branch called xml_enum_gen. It is as the name indicates a branch, which introduce a perl script, which generates an enum for each xml file. It's work in progress, incomplete and highly experimental. I will work a bit on this branch and when/if I get it to work as intended, then I will write about it here and merge if there are no sideeffects.
 
I am having fun with crashes at the moment... FTTW works fine on two of my computers, put it on someone elses out there on the interweb and it crashes!! with the exact same build... I am so utterly confused with everything at the minute!! :wallbash:

Just downloaded it and will try it on my laptop...4720HQ, 12GB RAM, SSD, 970M graphics. I will report.

Edit: Does not load for me :( I have Win 8.1
 
After much debugging, it turns out that the reason why I can build all the improvements from the start is because the enable improvement tag in CivEffect xml files vanished in the conversion. With no info on how to enable them in xml, the DLL is intended to enable those from the start, which is precisely what it does :wallbash:

There are still other issues, but now with improved debugging features I walked through a bunch of new code, which all happened to work just as intended. I guess that's not all bad. Also I found two locations which begs for performance upgrades. I added TODO comments and will go back to those locations once the gameplay is working again.
 
Top Bottom