Current and Future Developments

[Kailric]

They will be separate, but we need a version system for Git. What you stated Night is good. I'll fix the alt equipment one. And I'd vote no on asserts. Most people will have no idea what that is and may click the wrong button causing a crash. Then if their system just so happens to explode at the same time we could get sued

 

[Lib.Spi't]

You mean like making a zip with just the changed files since last release? If so, we are not doing that with the next release. Changed files are more or less all of them, including all graphics (they are repacked).

No that is the next realse version, M:C 3.0. At which poiunt all 2.0 stuff is removed from download.

Then we discover that something was bugged/fixed after 3.0. So we update the 3.0 download with the new bits and a date of change and add 3.0a with just the changes and the release date, for those who already had 3.0 before the date of change, thus saving them DLing 100mb for just needing the new dll and 3 xml files (or whatever the patch happened to be)

The DLL for Col is a much trickier situation than with a Civ DLL, because yields are in the dll and yields are all different for mods. So for the most part it is just changing the yield dll file, for each one. the problem comes when yields have been hardcoded in to the dll for such things as AI.

I remember you mentioning that at the moment WEAPONS, and TOOLS are hard coded in the AI, which causes a problem for WH, as we have multiple WEAPON and TOOL types so right now we have 2 useless yields in WH, just to accomodate the AI and make the dll compile. This means either WH needs a seperate dll to everyone else (as in all those AI points changed) or those 'crucial' AI decision making yields need to be changed to an XML tag that defines what yields the AI should consider when considering those specific situations (I presume it is all connected with equipping or making workers and warriors?)

When it comes to 'release packages' I would say it is not really necessary to release the source code, as this is the version for players not modders, the Source code would be a completely seperate download/release for those who want access to the source code for M:C. Or 2071, or WH, or the blank slate USM.

 

[Nightinggale]

I made a draft for the changelog: http://sourceforge.net/p/colonizati...a600a8d1411e7df210b1b6/tree/changelog.txt#l92

 

[Lib.Spi't]

nice list!

Does anyone else feel like this version has moved way beyond a 2.1?

Like all this jit array and all that other stuff, feels like the game has moved much farther than by just 0.1!

 

[Nightinggale]

No that is the next realse version, M:C 3.0. At which poiunt all 2.0 stuff is removed from download.

Then we discover that something was bugged/fixed after 3.0. So we update the 3.0 download with the new bits and a date of change and add 3.0a with just the changes and the release date, for those who already had 3.0 before the date of change, thus saving them DLing 100mb for just needing the new dll and 3 xml files (or whatever the patch happened to be)

I don't get what you are trying to say

The DLL for Col is a much trickier situation than with a Civ DLL, because yields are in the dll and yields are all different for mods. So for the most part it is just changing the yield dll file, for each one. the problem comes when yields have been hardcoded in to the dll for such things as AI.

When it comes to 'release packages' I would say it is not really necessary to release the source code, as this is the version for players not modders, the Source code would be a completely seperate download/release for those who want access to the source code for M:C. Or 2071, or WH, or the blank slate USM.

First you complain that certain stuff can only be modified by changing the DLL. Next you say the source code for the DLL shouldn't be included

We will include the source code as it will be naturally to upload the releases to sourceforge. Sourceforge is made with open source in mind and I think it could be an issue if we skip the source code.

I wonder if we should make the source code a separate zip file. That way people will not complain about it and it will be clear that we included source code. On top of that we get per file download statistics meaning we can see how many just download the mod and how many also download the source code.

The source code can be zipped into 2.2 MB, which includes fastdep.exe, jom.exe and source code for jom. I added the source code as jom is GPL and can be redistributed freely if you include the source code and do not take credit for the work of others. I decided to distribute jom in the source submodule to make the fast targets work without people installing 3rd party software themselves.

I remember you mentioning that at the moment WEAPONS, and TOOLS are hard coded in the AI, which causes a problem for WH, as we have multiple WEAPON and TOOL types so right now we have 2 useless yields in WH, just to accomodate the AI and make the dll compile. This means either WH needs a seperate dll to everyone else (as in all those AI points changed) or those 'crucial' AI decision making yields need to be changed to an XML tag that defines what yields the AI should consider when considering those specific situations (I presume it is all connected with equipping or making workers and warriors?)

I plan to look into this eventually. Hopefully I can make the AI behave a bit more clever when it comes to yields. In fact it might become more self-configuring. There are plenty of other tasks I need to look into first meaning it is a project for a distant future.

WH is NOT getting a source code for itself. That would ruin the whole concept of shared code.

 

[Lib.Spi't]

I wonder if we should make the source code a separate zip file. That way people will not complain about it and it will be clear that we included source code. On top of that we get per file download statistics meaning we can see how many just download the mod and how many also download the source code.

Quote:
Originally Posted by Lib.Spi't View Post
The Source code would be a completely seperate download/release for those who want access to the source code for M:C. Or 2071, or WH, or the blank slate USM.

That is pretty much the exact sentence I just wrote before you said I complained about something I didn't, you lumped two sentences together that were about 2 completely different things

Sentence 1:
This is about a shared .dll, in Civ a shared .dll is much easier, in Col. yields are pretty much central to any mod concept, every mod having it's own yields, which means source files have to be changed for every mod (unless you use an exact set of yields with a different paint job)
This is fine if all that needs channging is the 1 yield file.

However for WH this is not the case, as certain yields are hard coded by M:C (such as armours in the armoury and weapons and tools) So until specific yields are 'de-hard-coded' out of the rest of the source, USM is not possible.

Sentence 2:
Not including the source code was for those who are GAMERS, not MODDERS.

See the difference between the 2 statements now?

 

[Lib.Spi't]

Quote:
Originally Posted by Lib.Spi't View Post
No that is the next realse version, M:C 3.0. At which poiunt all 2.0 stuff is removed from download.

Then we discover that something was bugged/fixed after 3.0. So we update the 3.0 download with the new bits and a date of change and add 3.0a with just the changes and the release date, for those who already had 3.0 before the date of change, thus saving them DLing 100mb for just needing the new dll and 3 xml files (or whatever the patch happened to be)

I don't get what you are trying to say

Don't worry we are pretty much saying the same thing, you are just not understanding me when I say it

 

[Nightinggale]

Does anyone else feel like this version has moved way beyond a 2.1?

Like all this jit array and all that other stuff, feels like the game has moved much farther than by just 0.1!

Me too. However making the game 3.0 is bad too as if it takes that little gameplay to get a brand new version, we will end up with version 50 at some point. Maybe we should release 2.5 instead to tell the distance to 2.0i.

That is pretty much the exact sentence I just wrote

I'm tired. I better stop coding

However for WH this is not the case, as certain yields are hard coded by M:C (such as armours in the armoury and weapons and tools) So until specific yields are 'de-hard-coded' out of the rest of the source, USM is not possible.

The main problem with WH and yields is that it tries to add new types of yields and expects the AI to handle those yields in an intelligent way. Naturally the AI will not handle splitting tools into multiple yields without modifications to the AI code, which hardcodes tools.



I just had a vision regarding yields. We turn yields "upside down" and put virtual yields first. We then hardcode the first X yields.

If we then make AIs self-configuring the yieldgroups, we can hardcode the lower half of the yield enum and then yields past a certain number (like after YIELD_FOOD) we can use XML only. That way we can get rid of the mod specific C++ code for yields and really make a DLL, which really fits all mods without recompiling it.

As it will have a certain number of hardcoded yields, we will not have the performance impact that losing all the hardcoding would otherwise result in. Still performance is a major player here and it might be a killer for this plan. It's a major AI rewrite and it will take time, but it is a nice design goal.

 

[Lib.Spi't]

I'm tired.

That is me 24/7

Now try learning a third party program while someone also repeatedly slaps you in the forehead, and you start to get a picture of what it feels like to be me!

As it will have a certain number of hardcoded yields, we will not have the performance impact that losing all the hardcoding would otherwise result in. Still performance is a major player here and it might be a killer for this plan. It's a major AI rewrite and it will take time, but it is a nice design goal.

So what you are saying is that an xml tag for these AI yields would result in a significant performance hit?
Because it has to look for specific xml references every time it needs to 'think' about the decisions involving these currently hardcoded yields?

I remember in one of the YIELD files, that certain yields were 'grouped' into certain categories.
I don't know what these specific groups pertain to in the code or how the groups are used, but could we do something similiar for the weapons and tools situation?
So instead of it being hardcoded as YIELD_WEAPONS the hard code is WEAPONS_GROUP and then in the yield file we have all the different weapons in WEAPONS_GROUP, and it automatically favours the lowest one in the list(most advanced)
Same with tools.
or at least something like this, so again it will mean only reconfigurng one source file for yields and perhaps not result in such a heavy performance drop as going all the way to an xml tag?
(if that makes sense)

 

[Nightinggale]

So what you are saying is that an xml tag for these AI yields would result in a significant performance hit?
Because it has to look for specific xml references every time it needs to 'think' about the decisions involving these currently hardcoded yields?

Yields are hardcoded in the DLL in the first place due to performance concerns even in vanilla. However now that I think about it, vanilla uses getDefineINT(), which is rather slow. It might not be a problem if we use a proper and fast cache.

One goal of getting rid of the yield groups and replace it with self configuring cache is that such a cache could be updated at any time, like when changing civics. Imagine the AI mining gold and exports it. It then invents mints. After that it stops exporting gold, instead it use gold to make coins and exports the coins for higher profit. In fact it could even import gold to make coins and export those for profit. Such changes aren't possible with fixed yield groups.

Ideally the AI will figure out how much yield to stockpile of each type based on building needs etc. That way yields like tools or weapons will not have a special meaning in the DLL. Instead the AI figures out that it is a good idea to have as many weapons as possible based on what it needs for the professions it is allowed to use. Like the gold example, it will not be interested in YIELD_CROSSBOW until PROFESSION_CROSSBOW is invented and it can see the point in having that yield. Likewise once PROFESSION_CROSSBOW is rendered obsolete by gunpowder units, YIELD_CROSSBOW is something it will try to sell again as it has no other use anymore.

 

[Lib.Spi't]

nice!

 

[Kailric]

For some odd reason Smartgit pushed some sound files. Some of the monks files I was going to add and all the Unit sounds. I don't know what I did to commit those. I added no sounds. How do I undo this?

 

[Nightinggale]

For some odd reason Smartgit pushed some sound files. Some of the monks files I was going to add and all the Unit sounds. I don't know what I did to commit those. I added no sounds. How do I undo this?

git filter-branch --force --index-filter 'git rm --cached --ignore-unmatch *.wav' --prune-empty --tag-name-filter cat -- --all

Followed by a forced push.

The bad thing about doing this is that you delete everything on SF and then push every single change back to SF. I tried doing that, but the push just failed and I started all over
Hopefully it will work this time or we can be really screwed

Once it is done people might experience problems if they continue to work on it. Pull or clone before writing new changes.

 

[Kailric]

Hopefully it will work this time or we can be really screwed

Did it work, or are we screwed? Or am I suppose to do something? I don't understand why it would just randomly decide to add those sound folders. Smartgit is either way more intelligent than I am or it's not all there in the head. When I pushed last time I had options popup that I never seen before and I must have clicked the wrong option.

I started with Tortoise git, and I am starting to miss it

Edit: I was experimenting with some new art for the main menu also and it decided to add that as well. That's ok though as no one had those files.

 

[Nightinggale]

It just failed again. Since it is the newest commit, I will make a branch of the previous revision, delete master and rename the new branch to master. The broken commit will then go into "lost heads" and cleanup should remove it for good.... hopefully.

I think the reason why it did that is that it decided to add all files unless they are in the ignore list. Because of this I updated that one to ensure wav files are ignored (as well as .obj and some other files).

At some point I added .diff files by accident, presumably the same way. However due to their limited size and their content that really wasn't an issue.

 

[Kailric]

It just failed again. Since it is the newest commit, I will make a branch of the previous revision, delete master and rename the new branch to master. The broken commit will then go into "lost heads" and cleanup should remove it for good.... hopefully.

I think the reason why it did that is that it decided to add all files unless they are in the ignore list. Because of this I updated that one to ensure wav files are ignored (as well as .obj and some other files).

At some point I added .diff files by accident, presumably the same way. However due to their limited size and their content that really wasn't an issue.

Ok, so all I do on my end is pull again when you get that done or do I need to do something else?

Edit: Also, I was looking over the "Alternate Equipment" help text to see what needs changed. Actually, we call it "Alt Equipment" in the Pedia so all I need to change is the line "he will equip" to "you can equip" as the player is the one who does the choosing now instead of automatic by the unit. I will hold off changes for now though.

 

[Nightinggale]

I'm having great fun with temporal local branches at the moment. I found the culprit for the problem. It's the DLL file. If we didn't include the DLL, then you would have made a fast forward merge and this wouldn't have happened.

This brings up something which was the topic in the col207 thread recently: when M:C updates the DLL, the DLL in col2071 will be out of date and vice versa. Because of this whenever we pull we really should recompile anyway, so why add the DLL at all? All of us have compilers anyway.

Removing the DLL also saves quite a lot of space on git and hence bandwidth.

Naturally DLL files should be in the release. This is just about what is in git.

 

[Kailric]

I normally recompile after pulling anyway. If removing the DLL will help then lets do it. If someone is missing the DLL we can create a Dropbox link for it.

 

[Nightinggale]

I finally managed to get rid of the broken revision. I ended up deleting git on SF and then upload a brand new module. We better not do that again
I updated git to ignore certain files unless specified to ignore ignored file settings. Well we had a bunch of those, but now we have way more.

Now we have two branches called master and develop. We work on develop from now on and then master should point to the newest release revision. That is the standard git approach. This way testing something in the newest release is just a single branch switch away

Git has other very useful features regarding releases, but right now we should focus on actually making a release.

 

[Nightinggale]

Very important
Please do make a new clone to be 100% sure you don't have the revision with wav files hidden somewhere locally where it might resurface at some point. I really don't want to work that hard to get rid of it again.