[MOD] Colonization: 2071

[orlanth]

phew yes , using http via smartgit is working to pull

I initially got messages about failing to access a submodule:

Executing a command has failed
submodule failed (return code 1)

but after extricating various additional firewall git function blocks was finally able to get it to sync completely including the folder for sourceDLL. So with this new submodule system, we can load the project file contained in sourceDLL of the branch, and this should be able to automatically build a DLL for that specific mod branch? I don't know by what dark sorcery you managed that but its an impressive feature

 

[Nightinggale]

So with this new submodule system, we can load the project file contained in sourceDLL of the branch, and this should be able to automatically build a DLL for that specific mod branch? I don't know by what dark sorcery you managed that but its an impressive feature

I made a deal with and surprisingly it only cost one soul. He will show up at your place on monday to pick it up

The submodules are really just pointers to other git "servers". On checkout it will then make new clones at specific locations. It also mean that you have to push each individually as they are 3 servers.

The reason why it knows which mod pulls it is because the source submodule lost most knowledge of individual mods. Instead it will look into sourceMOD, which is part of col2071 git, not the source git. Other mods also have the same 3 header files in there, but naturally they are all different in order to make the mods different.

The brilliant part here is that people can make a mod and have write access to those mod specific files without having write access to the source code. This way we can be more picky about who gets to write to the heart of CMC without preventing mod specific changes.

 

[orlanth]

I think the DLL build went ok; and I merged in the new schemas and adjusted the script to generate XML compatible with the new leaderheadinfos tags, but I'm getting the following assert:

Assert Failed
File: CvXMLLoadUtilitySet.cpp
Line: 274
Expression: GC.getNumControlInfos() == GC.XMLlength
Message: XML read error. "" is used more than once

followed by:
failed to load TXT_KEY_MEDIEVAL_CENSURE

then huge amounts of:

failed to load (null)
Current XML file is xml\text\civ4gamestext_2071.xml

does the change to the language XML mean that no text tags can be empty, or what other problem could this be implying?

 

[Nightinggale]

My guess is that it is responding to the fact that I only added the new language XML file to M:C, yet all mods needs it. Try to pull again.

Another thing. I used to have a script to pull XML schema from M:C to other mods. However it doesn't work with the new setup. Any idea what we could do instead without making schema updates an annoying hassle?

 

[orlanth]

It's still having those series of asserts on loading, which seem related to loading text xmls. Do you think the following one is related to that too?

Assert Failed
File: CvXMLLoadUtilitySet.cpp
Line: 274
Expression: GC.getNumControlInfos() == GC.XMLlength
Message: XML read error. "" is used more than once

I tried editing things to eliminate any <English> tags being empty but this didn't seem to help (not sure why its particularly failing to load TXT_KEY_MEDIEVAL_CENSURE and others like it).

Yeah I dunno what to do about the schemas, other than keep a list of what schemas have been updated in a major revision so would need to be updated for all mods. Fortunately most of them don't result in any mandatory tag changes in XML themselves but there are a few that do such as the recent leaderheadinfos civic diplomacy tags.

 

[Lib.Spi't]

that error looks a lot like the yield error I used to get... could it somehow have gotten the wrong yield list in the SDK?

 

[Nightinggale]

The control assert reveals that controls are hardcoded in the DLL just like yields. I'm not really sure what to do about that other than make a startup test like with yields. That might be annoying to keep up to date.

The commit, which broke col2071: http://sourceforge.net/p/colonizati.../ci/bde7a0a308b5525745eb30cd0842077b94f65f01/
Presumably adding those lines to the XML files will fix the problem.

 

[orlanth]

yes, that solved the CvXMLLoadUtilitySet.cpp assert

However I am still getting XML Errors on text loading, followed by a mysterious loading CTD

In civ4gametext2071_misc I was at first getting:
failed to load TXT_KEY
then a huge number of:
failed to load (null)
I searched through this file and found and removed several isolated TXT_KEY entries which fixed that.

However it's still giving XML errors in civ4gamestext_2071.xml:
TXT_KEY_MEDIEVAL_CENSURE
TXT_KEY_MEDIEVAL_TRADE_TECH
TXT_KEY_NATIVE_TECH
followed by a CTD without other message. These are in the file and I don't see why there would be a load error (I think these items are hardcoded requirements in M:C).

Some other things I have tried in my local copy so far but have not resolved it:
editing the script to remove all non- <English> tags
editing TXT_KEY_MEDIEVAL_CENSURE, TXT_KEY_MEDIEVAL_TRADE_TECH, TXT_KEY_NATIVE_TECH <English> tags to contain some other text

 

[Lib.Spi't]

Yeah i think there is something in the M:C Dll to prevent the NATIVE_TECH being traded or displayed... i think..

 

[Nightinggale]

I just removed all untranslated strings (scripted action). Next I will look into python errors. We use shared python meaning python and DLL shouldn't disagree on enums, but I get an error, which looks like they do it anyway
The error was caused by an outdated DLL. We may reconsider including the DLL as they will go out of date really fast and we will not compile a new one for each mod for every DLL code change.

EDIT: Found the bug. It is missing data in GlobalDefinesAlt.xml. However locating the missing data in the debugger and then adding it didn't help, because it just moved on to the next one. We had precisely this problem before and it will likely not be the last time either. I will look into improving the error message rather than fixing the XML. That way the debugger will not be needed next time we encounter this issue.

 

[Lib.Spi't]

Yeah, I would agree that for the 'Behind the scenes' group that a compiled dll is unnecessary, provided we have a makefile and some instructions that say where to put the important components like the windows SDK stuff, and the fastdep (if that is even used with col or this or whatever.). (I am so far behind right now ) Then actually compiling a functional set of files is not THAT difficult, and once you get it working once (everything in the right place) it always works, the software to do it is free, etc. (the only big problem for me is the third party programs to get the stuff... I always seem to get some thing from a wrong part or a missing part or whatever.)
But I am thick and that is why I have stopped trying till things stabilise and I can find one big red button to push!

So if the files are auto updated then compiling yourself (which i think is way easier then getting the stuff in the first place so if you are brave enough to get the latest version, you should be able to compile it no problem!) is really not a difficult job and takes maybe pushing a button and making a cup of tea to be done and if something is wrong it tells you where it hurts!

So someone can go whoops I will look at that or fixed it!

As for finding testers (who have no desire to do anything remotely related to modding) then you need to make 'Test Release' version that are fully collected and compiled with a debug dll and all that jazz, then you collect all the errors and when you have hit all the latest big ones or ones that make it impossible to really test beyond the error release the next test release version.

I think what has happened is that we currently have so much foundation stuff being laid for the ultimate mod code system (Which is brilliant by the way) that everything has gotten a bit chaotic and no one is entirely sure what is what.

So part of me thinks we need to just wait for the foundations to get laid and settled (like the JIT stuff, or language stuff or new file saving system... I am so lost at what stage you are at right now!) in the M:C platform, then once we know that these new systems (that basically seem to be redesigning the actual game rather than a specific mod... impressive) are bug free and doing as intended wie can then spend a bit of time getting the 'spinoff' dlls functioning properly, then everyone can move forward in awesome fashion together!

But i think while we mess will foundational game code and try to fix mod specific issues as well... I think chaos is going to overwhelm everything...

I would suggest that specific mods take a bit of a coding break and go into mega design mode ready for the new dll, so that we know the foundations of all mods are not broken. Then we look at why the foundation is not compatible with the specific mod...

This is what I have decided to do with WHM, just so that Night can focus on the crazy evil witchcraft he is doing, and get that all nice and neat and stable, rather than splitting his attention so much.

Sorry this shouldn't really be in 2071 thread!

I just suggest it incase anyone else is getting a headache from the chaos I get them easy though so I am not a good bench mark

 

[Nightinggale]

Yeah, I would agree that for the 'Behind the scenes' group that a compiled dll is unnecessary, provided we have a makefile and some instructions that say where to put the important components like the windows SDK stuff, and the fastdep (if that is even used with col or this or whatever.).

Kind of off topic for col2071, but ok. Here is the new setup.

• fastdep.exe is in git and is used to recompile as few files as needed
• jom.exe is in git and is used to compile on all CPU cores at once (instead of just one like Microsoft intended)
• sourceMOD is used meaning no custom Makefile.settings is needed to compile for the right mod.

In short, pull, open the project, select "Assert-Fast" target and compile. No other setup is needed if you installed MS VC++ 2010.
You don't really need to know about the other targets unless you intend to modify the DLL code.

While taking screenshots for cloning, I also made two of how to compile for people, who never tried it before. I intend those to end up on the wiki together with a modified version of the compiler install guide from modwiki. I will get around to write something on the wiki, but it takes priority to modify the DLL to get col2071 running.

 

[Nightinggale]

New stuff arrived while I wrote last post

I would suggest that specific mods take a bit of a coding break and go into mega design mode ready for the new dll, so that we know the foundations of all mods are not broken. Then we look at why the foundation is not compatible with the specific mod...

I have a plan for this. Well git common practice has a plan for this, which I already proposed that we switch to. The answer is branches.

We use two branches: development and master.
Development is the one we commit new stuff in.
Master is the one we merge development into once development is stable.

Other branches might spread out from development, just like Civic Screen. They are merged back into development once the feature is complete.

The idea is that people, who just want a stable DLL will be able to pull master and it should hopefully always run stable.

We will likely set up this branch system after we release next version of M:C.

Oh and to those, who mess up branches: I have screenshots of switching source branch. It shouldn't be difficult to set it to master and then not touch it afterwards

 

[Nightinggale]

I added a new error message to the DLL. Now it will assert with the following message on startup:

Message: DEFAULT_GROWTH_UNITCLASS not set in GlobalDefinesAlt.xml

Mysteriously it only appears in debug builds while assert builds crashes without asserting
Looks to me like a compiler bug, but then again M:C works just fine with the very same source code. We will use debug for col2071 for now, but if it keeps crashing once debug no longer has asserts, then we have a serious problem.

EDIT: Found the cause of the crash. It is in CvXMLLoadUtility::SetFeatureStruct() where some memory allocation fails. The next question is why it fails. Right now I have no idea.

Also I kind of solved the XML schema issue. The DLL now has the MD5 checksum for all schema files and will throw an error if there is a mismatch. This way we will know if somebody else updated a schema file in another mod. I also updated the schema files as the new check requested. It was mainly incorrect indenting, but there were a few lines about civic screens.

 

[Nightinggale]

I pushed a fix for the memory allocation bug. I still don't know why it failed, but I changed the vector of vectors into an array of JIT arrays and now it works just fine. The previous vanilla code should work just fine as well and it used to do so. I have no idea why it suddenly started failing

The good news is that now col2071 will reach main menu once all issues in GlobalDefinesAlt.xml have been solved. Considering the assert message now tells what it failed to find, this shouldn't be a major issue.

One thing about XML editing and committing. Whenever any XML file is modified, prior to committing, do run

sourceDLL\scripts\indentXML.pl

That way we have a standard indenting and we will avoid a number of merge conflicts. I intend to write a pre commit checklist on the wiki with info like this.

 

[Nightinggale]

I think what has happened is that we currently have so much foundation stuff being laid for the ultimate mod code system (Which is brilliant by the way) that everything has gotten a bit chaotic and no one is entirely sure what is what.

Luckily you said "no one" and not nobody, because then you would indicate that I would be "nobody"
I have to say I really like the new setup and the more I use it, the more I like it. It is working like I dreamed up the shared source code should be in the first place, but back then we already had a git server. The current system is partly based on the experience gained with the old system.

once we know that these new systems (that basically seem to be redesigning the actual game rather than a specific mod... impressive)

This has been my design goal ever since I came up with the shared DLL code concept. In fact I'm not sure I have written mod specific code since, except for bug fixes. The only thing we really need is more optional DLL features, or features with different settings. Using something like that we will really gain different mods even if we share the code. The fact that M:C is the only one allowing units to feast is what I mean. We should be a bit more careful that it will not end up with M:C using different graphics. The mods should feel more different than that.

 

[Lib.Spi't]

Yeah I don't think diversity will be a problem, you would be amazed at what can be done with feature X that you never even thought about using it for, especially if it can be configured and modified through xml tags.

Generally speaking I don't think modders are interested in making clones (well most of them) and I think that is why a lot of mods die, a modder comes up against a wall that they cannot get over for their dream feature, so they lose interest in it.

But with solid xml tag designs you can do so much beyond what was initially envisioned by th designer.

One of my favourite bts mod comps is archids slavery mod, because he added a prereq civic xml tag, that shuts down units and buildings when the civic changes, hismod only uses this for 1 civic, 2 buildings and 4 units, but i intend to make this a major corner stone of my unit and building design, so that if you do not think, your choice of civics could bring your whole empire to a stand still, 1 tag... limitless possibilities for diversity!

 

[Sinfullyvannila]

Hey, I love this mod, but the lab bug is ridiculous for me. I can get maybe 1 in 3 games where it doesn't show up.

Any progress on this?

 

[Nightinggale]

Hey, I love this mod, but the lab bug is ridiculous for me. I can get maybe 1 in 3 games where it doesn't show up.

Any progress on this?

The mod is currently being rewritten to use a new DLL file. Hopefully the problem will go away on it's own during that process. If not, then it will be fixed.

 

[PatriotNorwood]

Hello Hello! How is everything going? I am here orlanth just tell me what you want me to do...

Um, since I am helping with this, can I add the Colonization: 2071 link to my signature?