Current and Future Developments

[Lib.Spi't]

well done!

I am looking forward to seeing the final results.

 

[knuckledragger]

After much debugging, it turns out that the reason why I can build all the improvements from the start is because the enable improvement tag in CivEffect xml files vanished in the conversion. With no info on how to enable them in xml, the DLL is intended to enable those from the start, which is precisely what it does

There are still other issues, but now with improved debugging features I walked through a bunch of new code, which all happened to work just as intended. I guess that's not all bad. Also I found two locations which begs for performance upgrades. I added TODO comments and will go back to those locations once the gameplay is working again.

Progress! Excellent news!

 

[Nightinggale]

Ok, now CivEffects seems to work. At least it's not like it breaks the gameplay at turn 2. It has two bugs, which both turned out to be typos. One changed +0% modifiers into -100% modifiers, making promotions cost 0 xp. The other compared cost of a founding father with = instead of ==, which changed the cost to 0, making the AI get all of them when you end your first turn.

I'm sure the first one caused issues for everything calling the code in question, but unlimited free promotions is quite hard to miss, meaning I didn't get far enough into the game to find the other issues.

The latter problem would have been a warning if the warning level was set to 4. I have considered doing that once in a while and now I think I will... after next release. The problem is that vanilla was coded with warning level 3 and as a result, using warning level 4 will cause a flood of warnings, which should then be fixed. This mean changing warning level is a whole lot more work than just changing a setting. Still if it had caught this error, then it would have saved me quite a lot of bughunting this time, not to mention frustrations.

 

[Lib.Spi't]

Well done Night!

I look forward to winessing the magic!

 

[Kailric]

Ok, I have things setup on my new PC to do some modding finally.

What I aim to work on is polishing up the current state of the game. I don't want to add any new features until this is done. We keep getting further and further behind on this part and it makes it so the people who play the mod don't understand what is happening or what they are suppose to do. I'll be helping Night finish up the Civeffects thread as well.

 

[Nightinggale]

I had a chat with Kailric about the ability to hardcode xml values in the dll. After that I have been wondering about how to improve the idea to counter all the issues we can predict it might result in. I came up with this:

At compiletime, we either set or not set a flag. Let's call it XML_HARDCODE for now.

When XML_HARDCODE is enabled, the compiler should include whatever it can from xml if it results in increased performance. It's fast, but it requires a recompiled dll if certain xml values are altered.

When XML_HARDCODE is disabled, we should not aim for using the current code. Instead we should reduce hardcoding as much as possible. Ideally any xml can be changed without using a new DLL, including yields. This will allow compiling a single DLL file, which should work on any submod. This will be slower than the current code, but it is a modding DLL, not a player DLL, meaning it is an intended tradeoff.

I'm not sure when to use one system or the other. Perhaps we should set it to follow asserts unless manually specified otherwise. Alternatively we could gain a new xml modder target, which would provide a non-hardcoded optimized dll with asserts enabled (ideal for xml modders).

Kailric's greatest concern appears to be people downloading a release where they start to mod without reading a readme telling them not to. That will cause problems in any colo mod if you touch the wrong xml file. Still we can improve on that as well. I'm thinking of adding a new kind of assert. One, which is only active when xml data is hardcoded and it will bring up an error window if the check fails even if asserts are disabled. Any hardcoded data will then have to be verified through this system, meaning it should not be possible to be hit by hardcoding without being told. People changing xml data should use assert dlls anyway.

I think the best implementation plan would be to postpone this to after next release. It can then be a branch based on the release, meaning once it works, it can be used to produce a dll compatible with xml files from the release. This mean it will be quite useful as soon as the hardcoding of yields is optional.

There is some hardcoding, which should stay hardcoding. AI commands for instance is so integrated into the dll that it wouldn't make sense to change it. I'm not sure about domains though if the mods can't agree on the number. Time will tell if that one turns into a problem.

That is my idea for a concept for now. It's not a promise to code it or anything because first step is that we should agree on the concept

 

[Lib.Spi't]

Yes, this seems like a good idea.

I think getting yields out of hardcoding would be good, especially if we could say replace the hardcoding with something like an AI_TYPE for yields, so we (if possible) remove the hardcoded list of yields to make adding/changing yields easier, but the way the AI thinks about a yield could come from a set of AI Options like:

AI_YIELD_CONSTRUCTION
AI_YIELD_WEAPON
AI_YIELD_TRADE
etc.

Then if I understand things correctly, ai decisions can be made via hardcoded data (faster?) but modders can add as many yields as they like and just set how the AI needs to think about them.
If that makes sense.

I think domain types are a fairly limited concept (Although I don't really know what they are capable of in full)

At the moment I would guess we have 4? (potential)
LAND
SEA
AIR
(Like bts has) and
SPACE

for mods like 2071 where we might want planetary air craft, naval 'inland sea' craft and then the space craft to fly between world islands.

I think any other DOMAIN would require .dll modding anyway for the AI to be capable of using it, or for it to carry any unique game properties (like DOMAIN_AIR) so I don't think these would need to be xml moddable (In terms of adding new ones), they can remain the same as in Bts.

At the moment I can't think what other hardcoding there is entirely.

I think we said
NOBLES
SLAVES
INVENTORS

were hardcoded units.
(I don't know exactly what about them is hardcoded, I am assuming something like luxury food growth, something about inventing and slaves are maybe from capturing? I don't know.)

If they have some hard coded concept link, then in the relevant files we can just say put them at the top of the list, along with a note that says what they are for, what they are connected to and why they need to stay there. (Then a modder can just use that concept however they want, or make them unbuildable(graphical) or whatever to just not use that game concept.

I guess city types are hardcoded at the minute too. This might be a function that would be nice to have some flexibility with also, so you can change the effect that city types have on the game. Having them as Prereqs for buildings is already quite a significant modder ability, as this allows you the potential to tailor make how those cities operate.
Perhaps again making some kind of 'set' of switchable AI types that can be assigned to these town types would be helpful, that makes the AI treat them in ways relevant to the kinds of yields they produce most effectively or if they are focussed on unit production or whatever?
(A bit like what BTS has with it's AI_FLAVOUR settings, that tell the AI to use it if it wants to achieve a certain kind of goal, like military, science, growth or gold.)

Those are just my uneducated thoughts thus far!

 

[Nightinggale]

I didn't really explain what I meant when I mentioned domains. Yes they do need DLL modifications. However my plan so far has been to add a compiler flag WITH_DOMAIN_SPACE and if that is set, it will compile the space domain code. If not, then it will not be compiled. The domain code touches some performance critical parts and adding space because one use space, air because another use air and so on is doable, but would hurt performance.

I'm not overly concerned about this. I'm quite sure we will figure out something if/when it becomes a problem.

As for other hardcoding, sure we should get rid of that in the dll. There is some leftover feast thing, which has a yield cost using only M:C yields. It's only compiled when including the M:C header. That's not the right solution, but at the time I didn't really want to spend ages redesigning that part of the code. It's not like it's critical.

 

[Kailric]

There is some leftover feast thing, which has a yield cost using only M:C yields. It's only compiled when including the M:C header. That's not the right solution, but at the time I didn't really want to spend ages redesigning that part of the code. It's not like it's critical.

I remember this Command idea I had, but never fully implemented it. You know how in Civ there would be a "Celebrate The King Day" if the conditions where right and you would get some bonus to that city, well Holding a Feast would give similar bonuses to a city for a short time. This could easily be changed to something more fitting to other mods. You are basically giving back to the people to win their gratitude.

Anyway, yeah that is a good idea on the hardcoding xml stuff. The greatest percent of users don't make any post, unless something is broken and they really really want to play, so we really don't know what types of people are playing the mod. And the mod gets downloads everyday, 2 more today. Some of these users will want to tinker with the xml values and those are the ones we need to look out for.

 

[Nightinggale]

Kailric's computer decided to riot and will hang fastdep whenever it gets near it for no reason whatsoever. This makes compiling problematic. Since fastdep works for me and the hang issue even hangs the debugger, I have decided to give up on solving the problem. Instead I wrote a fastdep in perl. It seems to work and hopefully Kailric can actually be useful from now on.

I'm not sure what baffles me the most. The fact that fastdep just hangs for no reason or that I managed to recreate fastdep overnight. The source code for fastdep is massive and not that reader friendly. My recreation is just 118 lines of perl.

The makefile will still used fastdep.exe unless specified otherwise in Makefile.settings. More details in the git log.

And the mod gets downloads everyday, 2 more today. Some of these users will want to tinker with the xml values and those are the ones we need to look out for.

Speaking of downloads, I released RaRE less than a month ago. 38 downloads so far, which I guess is quite good considering all I did was tell about it in a forum post

More interesting is the download statistics. It's way more than just a number: link. Worth considering for a host for releases since there is no approval delay, unlimited space and stats like this.

 

[Kailric]

More interesting is the download statistics. It's way more than just a number: link. Worth considering for a host for releases since there is no approval delay, unlimited space and stats like this.

Those statistics are cool. I guess what's cool about mod data base is that you can add screen shots and such. But, the main reason people know about the link is because they come here first and we can put all the screen shots here and then set the link to sourceforge. We can set that up on the next release.

 

[Nightinggale]

SF supports screenshots too. A feature it is nagging me about not using since I actually released something

I think getting yields out of hardcoding would be good

I think so too. What I can't figure out is the performance impact. Right now if the dll tests something with yields, it knows the id and hardcodes it into the dll. Say we assume it to take a single cycle (it might take 0). Replacing it with runtime read numbers mean it will have to use getYieldInfo and then some function from it where it reads some memory. If the needed data isn't in the CPU cache, it will take thousands of cycles. Since yields are used quite a bit, I suspect we will be able to see the game slowing down. If nothing else, it would be stupid to hardcode yields in vanilla if it can't be justified with performance concerns.

The DLL is quite fast, which mean a slower dll would be ok for testing purposes. Nobody said anything about testing on the biggest possible maps. Also I would be very surprised if the performance drops to col 2071 speed (the last release).

 

[Lib.Spi't]

It may well be hardcoded, because the 3-7 yields in bts are hardcoded, therefore they just modified the existing yield system into a Col. one, There probably is something of a performance effect, as there are 30 odd yields, so anything like that amount would mount up, it just depends if it is any worse than many of the other xml data points..

From my understanding the slowest part of things is the AI 'looking' at things to see what is available, then running that data through some kind of decision code, then picking the one that has the best number result.

So my thinking is if we could make some AI codes, that treat yields as say one of 8 decision concepts. (Just like it does with units and buildings in BTS via AI_Whatever and FLAVOR_Whatever.)

Then that might speed up any lagging processes due to number of yields, because actually they would only be one of 8 'types'. (As there aren't that many things you do with yields, some are just better/worse at making money, growth, or are for buildings, units, or production input. That's Six AI Categories.)

 

[Nightinggale]

Then that might speed up any lagging processes due to number of yields, because actually they would only be one of 8 'types'. (As there aren't that many things you do with yields, some are just better/worse at making money, growth, or are for buildings, units, or production input. That's Six AI Categories.)

That's essentially the idea. However the lag comes from looking up the yield category at runtime rather than at compiletime.Hardcoding vs xml runtime settings is basically just a matter of runtime requires to read from memory with the lag that brings. Hardcoding allows writing the number directly into the program code itself, hence no memory read delay.

 

[Kailric]

What I aim to work on is polishing up the current state of the game. I don't want to add any new features until this is done. We keep getting further and further behind on this part and it makes it so the people who play the mod don't understand what is happening or what they are suppose to do. I'll be helping Night finish up the Civeffects thread as well.

Ok, In this respect I'm working in the Trader AI branch. I have disabled linked cities with the CityTypeYields in BuildingInfos as well as the Domestic Market feature stuff. I have turned off Prospecting as well. All these features needs more testing as well as being well documented so that players understand what is happening if we decided to keep them.

From now on if we add new features that will become a permanent part of the mod it should be required that the pedia and tutorial should be updated to explain these changes.

I'm going to be testing out the mod as it stands now and testing the Trader AI stuff just to remember where we are in this mod.

 

[Lib.Spi't]

It's beginning to feel a lot like a modding Christmas! Every where you go!

It seems like everyone has come back to modding in the last few days. How exciting!

 

[M07]

It's beginning to feel a lot like a modding Christmas! Every where you go!

It seems like everyone has come back to modding in the last few days. How exciting!

Hello Lib.Spi't and everyone,

I was wondering if the other modders were still devellop this game and apperently yes.
I am glad to see that. You are may be right, it is like a modding Christmas

See u

 

[Nightinggale]

I just had an idea about adding more domains. Right now the code use switch-case setups on domains. This mean each domain has to be explicit mentioned in the code and made hardcoded. It suddenly occurred to me that we can add a "default" domain in the code. This code will be used whenever the type (ID read as int) isn't mentioned in the hardcoded code. Adding a few basic rules like the unit can only move on the same domain as it is itself (+cities), then we will have come a long way towards allowing adding more domains in xml only. This would help towards making a generic "xml modder" dll.

See u

This is great news indeed. I feared DoaNE was dead due to the rather idle forum.

 

[Nightinggale]

From bugreports thread:

I think I should figure out a way to write info like that in the dll, which will then put it in the savegame. However that is a future task.

I went ahead and researched this anyway

I found a line of code, which produce this line/string:

MC_2.5.41-416-ga25839bf7c7ba53598b1a8a16402a6a90b79233e

First is the last tag, next is the number of commits since that tag (wow, 416 ) and last is the sha1 code of the commit, which is a unique identifier for each revision. Needless to say, adding tags to each branch makes this string more human readable. None of us can at a glance tell that it is the CivEffect branch.

I'm not sure why the sha code starts with g, but if you copy paste the number except for the g and search in the log in smartGIT, it will find the revision.

I'm thinking of adding more instructions to the makefile to generate this string and place it in a header file. It can then be saved in CvGame as this will save it once in each savegame. On load, we can always see which dll file, which was used to make the savegame, possibly with assert, debugger, log or whatever we want to check it with. The method of checking isn't important, it's more how to add the info to the savegame in the first place.

I think it will be fairly fast to generate this info. However it will add requirements to whoever compiles. It would require both perl and git to be present. Installing smartGIT and Strawberry Perl should be enough, but right now the dll can actually compile without those. Can we agree on adding these requirements for compiling? THese requirements applies only when compiling the dll itself, not when using the dll for playing or xml modding.

 

[Kailric]

Can we agree on adding these requirements for compiling?

Sounds reasonable.

I just pushed another batch of textmgr additions. That's a pain in the rump. There is still a bunch to go so if you want to knock some of that out it be great. The getAllowsImprovementslooks the most confusing and I've been skipping it