Current and Future Developments

Git log said:

GameFont: More XML setup and billboard detection

Each class with getChar() can now set iChar in XML.
This mean that you can take say a specific building and assign an icon outside the building row.
The new char could be anywhere, including rows used by other classes (like yields)
It can also be used to set a single icon in the billboard range while the rest of the row is outside (preserving billboard range IDs)

More options in CIV4GameFontInfos.xml
Ability to turn the GameFont debug screen on and off (no need to edit python for that)
Set the ID of the first icon to display in debug screen

Debug screen now has a row telling if the icon can be displayed on billboards

Click to expand...

I added a screenshot of how the debug screen is displayed now. Accessible tells if any getChar() will return the icon in question. Billboard tells if the billboards can display the icon in question.
If you haven't guessed, happy icon mean yes and unhappy mean no.

I think it should now be much easier to edit GameFont. At least there should be no more trial and error to get the icons to be displayed ingame.

I added Units to GameFont and made Domestic advisor use those. I also made GameFont debug mode enable all columns in Domestic advisor (no more hiding yet to be invented yields and so on). As I consider my GameFont work to be done, I merged the work into develop.

There is one thing, which makes me wonder. There is graphics for an alemaker (a guy holding a barrel). I think I have used that unit ingame, but now it looks like no units will use those graphics. I added the icon to GameFont as a spare one in case this is fixed.

Again, nice work. I had removed that Ale Maker when the Tavern no longer produced anything. Did you use the Unit Buttons? Wow, that would be shrinking them really small.

I merged delayed diplomatic response and trade relations into develop. I think we have the newest code in develop right now. No more feature branches. This mean there is a lot of new stuff to play around with as well as testing

Kailric said:

Did you use the Unit Buttons? Wow, that would be shrinking them really small.

Click to expand...

GameFontEditor can import imagines. You just click on the slot where you want it to be and then click import. It can use tga, dds and png. This mean I could import buttons directly. It complains that the size is wrong and ask if I want to resize. It does a much better job at making such small icons than I expected. It is a whole lot easier to tell the units apart than I thought when I started.

I started getting rid of the yieldgroups. I have removed the billboard one and the billboard offset one. Yield XML now has tells if a yield should be put on the billboard.

Next I planned to remove native yields and put those in XML too. However my plan broke, when it turned out that we have 2 yieldgroups and one XML tag and they overlap in some cases, but not all. Rather than making 3 XML tags I think it would be better to stop up and rethink what we actually want the natives to do with yields. I think it would be best to forget about the code, (which is a relic of multiple changes btw) and start over. How should natives behave differently than non-natives when it comes to yields? How do we set that up in a clean way in XML?
Once we know that, we can start to think about the code itself. Coding without clear goal will go wrong eventually.


I added new functions to CvPlayer.
They do overlap a bit with yieldgroups and that is intentional. Eventually such functions will replace a bunch of them.

The good part about these functions is that they reply based on allowed professions. This mean unlike XML and yield groups, the reply from these functions depend on current tech level meaning they can be set differently for each player and even change for a player during a game. Hopefully this will eventually make the AI more adaptive.

Another nice feature about these functions is that they are self configuring. We don't have to tell the AI what to do with yields. It will be more likely to figure it out on its own. Improved AI and less XML modder work. Sounds perfect.

Also in case you haven't guessed: this all relies on a cache, which is recalculated when civics change. This mean we shouldn't see a noteworthy performance drop from not using hardcoded yield data anymore.



Another thing I changed is the XML schema checksum. I got tired of updating checksums and get merge conflicts. I added sourceDLL\schema and placed two perl scripts there. One to copy all schema files into this location and one to copy them back. The DLL will then calculate checksum for both locations at startup. It skips all checks where it fails to find one of those, which needs comparing meaning the DLL isn't broken if the source code is moved away.

nice one night, onstep closer to fixing WHM yield issues! Whoot!

sounds good

I don't think natives need to behave in a totally different way from other civs regarding yields; with trade becoming more important it makes the most sense to have a single system that can apply to everyone, so that each civs demands and prices can react more flexibly to what they actually need and produce rather than being stuck with fixed/artificial hardcoded values.

If you want to be able to have some difference between natives and others regarding prices, instead of having a totally separate and inflexible <iNativeBuyPrice> and <iNativeSellPrice> in YieldInfos like vanilla you could have <iNativeBuyPriceLow> and <iNativeBuyPriceHigh> , and let these work using the same supply/demand system of other civs, while also being able to set special price ranges for them however you want. (Perhaps the intermediate "Minor Civs" would have price ranges midway between those of native and "Major" civs.)

A lot of the vanilla hardcoding like the function for "native yield destruction" is no longer needed, since it is effectively replaced by the Domestic Markets demand system / consumption, which meets the same goals while being a lot more moddable (ie you can much more easily adjust and balance demands for native or other civs to achieve any result you want to). If you change Native civs to using regular demand system instead of this, then maybe <iNativeConsumptionPercent> can be converted to instead give natives a percent modifier to certain yield demands if modders want to make these different for natives. I guess <iNativeHappy> could still be a useful tag to enable diplo effects from trading (it could even be interesting if this can be a negative value for some yields, to allow contraband yields such as Opium where trading may be highly profitable but can antagonize the government of the city where you're selling it)

We now have a trade screen class (WIP). I wonder if we can add that to CvPlayer and use that for domestic prices. The AI will then trade using those prices, except they will buy at -2 and sell at +2 (or something). We can then use regular unit and building demands to control prices.

Natives can then have different min and max prices, though any player having a demand for a yield he can't produce himself will naturally increase the price quite a lot.

We can then use the new functions to generate a "fake" demand for yields needed for professions. If the AI can use leather armors to make infantry, then it will be willing to buy those and stockpile. They will however not buy scale mails until they invent a profession, which demand those.

Now that I think about it, I think the new yield functions are too simple. To figure out how much the AI wants of each type they should likely tell how much the most expensive profession cost of each yield. It would also be useful to know the power and combined costs if you make one of each profession. That should make it easier to make the AI figure out which yields it wants to buy. For instance it would pick scale mail over leather armor because scale mail results in more power. At the same time leather armor isn't useless and if they are cheap enough, they can come up with the idea to buy lots of them to have many cheap troops instead of a few expensive ones.

I'm not sure if it already does this, but with the greatly added number of professions, it would be a good idea if AI troops check each turn if it can change profession into something with greater power. That way it will replace armor of existing troops when inventing something better.

Yields in XML also needs a cleanup. For instance bIsMilitary is the same as iPower > 0 and as such do not need to be set in XML. bIsMustBeDiscovered is calculated by the DLL on load and the value in XML is ignored. bCargo is also set in XML, yet ignored by the DLL. There are plenty of minOccurs="0", which isn't supported by Civ4 XML Editor.

Looks like my little task of moving yield groups into XML has turned into a massive task

EDIT: added bIsLightArmor and bIsHeavyArmor to XML and removed the 3 yieldgroups regarding armor. I looked at some of the AI code using armor checks. Looks like the AI looks for YIELD_WEAPONS, YIELD_HORSES and isArmor(). Looks to me like we should replace those with CvPlayer::isYieldMilitaryCost(). That will make the AI dynamically produce and import yields used to make troops and we will not have to hint which yields can be used for this.

I have now removed 5 yieldgroups and we have 7 left. 6 of those are related to trading meaning rather than moving those to XML, it would be good to render them obsolite by implementing a better AI trading system. The last yieldgroup is luxury food. Both Kailric and me have proposed the idea of having it set at runtime to make it change with civics/civilization or something else. Yieldgroups are hardcoded and can't change during the game, which mean a changing luxury food would render that group obsolete.

I added a new feature to feeder service. In import/export I added a new spin button to the top of the window. This number tells how many turns you want raw materials for.

Example:
You set it to 4 turns. You have room for 2 carpenters and they each consume lumber each turn.
Your import threshold is then set to 2*3*4= 24 lumber. This is true even if you do not have any unit actively working as a carpenter.

This is useful when using feeder service to supply your buildings as it will automatically increase threshold when you build a new building, which gives you more potential workers and more work from each unit.

While you might find this useful, your usage of this feature is nothing compared to what the AI can gain from using this. It can move raw materials to the cities, which can process them into something better. It can loop all cities and add up thresholds and storage. It then wants to export storage - threshold, which mean it may consider buying if that number is negative. It will not export yields used to make professions (like YIELD_WEAPONS).

Looks to me like there is great potential for an AI, which can become self configuring regarding yields without any special info from modders. This mean most of the yieldgroups can be replaced by this setup while at the same time the AI will no longer need to cheat as much (build buildings without using stone). On top of that the AI might do better than it does right now because it will avoid some limitations. For instance without lumber, it can't make a carpenter. With the new system, it will detect a shortage of lumber because some cities want to import and it can encourage production in the cities where it is able to do so. No city will then decide not to make a carpenter just because the city has no lumber.

Two known issues:

• the new threshold is not in the GUI (it's not even in python)
• I added a new string name, but not the string itself

Comming up with things to better teach the AI is a good thing for human players. The hard part is comming up with such things. I hope it works.

I worked on the Russian translation branch rather than yields as it turned out displaying Russian chars were buggy. I located the cause of why the DLL didn't read UTF-8 and fixed it. At the same time I fixed all the vanila chars like é and ö. We should be able to use all of them by simply writing UTF-8 in the XML files.

Usage of escapes like & #198; is not only no longer needed, it is unsupported. We have to convert all of those cases eventually, but for now I just did it for English. We will get asserts if they are used anyway and a ¿ will be written ingame.

The forum seems dead and there haven't been any updates in a while

I fixed a number of issues related to unicode and Russian appears to be working ok now. There is an issue where .upper() will not change the text to uppercase. This is due to python expecting unicode and we can't use the correct numbers for unicode because we can only use IDs usable on billboards. I have written my own function to do the task. Now we just need to change all myString.upper() to gc.toUpper(myString) in python. There is a gc.toLower() as well. So far they are only in Russian_translation, but they will likely end up in develop quite soon.

There is a known issue where Russian font is always white even when the game requests other colors. No idea what to do about that yet.


I investigated why yield icons on the map can be all black in RaR. It turns out to be a game limitation on some graphic cards, though precisely which and why is unknown. The issue is that resource_icons_64ea.dds is limited to 32 different icons. After some experimentation I learned that the limit really is a width of 2048 pixels, which just happens to be the same as 32 icons. Scaling down the icons to make more fit in 2048 pixels will allow more icons. I'm considering making the DLL use resource_icons_64ea_scaled.dds if "Low resolution textures" is used. That way we can make releases where yield icons are full resolution, but people can switch to lower resolution files if their computer can't handle it. This will give us the freedom not to consider this limit.

Well, at the moment my SmartGit is frozen. It is requiring me to purchase the full version and I really don't want to do that. There is an option to select the none commercial version but it is greyed out. I seen on the net where you can delete the settings folder and try to reset it but I don't see any settings folder. It's always something!

I just want to play test a while any way. I've added so many new features that I really just want to test them all out.

I hope I can get the smartgit running again, if not I guess I will have to go back to turtle git, or tortoise.

Hey Kail, would love to get my hands on your latest version to test, if you are willing to upload it and send me a link that would be awesome!

Lib.Spi't said:

Hey Kail, would love to get my hands on your latest version to test, if you are willing to upload it and send me a link that would be awesome!

Click to expand...

Ok, let me play test a bit to see if there are any immediate failures and we'll hook you up one way or another. I've been on vacation for a while and trying to get caught back up on all my stuff so I haven't had much time for M:C yet.

I did just now discover how to reset my settings for SmartGit so it looks like I am back in business.

It is so aggravating when you look up something that you need help with and the help providers speak to you like your are a computer guru and know what they are talking about. If I was that savvy on computer ins and outs I wouldn't need help. I have no idea where the hell %AppData% is on my computer, or even if it is on my computer, or if there is just some kind of weird error producing the % signs, or if %AppData% is some kind of special guru lingo for something else. But, I finally found a site that gave me the help I needed. Now, I can start up SmartGit once again.

yeah I think that appdata is a 'hidden' folder, so you first have to turn on show hidden folders somewhere, I forget where, then you suddenly discover there are a whole load of new folders in you c: drive

Kailric said:

It is so aggravating when you look up something that you need help with and the help providers speak to you like your are a computer guru and know what they are talking about.

Click to expand...

Yeah, now you imagine what I feel most of the time when talking to you or Nightinggale.

I know you both really try to put things as simple as you can for me, and I appreciate it. It's just that sometimes it takes time and effort on my side to realize it.

Remember that "The 13th warrior" movie? It's where Mr.Banderas was sitting by the fire with those Norse men and listened to their conversations in their language he did not know, and then he started to pick up some bits here and there, and more every time, and finally he learned the language.

That's exactly what being on this forum feels like to me.

Nightinggale said:

There is a known issue where Russian font is always white even when the game requests other colors. No idea what to do about that yet.

Click to expand...

Well one idea you mentioned is adding pre-colored character sets in the TGA files. You have as well discarded this idea as it seemed too blunt, but since there is a lack of more elegant ideas, perhaps it still can be considered

EDIT: However, it does not go only for color. It is also applied for any "Font Formatting" thing. Such as resizing for main menu, for instance:



Is there a limit to how many characters can be added to the TGA files?

Daw said:

Well one idea you mentioned is adding pre-colored character sets in the TGA files. You have as well discarded this idea as it seemed too blunt, but since there is a lack of more elegant ideas, perhaps it still can be considered

Click to expand...

There are two problems with that approach. One is getting a giant GameFont due to having multiple instances of each character. The other is to figure out which of the versions to pick at runtime.

Daw said:

Is there a limit to how many characters can be added to the TGA files?

Click to expand...

286 on city billboards. No known limit for anywhere else until we run out of addresses, which is a limit around at least 24k characters. Non-white and non-standard sized characters aren't used on billboards meaning we will not really have an issue here.


With all the problems encountered so far, I wonder if we can fit the characters inside the first 256 addresses. This will presumably solve the problem. If we start by replacing characters unlikely to be used like †, ‡ and ¥, then we can investigate how many more we will need. Surprisingly we have •, which is a different bullet char from [BULLET_CHAR]. We have a number of different quotation marks and such and they all add up to quite a number of characters.

It doesn't look like it will be enough. We can then consider which languages we will stop supporting. Candidate for dropping would be à and Š.

I prefer not to drop support for anything and figure out a solution, which will not have to modify GameFont. The real question is how much is handled by the DLL and how much is handled by the exe. Also if tasks in the exe can be rewritten in DLL. This requires some investigation in the DLL/debugger.

Hey gang, not much noise has been coming from this forum in a while. I am still and always will be around. I have several things that are taking my attention at the moment but I am still play testing the development branch when I have the chance. When I think it is mostly stable I want to post a test release so others can check it out, however you can always check out the latest by visiting the sourceforge links here...

http://sourceforge.net/p/colonizationmodcollection/Medieval_Conquest/ci/master/tree/