Current and Future Developments

[Nightinggale]

I finished the debug screen
I figure I better post a screenshot to make the previous post easier to understand.

 

[Kailric]

Wow, interesting. That took some investigating, muy bueno! I have no objections.

 

[Lib.Spi't]

Wow this is going to be so helpful in fixing up the icon displacement in WHM... nice work!

 

[raystuttgart]

Wow this is going to be so helpful in fixing up the icon displacement in WHM... nice work!

You guys do know that there are already built in debug functionalities for testing ?
(Stuff like Gamefonts can be tested as well.)

Spoiler :

It wouldn't help to find out the number references (that Nightinggale was looking for) to my knowledge.
But those debug functionalities have been quite useful for me in the past nonetheless.

 

[Daw]

Having notepad++ installed (thanks to Nightinggale for the link provided) I discovered that it matters from what encoding you convert to UTF-8.

The Advisor.xml was in ANSI and Russian strings became proper when converted into UTF-8.

On the other hand, Chalremagne Text_BtS_Objects.xml was in ISO-8859-1, and when converted into UTF-8 showed something like "Ïàïñêàÿ îáëàñòü" instead of Cyrillics - and those symbols seem to be exactly what can be seen on Ray's screenshot.

In both files Russian text was entered with WordPad and looked OK then (and still look OK there), but looked unreadable in Notepad++ with default character set before conversion to UTF-8.

And Cyrillic part of 'Chalremagne' is unreadable in any encoding offered by Notepad++. But properly looking new Cyrillics strings can be entered there in Notepad++.

EDIT:

I managed to make 'Chalremagne' readable by opening is with ordinary Notepad, saving, and - voila - it looks good in Notepad++.
I can also make it unreadable again by opening it with WordPad, saving, and - voila - Notepad++ shows weird symbols.
And then readable again just by saving in ordinary Notepad... man, I could play with that the whole evening!

 

[Nightinggale]

Having notepad++ installed (thanks to Nightinggale for the link provided) I discovered that it matters from what encoding you convert to NTF-8.

It has to be UTF-8 without BOM. I learned that the hard way today

On the other hand, Chalremagne Text_BtS_Objects.xml was in ISO-8859-1, and when converted into NTF-8 showed something like "Ïàïñêàÿ îáëàñòü" instead of Cyrillics - and those symbols seem to be exactly what can be seen on Ray's screenshot.

Known issue. The issue is that each character is actually just a number. The game and the XML file then have to agree on what numbers means what. Right now there is a great mismatch (much greater than meets the eye) because vanilla is designed to only work with ISO-8859-1. I have added more characters, but they aren't on a default location. I have written a converter to make them appear correctly. It sort of work, but for some reason whenever it converts something, it adds a space and the text becomes.... odd looking and very wide. From the looks of it, I will have a working solution within days where the XML files can be regular UTF-8 (without BOM) and and the game will display the characters correctly. We have around 100 characters like û and ë, which are also hard to reach. I plan to convert those on load as well once Russian is fully working.

 

[Daw]

It has to be UTF-8 without BOM. I learned that the hard way today

Noted.

Speaking about adding new characters and having characters like û and ë. I don't know if it might help or be irrelevant or would only mess things up, but:

1. although Cyrillic formally has 33 characters, which yields 66 because of lowercase/capitals, only 32 are used in common practice ('Ee' is commonly used instead of 'Ёё' as well)
2. Three characters ('Ьь', 'Ъъ' and 'Ыы') have no use of capitals but in ALL CAPS STYLE. This is because no words start with these symbols.
3. Many symbols look exactly the same (though have different meaning) in both Latin and Cyrillic ('Ee', 'Oo', 'Aa', 'Cc', 'Pp', capitals 'T', 'M', 'B', 'K', 'H', lowercase 'y')

This can reduce the number of images you need to show Cyrillic. Though I am sure you know a better way of doing it.

 

[Nightinggale]

I copy pasted the characters from the Russian GameFont. That provided us with 64 characters ('Ёё is missing). We have the characters we need. The problem is the conversion, which turned out to be much harder to do than expected. I solved the space issue, but somehow it maps the characters incorrectly. The question is where it goes wrong, which is why I started walking through the entire alphabet. I wanted to copy paste it, but I can't find anywhere where it is written in plain text

I couldn't copy paste the size of the characters and I had to split them manually. I nearly turned Ы into two characters

 

[Daw]

I can't find anywhere where it is written in plain text

The alphabet is one of the few things I know really well How do I make a plain text for you to copy?

I think I did it. Please try.

 

[Nightinggale]

The alphabet is one of the few things I know really well How do I make a plain text for you to copy?

Preferably all of it in both upper and lower case.

 

[Nightinggale]

It works. I have written normal UTF-8 in the XML file and it appears like all characters appear correctly ingame. Now I need to add the ID of А to XML (in it changes if more icons are added later) and add the big font version as well. Right now only small font appears ingame. Cleaning up and reviewing the new code is likely also a good idea.

This mean it's still not there, but it's really close and we know that it will work. Surprisingly it appear to be better than the Russian version as this runs on an unmodded exe and it didn't break compatibility with other languages.

We might have to look into the graphical work of the new characters through. Some lowercase characters appear to be bigger than the uppercase version. We can always redraw the characters later if we like. It seems like a minor detail compared to the task of gaining the characters in the first place.


Oh yeah, the alphabet was helpful. I could quickly add lots of characters an ascending order and spot where the offset bug appeared. Knowing precisely where and by how much the ID was wrong, it became trivial to fix.

 

[Nightinggale]

I have pushed the Russian language support to SourceForge. The branch Russian_Translation is updated and there is a new branch called GameFont. The GameFont branch will be developed further to gain support for more characters. Ideally we will no longer have to use escapes to add any characters.

@Daw:
I plan that you use Russian_Translation to add your translated files to. You use GameFont for the two submodules.

I added Russian as a language in XML. This mean it can be picked in options ingame. I see no reason to hide it since there should technically not be any issues left.

I figured out why I had so many problems getting the Russian characters to behave as intended. The Russian GameFont files lacked uppercase Ч and I added it to our DLL. Rather than hacking the DLL to ignore it, I added something, which looks like Ч to GameFont. GameFontEditor view the Russian addition as regular icons. This mean it is easy to replace one or more if somebody draws/finds something better. It's a bit more tricky to alter size of a character, but that's also doable.

 

[Daw]

Incredible! It seems like you have just rendered all those 'localizators' jobless.

However, there's something I do wrong or... I can't commit.

 

[Nightinggale]

However, there's something I do wrong or... I can't commit.

You are using a read only URL. Under branches (bottom left), there is a line saying Origin (5): (some URL). Select that line, right click and select "Edit URL".

You get the correct URL here: https://sourceforge.net/p/colonizationmodcollection/Medieval_Conquest/ci/Russian_Translation/tree/
You need to be logged in to get it as it will put your username into the URL. You need either RW or HTTP access (RW is technical better, but less firewall friendly).

 

[Nightinggale]

I gave Fullerene git write access and added him to the list on the previous page. Maybe we should put that list somewhere else where it is easier to find it.

 

[Nightinggale]

I have a problem. I can't remember what I'm supposed to do now

1. Add Russian support
2. Make new savegame code design
3. Make a trade screen class
4. Mystery feature

I was working on #4, then the need for #3 appeared and so on. Now after I have working solutions for the first 3 I can go back to continue my original work, but what was it

If nobody can remember I will have to leave it half finished and start a new task. It's not like I can't figure out something to do, but I would like to figure out what I left half finished somewhere and where somewhere is.

 

[Kailric]

I have a problem. I can't remember what I'm supposed to do now

1. Add Russian support
2. Make new savegame code design
3. Make a trade screen class
4. Mystery feature

I was working on #4, then the need for #3 appeared and so on. Now after I have working solutions for the first 3 I can go back to continue my original work, but what was it

If nobody can remember I will have to leave it half finished and start a new task. It's not like I can't figure out something to do, but I would like to figure out what I left half finished somewhere and where somewhere is.

Perhaps one of the things in this post of yours?
http://forums.civfanatics.com/showpost.php?p=13266300&postcount=117

 

[orlanth]

I think u were talking about the economy and price effects from supply/demand most recently. But only you know what goes on in that mysterious Nightingale brain!

 

[Kailric]

I think u were talking about the economy and price effects from supply/demand most recently. But only you know what goes on in that mysterious Nightingale brain!

Yeah, you had some ideas you was going to work into the new trade screens and economy.

 

[Nightinggale]

But only you know what goes on in that mysterious Nightingale brain!

Apparently I don't know that either

If we use just a little bit of logic, #4 task can't be anything related to trade screens as it was something I worked on before the trade screen existed.

Whatever it was, it's gone. I think we should do as I usually do with a failure, which is to figure out a way to try to prevent the mistake from repeating itself. We should likely add all unfinished tasks to the ticket system to keep track of what we do. That might actually be beneficial in more ways as we can look up what other people work on or intend to work on. As the number of modders increase, organizing work could be beneficial.

A bonus for using the ticket system is that we can set a version we want the task to be finished in. We make a pseudo task called develop, which is tasked closed with the result in pushed to develop. Once we know what the next release will be called, we change milestone for all develop tasks to that number and we can then look at all closed tasks for that number to get an overview of what to put in the changelog

I usually do with a failure, which is to figure out a way to try to prevent the mistake from repeating itself.

Writing that sentence made me remember what it was. I want to write a script, which runs automatically when git commits. It should do two things:

1. Indent modified XML files
2. Return error to git if it indents anything

As precommit hooks can't modify what you commit, step 2 is needed. It will make the commit fail, but since step 1 is executed you can commit the same again. Log entry and everything will be remembered meaning committing again should be easy.

You might want to verify that the checksum for xml schema files are still valid though.