Game Font Editor development thread

[Asaf]

Yes, you've read it right. An editor for the gamefont files.
Apparently they're evil

This editor is meant to free modders from the need to edit and maintain the grid that surrounds the actual icons, manage the 'used icon' markers, and generally prevent any possible corruption of the font files.
I also intend for it to automatically fix most of the already corrupted font files.

I've started working on this editor, but it is not yet usable. Since my experience with these files consists of adding 1 icon to each of the files, and I'm not completely sure how this editor should behave, I thought of getting feed back as early as possible.

What I have so far:


I have a program which loads the selected game font file, and analyzes it to the level of knowing where each slot is on the image, and whether it is used or not (has the cyan marker).
It ignores problems in the grid's alpha channel and will be able to fix them.

As you can see, the current behavior is such that when hovering over an icon it is displayed on the side, along with some information on it (whether it is used or not, size and position).

In case you were wondering, it works on both gamefont and gamefont_75 (and analyzes them correctly - including the funky line where there's one icon which is smaller than the others), as well as on the BUG gamefont file.

What I plan to do:

Must-have features
- First and most importantly, to allow editing of specific slots. That means to enable exporting specific icons to image files, and allow importing of image files into specific slots in the fonts image.
- Allow enabling and disabling of certain slots with a click of a button (will be done by adding/removing the cyan marker).

Important features
- Automatic fix of the grid's alpha channel.
- Identifying the slots even if some of the grid is corrupted.

Optional features
- Allow opening of an image editor for a specific slot (in a similar manner to DXTBmp, only for a specific slot) instead of saving to a file and loading to a file. Maybe split to alpha/non alpha.
- Adding automatic meta data: for example - read the religion/corporation XML files and display for each icon which religion/corporation it is used for.
- Automatically create an alpha channel for an icon from the image (using a "background color").

There are more features that crossed my mind, but this is what I can remember now.

What I need from you:

Quite a few things, actually.
Since many of you worked with the font files much longer than I have, I'd appreciate any advice I can get on this.

Specifically, these are the things I though of:

Questions
These are questions I have with regards to how the gamefonts work. I intend to start experimenting with these, but I thought that maybe someone already has some of the answers.

1. Do the markers have to be in a specific position? It is mostly consistent within the same file (GameFont.tga - 17 pixels below the seperator line, GameFont_75.tga - 14 pixels), but within the 75, two of the commerce types have their markers moved by a pixel.
2. Can slots be skipped? For example - can the first 3 slots in a specific row be used, the 4th unused, and then the 5th used again? I'm talking about the markers, and not the code that assigns symbol id's (either DLL or Python in BUG's case).
3. Can you add additional rows to the font file beyond those already surrounded with grid? Same questions for more slots in a specific row.
4. Is the iPadAmount (I think that's what it's called) in CvGameTextMgr::assignFontIds constant, or is it actually calculated according to the font file?
5. When is each of the font files used?
6. I'm assuming (because of BUG) that the two font files don't have to be exactly the same slots-wise?

Other requests
Please feel free to throw in ideas for other features for this editor, or changes that you think are needed for my ideas.
If you have specific issues that you've come across when working with these files (and that are not already explained in one of the tutorials), I also want to know.

I'd also appreciate if you can attach your corrupted game font files so I can take a look at them and see if I can add automatic fixes for them.

This is just a development thread. Once I have a utility that can actually be used I'll post it in a new thread, and the discussion will go there.

Thank you for your help.

 

[Baldyr]

OMG, yet another huge life-saver!

 

[The_J]

Oh my god, that will need a ton of work .
And will be awesome at the end .

Q 5: The normal .tga is used ingame, the _75 is used in the civilopedia. Not sure where it is used else.
Q 6: Should AFAIK be independent.

 

[God-Emperor]

Yes, you've read it right. An editor for the gamefont files.
Apparently they're evil

They are eeeeeviiilllll!

I had considered doing something like this myself a time or two...

Anyhow, here is a piece of info pertaining to Q1: I'm pretty sure the location of the Cyan dot marks the baseline for the symbol's vertical position (the position that the letters sit on in a line of text, excluding the descenders on letters like p and q). If you move the dot up, the glyph should be drawn lower by the same amount (assuming I'm correct). If the dot is missing then the baseline is at the bottom of the glyph. You can get a good idea of this if you make the normally white letters on white background up in the first few rows visible (for example, make the letters black instead of white - I did this by extracting only the alpha channel from the image then inverting the intensity then applying an alpha channel to it so that I have a layer with black letters and symbols up there and alpha-mask shaped black blobs for everything else, with this in a layer floating over a version of the original with the alpha channel removed, to show the borders and dots, you can see where the dots are relative to the letters).

Also, the _75 version must be used for the text in the city bars since the BUG icons show up there and are only present in the _75 version. And on the scoreboard too.

 

[Voyhkah]

WARNING! If you pull this off, I will have to start a religion worshipping you.

GameFont has been giving me hell, recently. NikNaks is helping me with it, but it would be so much easier if there was a simple editor.

KEEP IT UP!

 

[need my speed]

Yes, I've given up on adding resources because of this vile thing. You're already pretty awesome, but if you can actually make this work... Wow...

 

[Asaf]

OMG, yet another huge life-saver!

Wait until I actually publish something...

Q 5: The normal .tga is used ingame, the _75 is used in the civilopedia. Not sure where it is used else.
Q 6: Should AFAIK be independent.

Also, the _75 version must be used for the text in the city bars since the BUG icons show up there and are only present in the _75 version. And on the scoreboard too.

So I won't add special treatment to each of them, and no need to synchronize between the two.

Anyhow, here is a piece of info pertaining to Q1: I'm pretty sure the location of the Cyan dot marks the baseline for the symbol's vertical position (the position that the letters sit on in a line of text, excluding the descenders on letters like p and q). If you move the dot up, the glyph should be drawn lower by the same amount (assuming I'm correct). If the dot is missing then the baseline is at the bottom of the glyph.

Oh! This is new to me. So it's not an issue of enabling/disabling... interesting. I'll experiment with the positioning.
So basically the marker is not actually needed for the icon to be used?

EDIT: If so, then that answers question 2 as well...

You can get a good idea of this if you make the normally white letters on white background up in the first few rows visible (for example, make the letters black instead of white - I did this by extracting only the alpha channel from the image then inverting the intensity then applying an alpha channel to it so that I have a layer with black letters and symbols up there and alpha-mask shaped black blobs for everything else, with this in a layer floating over a version of the original with the alpha channel removed, to show the borders and dots, you can see where the dots are relative to the letters).

Yes, I've already done that.
I tried at first displaying the game fonts image with applying the alpha channel, which means that the grid was invisible and the actual font was displayed properly on the darker background.
But then I thought it would be better to display the grid for easier icon selection, since no one edits the actual font anyway.

Which do you think is better? I'm thinking of completely trimming the font section for the display.

Yes, I've given up on adding resources because of this vile thing. You're already pretty awesome, but if you can actually make this work... Wow...

WARNING! If you pull this off, I will have to start a religion worshipping you.

 

[Dancing Hoskuld]

Optional features
- Adding automatic meta data: for example - read the religion/corporation XML files and display for each icon which religion/corporation it is used for.

As long as it does not complain because that icon is used by a religion/corporation which is defined in a mod in the Modules folder.

Many graphics modders of resources provide a file containing the icon for the fonts file so being able to find and pick it for pasting into a slot would be good.

 

[Asaf]

As long as it does not complain because that icon is used by a religion/corporation which is defined in a mod in the Modules folder.

Many graphics modders of resources provide a file containing the icon for the fonts file so being able to find and pick it for pasting into a slot would be good.

I've never worked with a Moduled mod, so I'm not sure what the structure there looks like...

Do you mean read the icon path from the XML?

 

[Dancing Hoskuld]

I've never worked with a Moduled mod, so I'm not sure what the structure there looks like...

Do you mean read the icon path from the XML?

I was not suggesting you did, just that it not fail if it can't find xml for the religion/corporation in the standard place.

The second point I was trying to make is that sometimes a graphics modder produces a resource providing all the art in the folder including a file which contains the bit to put into the fonts file(s). It would be nice if this program could get the graphic from that file and place it into the fonts file. For example SaibotLieh's rabbits resource includes such a file.

 

[NikNaks]

I've never worked with a Moduled mod, so I'm not sure what the structure there looks like...

Do you mean read the icon path from the XML?

No, I think he's just pointing out that for that to work, you'd need to run a sweep of the modules folder to check whether there are any additional religions or corporations added. For what it's worth, the game doesn't know which religion is which, so for modules, we normally have to put them into a pack so we can control that. I think the WoC added a tag for that. See below.

Oh! This is new to me. So it's not an issue of enabling/disabling... interesting. I'll experiment with the positioning.
So basically the marker is not actually needed for the icon to be used?

EDIT: If so, then that answers question 2 as well...

I didn't know that was the case! I'll have to test that out.

Which do you think is better? I'm thinking of completely trimming the font section for the display.

I think for a release, that'd be easiest, yes, but I have seen one custom font set used, so maybe down the line, if you're mad enough to try, there could be a custom font dialog.

Questions
These are questions I have with regards to how the gamefonts work. I intend to start experimenting with these, but I thought that maybe someone already has some of the answers.

1. Do the markers have to be in a specific position? It is mostly consistent within the same file (GameFont.tga - 17 pixels below the seperator line, GameFont_75.tga - 14 pixels), but within the 75, two of the commerce types have their markers moved by a pixel.

2. Can slots be skipped? For example - can the first 3 slots in a specific row be used, the 4th unused, and then the 5th used again? I'm talking about the markers, and not the code that assigns symbol id's (either DLL or Python in BUG's case).

6. I'm assuming (because of BUG) that the two font files don't have to be exactly the same slots-wise?

I think these have been answered; for the others:

3. Can you add additional rows to the font file beyond those already surrounded with grid? Same questions for more slots in a specific row.

I don't think this has ever been tried, but it'd be easy to experiment. You'd just need to edit BUG to call something from a row further down. I'd guess it's possible, though, seeing as I know that you can extend the file sideways in size, and add many more blocks to each row; we did that for WoC, so it's definitely doable, but whether or not MrGenie added in some extra functionality into the DLL to actually parse it properly, I'm not sure. He certainly messed with it (he added a locator integer tag so that modules picked up the right set of icons) so I'm not sure if it'd work in a simple mod. I hope that makes sense.

4. Is the iPadAmount (I think that's what it's called) in CvGameTextMgr::assignFontIds constant, or is it actually calculated according to the font file?

Can you clarify where this is used? That might help to work out whether it would be or not.

5. When is each of the font files used?

I know this has been answered in part, but to clarify, the main file is used in the main interface, while the 75 is used on the scoreboard and pedia, and, at least in BUG, also some advisor screens.

If you have any more questions, please ask!

 

[Asaf]

UPDATE: I've finished implementing the following:
- importing/exporting images to/from specific slots
- saving the modified image
- moving/adding/removing the marker.

I hope to release a first (beta?) version of the editor in the next few days.

Regarding the marker - it seems that it is indeed not needed for using the icon, but I didn't see any effect in the game when I moved it up/down.
Maybe it only affects the font itself (which I didn't edit)?

@Dancing Hoskuld - thanks for the rabbits link. I used it to test importing, which works
And now I understand what you meant with the modules... (see below)

@NikNaks - Thanks
iPadAmount is used in the DLL code as a parameter to the function which assigns the symbols id's to the different game infos. This parameter is sent by the exe. It is used for skipping to the next row.

Looking at WoC's gamefont file it seems that adding rows/columns is indeed possible. I'd have to try to see if this affects iPadAmount. (Where can I find the actual TGA file, BTW?)

I don't think the editor will have a problem handling these files. The problem will be with auto-detection of what an icon is used for (using the XML). I think I'll add this as an optional feature which will only work for non-WoC mods.

@Everyone -

* What are the most common reasons for a gamefont file to get corrupted? I know about alpha channel messing of the grid. Do the other channels (R/G/B) also get corrupted?
What else? (Any file samples would be also good)

* Is there a need to resize a slot (width only)?

 

[The_J]

You can mess the gamefonts up by:
- resizing them
- Putting the image itself to much to the write/left. If an icon goes over the grid horizontally, the game will treat both slots as one, which leads to a shift in the grid. Not sure what happens vertically .

And i don't think the gamefonts will tolerate resizing a slot, and i would not even try that.

 

[NikNaks]

@NikNaks - Thanks
iPadAmount is used in the DLL code as a parameter to the function which assigns the symbols id's to the different game infos. This parameter is sent by the exe. It is used for skipping to the next row.

I imagine, then, it's probably done dynamically, considering how it reacts to extra rows and columns.

Looking at WoC's gamefont file it seems that adding rows/columns is indeed possible. I'd have to try to see if this affects iPadAmount. (Where can I find the actual TGA file, BTW?)

I'll try to dig up the old SVN libraries on Sourceforge.

I don't think the editor will have a problem handling these files. The problem will be with auto-detection of what an icon is used for (using the XML). I think I'll add this as an optional feature which will only work for non-WoC mods.

Excellent!

@Everyone -

* What are the most common reasons for a gamefont file to get corrupted? I know about alpha channel messing of the grid. Do the other channels (R/G/B) also get corrupted?
What else? (Any file samples would be also good)

Yeah, the game also crashes when the pink hue is wrong/missing from the grid entirely/broken (i.e. when there's a gap in one of the vertical lines between icons). If you insert an icon and don't take care with the RGB channel, you can either lose the grid entirely or break a part of it, which can bork the font.

* Is there a need to resize a slot (width only)?

Considering that the text has differing widths, it might be interesting to look into it. I don't recall a time in BUG when a wide icon was more useful, but a thinner one might occasionally be handy. Experimentation required again.

 

[God-Emperor]

You can use symbols of different widths than the existing box. Just make sure to shift the later stuff on the same line appropriately and maintain proper borders.

For new version of FFP with BUG incorporated I had to merge the FF font file with the BUG font file. The FF font files replace the power and great people symbols to use them for different things (the delta and omega symbols used for the 2 later ship generations - in some future version of FFP that brings back great people this will have to be changed by adding another symbol to get the GP symbol back). The delta and omega symbols were narrower than the symbols they replaced (well, one of them was - I think the other was about the same), but I didn't want to expend the effort to narrow them down in the _75 file (by shifting everything else around) so I made them use the entire box (just added transparent pixels to the width) so in the regular file they are comparatively narrower than they are in the _75 file. It would probably look slightly better in a few places if the symbols in the _75 file were made narrower to match, l but it isn't really noticeable unless you check. So we finally get to the point of this paragraph: not only can you use different symbol widths, they don't have to be proportionate across the two files.

 

[Asaf]

Thanks for the reply's.
I'll think about adding slot resizing later, but in the mean time I've implemented some file corruption identification and auto-fix.
I've taken the liberty of trying to fix Voyhkah's corrupted files.

@Voyhkah - Can you please test these to see if they work in your game?
(I'll post it in your thread as well).

 

[Xyth]

There are a couple 'quirks' I've encountered while working with gamefont files. I mention them in case they are of use to you while developing this tool.

1) This might be a problem with BUG or something I've done wrong but I've noticed that if enough new religions or corporations are added (without adding any new rows), the row that starts with the Happiness/Unhappiness/Health/etc 'shifts' differently for different parts of the UI.

Specifically the icons above the city nameplate that indicate unrest/unhealthiness/defense bonus/power/etc disappear whilst all other occurrences of these symbols (pedia, etc) work fine. I solved this in my mod (and The Capo's) by having two rows of those symbols in Gamefont.tga (but not needed for Gamefont_75.tga).

This problem also prevents adding a new row of icons anywhere as that causes the symbols above the city nameplate to be replaced by resource icons. Though, is there anywhere in BTS or BUG that uses the resource symbols from GameFont.tga? I've been looking but so far only seen the smaller sized ones and resized button graphics.


2) If you have filled the second row of resource icons except for the last space and have the corresponding number of entries in CIV4BonusInfos.xml the gamefont files will 'break' - missing symbols, wrongly sized text, etc. However if you remove or add a resource (in both the tga and xml) it will work fine.

 

[Voyhkah]

Thanks for the reply's.
I'll think about adding slot resizing later, but in the mean time I've implemented some file corruption identification and auto-fix.
I've taken the liberty of trying to fix Voyhkah's corrupted files.

@Voyhkah - Can you please test these to see if they work in your game?
(I'll post it in your thread as well).

I really appriciate you doing that, but:

1. NikNaks is already helping me make the icons better;

2. I don't have my computer right now. (I'm using a borrowed one, without Civ, much less my modding stuff.)

 

[Asaf]

@Xyth - Thanks. I'll look into these.

I really appriciate you doing that, but:

1. NikNaks is already helping me make the icons better;

2. I don't have my computer right now. (I'm using a borrowed one, without Civ, much less my modding stuff.)

Only wanted to check if my tool's auto fix works fine
When you can, I'd appreciate it if you can try and let me know.

 

[NikNaks]

While I remember, the WoC fonts can be found here, along with the XML files that were added.