Most of the text in the game is displayed using a TrueType font file, normally sylfaen.ttf. It is found in the vanilla civ's Resource/Fonts folder (there is also a copy in Assets/res/Fonts next to the gamefonx*.tga files, but I think it uses the other one). Mods that use a different font (like Final Frontier) provide a different font file and change the theme files to use the different font. Based on the font properties extension (from MS's PowerTools) sylfaen.ttf does not have Hebrew characters (Greek, Cyrillic, and a couple of others, but no Hebrew). So if that is correct, you'll have to use a different font file and change the places in the theme files that mention it (I think the only references are the definitions in Civ4Theme_Common.thm). Civ4 also comes with arial.ttf in the same folder and that one does include the Hebrew character set, if you want to use a font that ships with the game.
On the other hand, the GameFont*.tga files are, I think, used for the city billboards. The thing that hovers in front of every city and tells you the name, population, what it is producing and such. I have no idea what you'd do about that. I expect the Finnish translation people would know since it must have come up for them too. (The Russian translation is essentially dead since some company produced an official licensed version in Russian. There might be someone from the Russian translation project still around, or there might not.) Possibly change it to use the characters in the "1255 Hebrew" codepage, or the "862 Hebrew" codepage. That would require much messing around with the evil GameFont*.tga files once you figure out what needs to be mapped to which character (I have no idea how the 2 different Hebrew codepages are arranged). But that would probably be difficult since the current symbols in there are mostly the ISO-Latin-1 or Windows-1252 character set, but have some differences, and all control characters are skipped and it starts at character 33 (not 32, which is the space character). Since it is not a direct mapping but a modified version (for example: there is normally a backslash, "\" between the square brackets, "[" and "]", but the GameFont files have the multiplication symbol, "×" instead and later where the multiplication symbol is supposed to be it is skipped - there is no empty space or different character, it is just completely skipped) I have no idea how you'd substitute a different character set. I'm thinking the mapping is probably hardcoded into the .exe, although strings with the mapped characters in order do
appear in the createTestFontString function in CvGameTextMgr.cpp that doe snot sound like it is defining the mapping.
Hm. Searching files for font related stuff, I came across an interesting note in CIV4ArtDefines_Misc.xml:
So it may be possible to change this to using the same TrueType font everything else uses by changing fSCale from 1.0 to -1.0 (and then adjusting the value to get a good size). In that case, all that stuff about the weird mapping of the text in the GameFonts files is irrelevant.
.
. -> you might maybe want to ask some of the russian guys around (e.g. Walter Hawkwood) how their gamefonts look. If they have cyrilic signs in them too, then you have a problem. If not, then Civ has to use another system 
