[UTILITY] XML text cleaner

[j_mie6]

The XML text cleaner! (V3)

Download

The XML text cleaner is a Python built utility that searches through a mods xml files and makes sure that all Descriptions etc have text tags in the Assets/XML/Text directory

for example say I had made a new civilization like this:

		<CivilizationInfo>
			<Type>CIVILIZATION_VENICE</Type>
			<Description>The Venetian Empire</Description>
			<ShortDescription>Venice</ShortDescription>
			<Adjective>Venetian</Adjective>
			<Civilopedia>Venice is a city in northern Italy bla bla bla...</Civilopedia>

while running the cleaner would convert it to this:

		<CivilizationInfo>
			<Type>CIVILIZATION_VENICE</Type>
			<Description>TXT_KEY_VENICE_DESC</Description>
			<ShortDescription>TXT_KEY_VENICE_SHORT_DESC</ShortDescription>
			<Adjective>TXT_KEY_VENICE_ADJECTIVE</Adjective>
			<Civilopedia>TXT_KEY_VENICE_PEDIA</Civilopedia>

with some text tags of

	<TEXT>
		<Tag>TXT_KEY_VENICE_DESC</Tag>
		<English>The Venetian Empire</English>
		<French>The Venetian Empire</French>
		<German>The Venetian Empire</German>
		<Italian>The Venetian Empire</Italian>
		<Spanish>The Venetian Empire</Spanish>
	</TEXT>
	<TEXT>
		<Tag>TXT_KEY_VENICE_SHORT_DESC</Tag>
		<English>Venice</English>
		<French>Venice</French>
		<German>Venice</German>
		<Italian>Venice</Italian>
		<Spanish>Venice</Spanish>
	</TEXT>

etc...

Installation and intructions for use!

usage is very easy, first install python 2.7 from www.python.org (this is required for either version).

open the module with IDLE, then edit the options to fit your mod's xml path/documents (see below) and press F5 to run. (this is is using V1 or if the variable bGraphics is set to False in V2+)

if using V2/V3 simply doubleclick the XML text cleaner.py file and it will open (like image 2 below) then press the options button to enter mod path etc and then press run Cleaner! In GraphicalInterface.py there are some options that allow you to change the colour, font type, font colour and font size of the main box!

there are many options that the user can use.

##Options##

#Enter mod xml location (Files are automatically backed up to the path specified in backupPath
xmlPath = "C:/Program Files/2K Games/Firaxis Games/Sid Meier's Civilization 4 Complete/Beyond the Sword/Mods/[your mod name]/Assets/XML"

#Backed up files will be places in this path in a folder called XMLBackup, MAKE SURE THIS POINTS TO A VALID FILE PATH!
backupPath = "C:/Users/User/Documents"

#Here add files that should ignore the checks (for example EffectInfos has many wrong references, but it is on purpose!)
lIgnores = ["CIV4EffectInfos.xml", ]

#The name of the new text xml (if one doesn't exist this will be the name of the new one!)
textFilename = "CIV4CustomText.xml"

#Language Options (set to True to create language node in xml!)
bFrench = True
bGerman = True
bItalian = True
bSpanish = True

#Search Options (set to True to enable searching for this node!)
bDescription = True
bPedia = True
bStrategy = True
bHelp = True
bShortDescription = True
bAdjective = True
bCityNames = True
bQuotes = True

if I was to set all the Language options to False any text tags created would be in the form of:

	<TEXT>
		<Tag>TXT_KEY_SOME_TEXT</Tag>
		<English>some text</English>
	</TEXT>

and turning search options to false ignores that type of node (e.g setting bPedia to False would mean that the program wouldn't bother to check the Civilopedia tags for incompete text.)

When entering paths make sure you use either / or \\ eg

C:/Users/User/Documents or C:\\Users\\User\\Documents

When the program starts it will backup the mods files in the path specified in options. When the program runs again it will delete the previous backup and create new ones. Make sure that when you are finished you check that there are no xml errors before running the program again!

Future features include an xml file for options instead of in the .py file (for non graphical setup) and italian and spanish translation fix, also more intelligent text tag management will be implemented to speed up translation time.

Post any bugs, questions or requests here!

Enjoy!

[dacubz145]

Very nice work, thanks for taking up my request

I am going to update my mod in a day or two then im going to run this, since my backup will be uploaded here

[j_mie6]

good luck!

[j_mie6]

oh and truthfully, this project was a lot of fun and a good learning curve Well spent 12 straight hours of programming

[The_J]

Congrats .
Sounds like an useful thing for everyone who started with the lazy way .

[j_mie6]

Ok I have finished work on the graphical interface

to run with the graphical interface you just need to double click on the XML text cleaner.py file and it is ready to go. Will upload later but going to put on a teaser picture

Notes
Will work on autofill of the options box so that user doesn't have to enter the path every single time they use it

[dacubz145]

Very interested to see the interface

[j_mie6]

Going to upload it by the end of the day (and update the original post, as you can use one of the options in the .py file to turn off graphics (means you don't have to enter in the paths every time, but I hope to fix that soon with my good old fashioned pickle jar (don't ask )))

but the pictures are up now

also you can change the bg colour, font colour, font size and font type of that main box in the GraphicalInterface.py file

[dacubz145]

Looks really good, this is definatly coming around real nice!

[j_mie6]

Version 2 is up!

[j_mie6]

Ok I have a method of translation and I am workin gon getting it up and running as soon as possible

[dacubz145]

that is nice! how did you figure it out?

[j_mie6]

myGengo API current having problems with italian and spanish though. The code is telling me they don't work

[PsiCorps]

This Utility program looks like it will be very useful for me however, you said

first install python 2.7 from www.python.org (this is required for either V1 or V2)

Specifically which variation of 2.7? I use Win XP and am not as computer literate as most.

•Gzipped source tar ball (2.7.3) (sig)
•Bzipped source tar ball (2.7.3) (sig)
•XZ source tar ball (2.7.3) (sig)
•Windows x86 MSI Installer (2.7.3) (sig)
•Windows x86 MSI program database (2.7.3) (sig)
•Windows X86-64 MSI Installer (2.7.3) [1] (sig)
•Windows X86-64 program database (2.7.3) [1] (sig)

Please help me out.

[j_mie6]

Really I should have put python 2.7.? because you just take the latest version ie 2.7.3

then it is up to you which file you choose however I would say go for the top one on the page:

•Python 2.7.3 Windows Installer (Windows binary -- does not include source)
option as then it will install easily

hope that helped!

[PsiCorps]

Originally Posted by j_mie6

Really I should have put python 2.7.? because you just take the latest version ie 2.7.3

then it is up to you which file you choose however I would say go for the top one on the page:

•Python 2.7.3 Windows Installer (Windows binary -- does not include source)
option as then it will install easily

hope that helped!

Perfect, just what I needed.

Thank you.

[dacubz145]

any update on the translator?

[j_mie6]

haven't got around to that yet, I still want my e-mail reply from the developer on why italian and spanish are both not working

[j_mie6]

right the translator is finished! when the cleaner makes a new text tag it auto translates but there is another button on the GUI that allows you to further translate all your mods existing text tags aswell. The danger is that if you have the typical:

<TEXT>
           <Tag>TXT_KEY_ETC</Tag>
           <English></English>
           <French></French>
           <German></German>
           <Italian></Italian>
           <Spanish></Spanish>
</TEXT>

and only select French translation ALL your text tags will become:

<TEXT>
           <Tag>TXT_KEY_ETC</Tag>
           <English></English>
           <French></French>
</TEXT>

so make sure you select all the languages you want!!! this will be uploaded soon.

if the translation fails the tags will be the english equivilents. For the moment the Italian and Spanish translations are not working (something to do with MyGengo) and are typical english.

How the translator works (for those that are interested):

Spoiler:

when the Translate button is clicked/a new tag is made by the program the program then interfaces with the Translator class. In the case of a new tag the translators static get*****() methods are used to collect language *****, it basically sends a request to the MyGengo API and waits for response. With the GUI's Button a new instance of the Translator class is created and passes the path and other options, the __init__ then removes unneeded options. next, doTranslate is called and gets the files from the Text folder, the Text tags are "Replaced" to sort out problems with missing tags (during translation itself) the replaceTags method actually calls the classmethod disectNode() which destroys a TEXT node and throws back its mangled body parts. the replace tags method preceeds to perform life saving operations on the data to reform the tags with the required language tags. now that its all fine doTranslate swaps the language info with translated string and saves the xml (after reformating). finished . pretty nice setup I think.

[dacubz145]

Can't wait till you upload it