Advertisement
Civilization Fanatics' Center

Go Back   Civilization Fanatics' Forums > CIVILIZATION IV > Civ4 - Creation & Customization > Civ4 - Utility Programs

Notices

Civ4 - Utility Programs Post useful utility programs for Civ4.

Reply
 
Thread Tools
Old Apr 06, 2012, 03:52 PM   #1
j_mie6
Deity
 
j_mie6's Avatar
 
Join Date: Dec 2009
Location: Bristol (uni)/Swindon (home)
Posts: 2,944
Gender: Male
XML text cleaner

The XML text cleaner! (V3)



The XML text cleaner is a Python built utility that searches through a mods xml files and makes sure that all Descriptions etc have text tags in the Assets/XML/Text directory

for example say I had made a new civilization like this:

Code:
		<CivilizationInfo>
			<Type>CIVILIZATION_VENICE</Type>
			<Description>The Venetian Empire</Description>
			<ShortDescription>Venice</ShortDescription>
			<Adjective>Venetian</Adjective>
			<Civilopedia>Venice is a city in northern Italy bla bla bla...</Civilopedia>
while running the cleaner would convert it to this:

Code:
		<CivilizationInfo>
			<Type>CIVILIZATION_VENICE</Type>
			<Description>TXT_KEY_VENICE_DESC</Description>
			<ShortDescription>TXT_KEY_VENICE_SHORT_DESC</ShortDescription>
			<Adjective>TXT_KEY_VENICE_ADJECTIVE</Adjective>
			<Civilopedia>TXT_KEY_VENICE_PEDIA</Civilopedia>
with some text tags of

Code:
	<TEXT>
		<Tag>TXT_KEY_VENICE_DESC</Tag>
		<English>The Venetian Empire</English>
		<French>The Venetian Empire</French>
		<German>The Venetian Empire</German>
		<Italian>The Venetian Empire</Italian>
		<Spanish>The Venetian Empire</Spanish>
	</TEXT>
	<TEXT>
		<Tag>TXT_KEY_VENICE_SHORT_DESC</Tag>
		<English>Venice</English>
		<French>Venice</French>
		<German>Venice</German>
		<Italian>Venice</Italian>
		<Spanish>Venice</Spanish>
	</TEXT>
etc...

Installation and intructions for use!

usage is very easy, first install python 2.7 from www.python.org (this is required for either version).

open the module with IDLE, then edit the options to fit your mod's xml path/documents (see below) and press F5 to run. (this is is using V1 or if the variable bGraphics is set to False in V2+)

if using V2/V3 simply doubleclick the XML text cleaner.py file and it will open (like image 2 below) then press the options button to enter mod path etc and then press run Cleaner! In GraphicalInterface.py there are some options that allow you to change the colour, font type, font colour and font size of the main box!

there are many options that the user can use.
Code:
##Options##

#Enter mod xml location (Files are automatically backed up to the path specified in backupPath
xmlPath = "C:/Program Files/2K Games/Firaxis Games/Sid Meier's Civilization 4 Complete/Beyond the Sword/Mods/[your mod name]/Assets/XML"

#Backed up files will be places in this path in a folder called XMLBackup, MAKE SURE THIS POINTS TO A VALID FILE PATH!
backupPath = "C:/Users/User/Documents"

#Here add files that should ignore the checks (for example EffectInfos has many wrong references, but it is on purpose!)
lIgnores = ["CIV4EffectInfos.xml", ]

#The name of the new text xml (if one doesn't exist this will be the name of the new one!)
textFilename = "CIV4CustomText.xml"

#Language Options (set to True to create language node in xml!)
bFrench = True
bGerman = True
bItalian = True
bSpanish = True

#Search Options (set to True to enable searching for this node!)
bDescription = True
bPedia = True
bStrategy = True
bHelp = True
bShortDescription = True
bAdjective = True
bCityNames = True
bQuotes = True
if I was to set all the Language options to False any text tags created would be in the form of:

Code:
	<TEXT>
		<Tag>TXT_KEY_SOME_TEXT</Tag>
		<English>some text</English>
	</TEXT>
and turning search options to false ignores that type of node (e.g setting bPedia to False would mean that the program wouldn't bother to check the Civilopedia tags for incompete text.)

When entering paths make sure you use either / or \\ eg

C:/Users/User/Documents or C:\\Users\\User\\Documents

When the program starts it will backup the mods files in the path specified in options. When the program runs again it will delete the previous backup and create new ones. Make sure that when you are finished you check that there are no xml errors before running the program again!

Future features include an xml file for options instead of in the .py file (for non graphical setup) and italian and spanish translation fix, also more intelligent text tag management will be implemented to speed up translation time.

Post any bugs, questions or requests here!

Enjoy!
Attached Thumbnails
Click image for larger version

Name:	Pic1.jpg
Views:	64
Size:	118.4 KB
ID:	318366   Click image for larger version

Name:	Pic2.jpg
Views:	40
Size:	52.8 KB
ID:	318367   Click image for larger version

Name:	Pic3.jpg
Views:	36
Size:	125.6 KB
ID:	318368  
__________________
My mods = Jamie's Rome Mod 1.3.2, No Global Warming Mod My units = Great Generals, Ninja!, Desert Marine, African Marines, Chinese Rocketeer My utilities: XML text cleaner, City Optimiser/Simulator

Maths A*, Biology A*, Physics A in A levels!!!! Going to study Computer Science at Bristol University!

State Property, the new Forbidden Palace?

Jamie

Last edited by j_mie6; May 12, 2012 at 01:10 PM.
j_mie6 is offline   Reply With Quote
Old Apr 06, 2012, 05:10 PM   #2
dacubz145
Deity
 
dacubz145's Avatar
 
Join Date: Jun 2010
Location: Windy City
Posts: 3,667
Very nice work, thanks for taking up my request

I am going to update my mod in a day or two then im going to run this, since my backup will be uploaded here
__________________
Travel Through Time Mod Series

Whether you like the history of Africa, or Europe, or The Natives, or Modern Warfare we have a mod for you.


Help welcome my new WW1 mod TTT - In The Trenches
dacubz145 is offline   Reply With Quote
Old Apr 06, 2012, 05:25 PM   #3
j_mie6
Deity
 
j_mie6's Avatar
 
Join Date: Dec 2009
Location: Bristol (uni)/Swindon (home)
Posts: 2,944
Gender: Male
good luck!
__________________
My mods = Jamie's Rome Mod 1.3.2, No Global Warming Mod My units = Great Generals, Ninja!, Desert Marine, African Marines, Chinese Rocketeer My utilities: XML text cleaner, City Optimiser/Simulator

Maths A*, Biology A*, Physics A in A levels!!!! Going to study Computer Science at Bristol University!

State Property, the new Forbidden Palace?

Jamie
j_mie6 is offline   Reply With Quote
Old Apr 06, 2012, 06:22 PM   #4
j_mie6
Deity
 
j_mie6's Avatar
 
Join Date: Dec 2009
Location: Bristol (uni)/Swindon (home)
Posts: 2,944
Gender: Male
oh and truthfully, this project was a lot of fun and a good learning curve Well spent 12 straight hours of programming
__________________
My mods = Jamie's Rome Mod 1.3.2, No Global Warming Mod My units = Great Generals, Ninja!, Desert Marine, African Marines, Chinese Rocketeer My utilities: XML text cleaner, City Optimiser/Simulator

Maths A*, Biology A*, Physics A in A levels!!!! Going to study Computer Science at Bristol University!

State Property, the new Forbidden Palace?

Jamie
j_mie6 is offline   Reply With Quote
Old Apr 06, 2012, 06:55 PM   #5
The_J
Say No 2 Net Validations
 
The_J's Avatar
 
Join Date: Oct 2008
Location: Germany / Netherlands
Posts: 29,085
Congrats .
Sounds like an useful thing for everyone who started with the lazy way .
__________________
Civ4-BtS-Mod "Mars, Now!"


Steam eats the souls of little gamers!!!
The_J is offline   Reply With Quote
Old Apr 07, 2012, 11:55 AM   #6
j_mie6
Deity
 
j_mie6's Avatar
 
Join Date: Dec 2009
Location: Bristol (uni)/Swindon (home)
Posts: 2,944
Gender: Male
Ok I have finished work on the graphical interface

to run with the graphical interface you just need to double click on the XML text cleaner.py file and it is ready to go. Will upload later but going to put on a teaser picture

Notes
Will work on autofill of the options box so that user doesn't have to enter the path every single time they use it
__________________
My mods = Jamie's Rome Mod 1.3.2, No Global Warming Mod My units = Great Generals, Ninja!, Desert Marine, African Marines, Chinese Rocketeer My utilities: XML text cleaner, City Optimiser/Simulator

Maths A*, Biology A*, Physics A in A levels!!!! Going to study Computer Science at Bristol University!

State Property, the new Forbidden Palace?

Jamie
j_mie6 is offline   Reply With Quote
Old Apr 07, 2012, 12:20 PM   #7
dacubz145
Deity
 
dacubz145's Avatar
 
Join Date: Jun 2010
Location: Windy City
Posts: 3,667
Very interested to see the interface
__________________
Travel Through Time Mod Series

Whether you like the history of Africa, or Europe, or The Natives, or Modern Warfare we have a mod for you.


Help welcome my new WW1 mod TTT - In The Trenches
dacubz145 is offline   Reply With Quote
Old Apr 07, 2012, 12:22 PM   #8
j_mie6
Deity
 
j_mie6's Avatar
 
Join Date: Dec 2009
Location: Bristol (uni)/Swindon (home)
Posts: 2,944
Gender: Male
Going to upload it by the end of the day (and update the original post, as you can use one of the options in the .py file to turn off graphics (means you don't have to enter in the paths every time, but I hope to fix that soon with my good old fashioned pickle jar (don't ask )))

but the pictures are up now

also you can change the bg colour, font colour, font size and font type of that main box in the GraphicalInterface.py file
__________________
My mods = Jamie's Rome Mod 1.3.2, No Global Warming Mod My units = Great Generals, Ninja!, Desert Marine, African Marines, Chinese Rocketeer My utilities: XML text cleaner, City Optimiser/Simulator

Maths A*, Biology A*, Physics A in A levels!!!! Going to study Computer Science at Bristol University!

State Property, the new Forbidden Palace?

Jamie
j_mie6 is offline   Reply With Quote
Old Apr 07, 2012, 12:40 PM   #9
dacubz145
Deity
 
dacubz145's Avatar
 
Join Date: Jun 2010
Location: Windy City
Posts: 3,667
Looks really good, this is definatly coming around real nice!
__________________
Travel Through Time Mod Series

Whether you like the history of Africa, or Europe, or The Natives, or Modern Warfare we have a mod for you.


Help welcome my new WW1 mod TTT - In The Trenches
dacubz145 is offline   Reply With Quote
Old Apr 07, 2012, 01:07 PM   #10
j_mie6
Deity
 
j_mie6's Avatar
 
Join Date: Dec 2009
Location: Bristol (uni)/Swindon (home)
Posts: 2,944
Gender: Male
Version 2 is up!
__________________
My mods = Jamie's Rome Mod 1.3.2, No Global Warming Mod My units = Great Generals, Ninja!, Desert Marine, African Marines, Chinese Rocketeer My utilities: XML text cleaner, City Optimiser/Simulator

Maths A*, Biology A*, Physics A in A levels!!!! Going to study Computer Science at Bristol University!

State Property, the new Forbidden Palace?

Jamie
j_mie6 is offline   Reply With Quote
Old Apr 09, 2012, 04:37 AM   #11
j_mie6
Deity
 
j_mie6's Avatar
 
Join Date: Dec 2009
Location: Bristol (uni)/Swindon (home)
Posts: 2,944
Gender: Male
Ok I have a method of translation and I am workin gon getting it up and running as soon as possible
__________________
My mods = Jamie's Rome Mod 1.3.2, No Global Warming Mod My units = Great Generals, Ninja!, Desert Marine, African Marines, Chinese Rocketeer My utilities: XML text cleaner, City Optimiser/Simulator

Maths A*, Biology A*, Physics A in A levels!!!! Going to study Computer Science at Bristol University!

State Property, the new Forbidden Palace?

Jamie
j_mie6 is offline   Reply With Quote
Old Apr 09, 2012, 04:18 PM   #12
dacubz145
Deity
 
dacubz145's Avatar
 
Join Date: Jun 2010
Location: Windy City
Posts: 3,667
that is nice! how did you figure it out?
__________________
Travel Through Time Mod Series

Whether you like the history of Africa, or Europe, or The Natives, or Modern Warfare we have a mod for you.


Help welcome my new WW1 mod TTT - In The Trenches
dacubz145 is offline   Reply With Quote
Old Apr 09, 2012, 04:27 PM   #13
j_mie6
Deity
 
j_mie6's Avatar
 
Join Date: Dec 2009
Location: Bristol (uni)/Swindon (home)
Posts: 2,944
Gender: Male
myGengo API current having problems with italian and spanish though. The code is telling me they don't work
__________________
My mods = Jamie's Rome Mod 1.3.2, No Global Warming Mod My units = Great Generals, Ninja!, Desert Marine, African Marines, Chinese Rocketeer My utilities: XML text cleaner, City Optimiser/Simulator

Maths A*, Biology A*, Physics A in A levels!!!! Going to study Computer Science at Bristol University!

State Property, the new Forbidden Palace?

Jamie
j_mie6 is offline   Reply With Quote
Old Apr 27, 2012, 05:00 PM   #14
PsiCorps
FF: Babylon 5 mod team
 
PsiCorps's Avatar
 
Join Date: Dec 2007
Location: Britain
Posts: 1,304
This Utility program looks like it will be very useful for me however, you said
Quote:
first install python 2.7 from www.python.org (this is required for either V1 or V2)
Specifically which variation of 2.7? I use Win XP and am not as computer literate as most.

•Gzipped source tar ball (2.7.3) (sig)
•Bzipped source tar ball (2.7.3) (sig)
•XZ source tar ball (2.7.3) (sig)
•Windows x86 MSI Installer (2.7.3) (sig)
•Windows x86 MSI program database (2.7.3) (sig)
•Windows X86-64 MSI Installer (2.7.3) [1] (sig)
•Windows X86-64 program database (2.7.3) [1] (sig)

Please help me out.
__________________
"The Corps is Mother, the Corps is Father"

The Babylon Project: A Babylon 5 Mod for Civ IV BtS
PsiCorps is offline   Reply With Quote
Old Apr 28, 2012, 03:15 AM   #15
j_mie6
Deity
 
j_mie6's Avatar
 
Join Date: Dec 2009
Location: Bristol (uni)/Swindon (home)
Posts: 2,944
Gender: Male
Really I should have put python 2.7.? because you just take the latest version ie 2.7.3

then it is up to you which file you choose however I would say go for the top one on the page:

•Python 2.7.3 Windows Installer (Windows binary -- does not include source)
option as then it will install easily

hope that helped!
__________________
My mods = Jamie's Rome Mod 1.3.2, No Global Warming Mod My units = Great Generals, Ninja!, Desert Marine, African Marines, Chinese Rocketeer My utilities: XML text cleaner, City Optimiser/Simulator

Maths A*, Biology A*, Physics A in A levels!!!! Going to study Computer Science at Bristol University!

State Property, the new Forbidden Palace?

Jamie
j_mie6 is offline   Reply With Quote
Old Apr 29, 2012, 12:27 AM   #16
PsiCorps
FF: Babylon 5 mod team
 
PsiCorps's Avatar
 
Join Date: Dec 2007
Location: Britain
Posts: 1,304
Quote:
Originally Posted by j_mie6 View Post
Really I should have put python 2.7.? because you just take the latest version ie 2.7.3

then it is up to you which file you choose however I would say go for the top one on the page:

•Python 2.7.3 Windows Installer (Windows binary -- does not include source)
option as then it will install easily

hope that helped!
Perfect, just what I needed.

Thank you.
__________________
"The Corps is Mother, the Corps is Father"

The Babylon Project: A Babylon 5 Mod for Civ IV BtS
PsiCorps is offline   Reply With Quote
Old Apr 29, 2012, 12:55 AM   #17
dacubz145
Deity
 
dacubz145's Avatar
 
Join Date: Jun 2010
Location: Windy City
Posts: 3,667
any update on the translator?
__________________
Travel Through Time Mod Series

Whether you like the history of Africa, or Europe, or The Natives, or Modern Warfare we have a mod for you.


Help welcome my new WW1 mod TTT - In The Trenches
dacubz145 is offline   Reply With Quote
Old Apr 29, 2012, 05:32 AM   #18
j_mie6
Deity
 
j_mie6's Avatar
 
Join Date: Dec 2009
Location: Bristol (uni)/Swindon (home)
Posts: 2,944
Gender: Male
haven't got around to that yet, I still want my e-mail reply from the developer on why italian and spanish are both not working
__________________
My mods = Jamie's Rome Mod 1.3.2, No Global Warming Mod My units = Great Generals, Ninja!, Desert Marine, African Marines, Chinese Rocketeer My utilities: XML text cleaner, City Optimiser/Simulator

Maths A*, Biology A*, Physics A in A levels!!!! Going to study Computer Science at Bristol University!

State Property, the new Forbidden Palace?

Jamie
j_mie6 is offline   Reply With Quote
Old May 06, 2012, 06:04 PM   #19
j_mie6
Deity
 
j_mie6's Avatar
 
Join Date: Dec 2009
Location: Bristol (uni)/Swindon (home)
Posts: 2,944
Gender: Male
right the translator is finished! when the cleaner makes a new text tag it auto translates but there is another button on the GUI that allows you to further translate all your mods existing text tags aswell. The danger is that if you have the typical:

Code:
<TEXT>
           <Tag>TXT_KEY_ETC</Tag>
           <English></English>
           <French></French>
           <German></German>
           <Italian></Italian>
           <Spanish></Spanish>
</TEXT>
and only select French translation ALL your text tags will become:

Code:
<TEXT>
           <Tag>TXT_KEY_ETC</Tag>
           <English></English>
           <French></French>
</TEXT>
so make sure you select all the languages you want!!! this will be uploaded soon.

if the translation fails the tags will be the english equivilents. For the moment the Italian and Spanish translations are not working (something to do with MyGengo) and are typical english.

How the translator works (for those that are interested):
Spoiler:

when the Translate button is clicked/a new tag is made by the program the program then interfaces with the Translator class. In the case of a new tag the translators static get*****() methods are used to collect language *****, it basically sends a request to the MyGengo API and waits for response. With the GUI's Button a new instance of the Translator class is created and passes the path and other options, the __init__ then removes unneeded options. next, doTranslate is called and gets the files from the Text folder, the Text tags are "Replaced" to sort out problems with missing tags (during translation itself) the replaceTags method actually calls the classmethod disectNode() which destroys a TEXT node and throws back its mangled body parts. the replace tags method preceeds to perform life saving operations on the data to reform the tags with the required language tags. now that its all fine doTranslate swaps the language info with translated string and saves the xml (after reformating). finished . pretty nice setup I think.
__________________
My mods = Jamie's Rome Mod 1.3.2, No Global Warming Mod My units = Great Generals, Ninja!, Desert Marine, African Marines, Chinese Rocketeer My utilities: XML text cleaner, City Optimiser/Simulator

Maths A*, Biology A*, Physics A in A levels!!!! Going to study Computer Science at Bristol University!

State Property, the new Forbidden Palace?

Jamie

Last edited by j_mie6; May 06, 2012 at 06:11 PM.
j_mie6 is offline   Reply With Quote
Old May 06, 2012, 06:32 PM   #20
dacubz145
Deity
 
dacubz145's Avatar
 
Join Date: Jun 2010
Location: Windy City
Posts: 3,667
Can't wait till you upload it
__________________
Travel Through Time Mod Series

Whether you like the history of Africa, or Europe, or The Natives, or Modern Warfare we have a mod for you.


Help welcome my new WW1 mod TTT - In The Trenches
dacubz145 is offline   Reply With Quote
Reply

Bookmarks

Go Back Civilization Fanatics' Forums > CIVILIZATION IV > Civ4 - Creation & Customization > Civ4 - Utility Programs > [UTILITY] XML text cleaner

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off



Advertisement

All times are GMT -6. The time now is 03:59 PM.


Powered by vBulletin®
Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
This site is copyright © Civilization Fanatics' Center.
Support CFC: Amazon.com | Amazon UK | Amazon DE | Amazon CA | Amazon FR