1. We have added the ability to collapse/expand forum categories and widgets on forum home.
    Dismiss Notice
  2. All Civ avatars are brought back and available for selection in the Avatar Gallery! There are 945 avatars total.
    Dismiss Notice
  3. To make the site more secure, we have installed SSL certificates and enabled HTTPS for both the main site and forums.
    Dismiss Notice
  4. Civ6 is released! Order now! (Amazon US | Amazon UK | Amazon CA | Amazon DE | Amazon FR)
    Dismiss Notice
  5. Dismiss Notice
  6. Forum account upgrades are available for ad-free browsing.
    Dismiss Notice

Automating Modding Tasks

Discussion in 'Civ6 - Utilities, Code Snippets & Art Assets' started by thecrazyscot, Feb 15, 2018.

  1. thecrazyscot

    thecrazyscot Spiffy

    Joined:
    Dec 27, 2012
    Messages:
    1,674
    As I've been fiddling about manually modding stuff, I realized that there's a good portion of the modding workflow which can be automated via scripts.
    • Some are available as exe
    • If the file is a py script, Python (2.7+) must be installed to use them
    • Run the exe or script by simply double-clicking on it
    • All inputs are given via a pop-up window
    I'll be adding to this repository as I have the time/inclination.

    SCRIPTS
    ModArt_Generator (exe, py script)
    Automatically generate your Mod.Art file! No more painstakingly placing each artdef and blp in its proper place!

    Input:
    valid Modbuddy project path with an existing Mod.Art file (eg: C:\Users\thecrazyscotsman\Documents\Firaxis ModBuddy\Civilization VI\TCS City States\TCS City States)
    Output: Scans the project folder for ArtDefs and XLPs and places them correctly within a complete Mod.Art file. Saves a backup copy of the original Mod.Art within the project as well

    CityName_Generator (py script)
    Input: .txt file with a single city name per line
    Output: Complete sql database entries and xml text entries for every included city name

    CityState_Generator (exe)
    Quickly create (almost) all the necessary code for any number of city-states in seconds!

    Input: .txt file with a complete entry per line (see the post below for entry requirements)
    Output: 5 files (CityStates_Database.sql, CityStates_Text.sql, CityStates_Icons.xml, Civilizations.artdef, and Cultures.artdef) - all you now need to do is go in and "fill in the blanks" by inputting Modifiers, Requirements, .dds file names, etc. Everything else is automatically created

    Civilization_ArtDef_Generator (exe)
    Input: Follow prompts in popup window
    Output: Civilizations.artdef and Cultures.artdef for a single CivilizationType entry
     
    Last edited: Mar 21, 2018
  2. thecrazyscot

    thecrazyscot Spiffy

    Joined:
    Dec 27, 2012
    Messages:
    1,674
    ModArt_Generator
    INSTRUCTIONS:
    1. Get the path to a project folder (the one with the civ6proj file in it)
    1.PNG

    2. Run the exe or py script, this window pops up
    2.PNG

    3. Paste your folder path
    3.PNG

    4. Hit "Enter"

    The zip contains both the exe and py script for those leery of running exe files.
     

    Attached Files:

    failuer, neonpinku and raen like this.
  3. thecrazyscot

    thecrazyscot Spiffy

    Joined:
    Dec 27, 2012
    Messages:
    1,674
    CityName_Generator
    v2 -- Now supports accented and other special characters.

    INSTRUCTIONS
    1. Create a text file with a single city name on each line
    4.PNG

    2. Provide the path for your text file
    3. Provide the output folder
    4. Provide the civilization type (any case, the script automatically converts it to upper case)
    5.PNG

    5. Hit "Enter"
     

    Attached Files:

  4. thecrazyscot

    thecrazyscot Spiffy

    Joined:
    Dec 27, 2012
    Messages:
    1,674
    CityState_Generator
    INSTRUCTIONS:
    1. Create a txt file with each entry in this format: City-State Name,Category,Ancient City Culture,Unique City Culture,Unit Culture (you can use the included Excel file to easily create entries in this format)
    csg_txt.PNG

    2. Run the .exe (you may need to run as administrator)
    3. Input the file path for the txt file you just created
    4. Input where you want to save your files
    5. Input your modding "tag"
    csg_input.PNG

    6. The results pop up so you can make sure you spelled everything correctly (the generator only takes what it's given, folks)
    csg_results.PNG

    7. The output files are now in a new folder called CityState_Output located wherever you specified it should be
    csg_output.PNG

    You can review the attached sample output to examine the individual files that are created.
     

    Attached Files:

    Laurana Kanan likes this.
  5. thecrazyscot

    thecrazyscot Spiffy

    Joined:
    Dec 27, 2012
    Messages:
    1,674
    Civilization_ArtDef_Generator
    INSTRUCTIONS
    1. Run the .exe (you may need to run as administrator)
    2. Follow prompts
    2a. CivilizationType needs to be in the same format as in your database files
    2b. The cultures will not let you pick a culture which is not in "the list", so if you type an invalid culture it will print a list of your options and ask for your input again
    2c. Output Folder can be your Modbuddy ArtDef folder (or anywhere really), or for speed you can just type "desktop" without the quotes and it will save the files to your...desktop
    Capture.PNG
     

    Attached Files:

    Last edited: Mar 21, 2018
    neonpinku and Laurana Kanan like this.
  6. TheLunarArmy

    TheLunarArmy Chieftain

    Joined:
    Jul 9, 2014
    Messages:
    58
    Gender:
    Male
    Location:
    South-Africa
    Looks good, is your source code Python? If so look at some basic tkinter for ui; I find that people tend to shy away from cmd and text editing-based programs :p
     
  7. Deliverator

    Deliverator Graphical Hackificator

    Joined:
    Feb 12, 2008
    Messages:
    3,990
    Location:
    London, UK
    A couple of scripts from me that I thought might be worth sharing.


    Mod File Pruner for Texture and Animation files that are included in base game

    By default Modbuddy will often build out a large number of ANIMATION and TEXTURE files that included in the base game and so do not need to be bundled with the mod. Removing these can significantly reduce the size of your mod download.

    The output under "Keep in Mod:" can be copy-pasted into your .modinfo for the Platforms section under <Files>. To actually delete the duplicated files rather than just list them uncomment the last two lines.

    Script (Python 3):
    Code:
    import os
    
    modroot = "E:\\mod\\MOAR_Units\\UnitExpansion\\"
    winblpfolder = "Platforms\\Windows\\BLPs\\"
    macblpfolder  = "Platforms\\MacOS\\BLPs\\"
    
    winmodroot = modroot+winblpfolder
    macmodroot = modroot+macblpfolder
    
    gameroot = "E:\\SteamLibrary\\steamapps\\common\\Sid Meier's Civilization VI\\Base\\Platforms\\Windows\\BLPs\\SHARED_DATA\\"
    
    modfilelist = []
    
    for path, subdirs, files in os.walk(winmodroot):
        for name in files:
            shortfilename = os.path.basename(name)
            modfilelist.append(shortfilename)
    
    unneededfilelist = []
    for path, subdirs, files in os.walk(gameroot):
        for name in files:
            basename = os.path.basename(name)
            if basename in modfilelist:
                unneededfilelist.append(name)
                modfilelist.remove(basename)
    
    print("\nKeep in Mod:")
    for name in modfilelist:
        print("<File>" + macblpfolder + name + "</File>")
    for name in modfilelist:
        print("<File>" + winblpfolder + name + "</File>")
    
    print("\nRemove from Mod:")
    for name in unneededfilelist:
        macfilename = macmodroot + "SHARED_DATA\\" + name
        winfilename = winmodroot + "SHARED_DATA\\" + name
        print(macfilename)
        print(winfilename)
    
        # Uncomment to Delete Files
        #os.remove(macfilename)
        #os.remove(winfilename)

    XML -> SQL Converter

    Takes a Database XML and converts it to Database SQL.

    Script (Python 3):

    Code:
    import xml.etree.ElementTree as ET
    
    xml = ET.parse("E:\\SteamLibrary\\steamapps\\common\\Sid Meier's Civilization VI\\Base\\Assets\\Gameplay\\Data\\Units.xml")
    
    root = xml.getroot()
    
    for child in root:
    
        print ("")
        print ("--"+child.tag)
        if child.tag not in ('UnitPromotionModifiers','UnitAbilityModifiers','Modifiers','ModifierArguments','RequirementSets','RequirementSetRequirements','Requirements','RequirementArguments'):
            for child2 in child:
    
                keysString = ", ".join(child2.attrib.keys())
                values = []
                for key in child2.attrib.keys():
                    values.append(child2.attrib[key])
    
                valuesString =  ", ".join("'{0}'".format(w) for w in values)
    
                print ("INSERT INTO " + child.tag + " (" + keysString + ") VALUES (" + valuesString + ");")
        else:
            for child2 in child:
    
                keys = []
                values = []
                for child3 in child2:
                    keys.append(child3.tag)
                    values.append(child3.text)
    
                keysString = ", ".join(keys)
                valuesString =  ", ".join("'{0}'".format(w) for w in values)
    
                print ("INSERT INTO " + child.tag + " (" + keysString + ") VALUES (" + valuesString + ");"
    
    
    Input:
    Code:
            <Row Type="UNIT_BRAZILIAN_MINAS_GERAES" Tag="CLASS_ANTI_AIR"/>
            <Row Type="UNIT_ANTIAIR_GUN" Tag="CLASS_ANTI_AIR"/>
            <Row Type="UNIT_MOBILE_SAM" Tag="CLASS_ANTI_AIR"/>
        </TypeTags>
        <Units>
            <Row UnitType="UNIT_SETTLER" Cost="80" BaseMoves="2" BaseSightRange="3" ZoneOfControl="false" Domain="DOMAIN_LAND" FormationClass="FORMATION_CLASS_CIVILIAN" FoundCity="true" PopulationCost="1" PrereqPopulation="2" AdvisorType="ADVISOR_GENERIC" Name="LOC_UNIT_SETTLER_NAME" Description="LOC_UNIT_SETTLER_DESCRIPTION" CanCapture="False" CostProgressionModel="COST_PROGRESSION_PREVIOUS_COPIES" CostProgressionParam1="30" PurchaseYield="YIELD_GOLD" PseudoYieldType="PSEUDOYIELD_UNIT_SETTLER"/>
            <Row UnitType="UNIT_BUILDER" Cost="50" BaseMoves="2" BaseSightRange="2" ZoneOfControl="false" Domain="DOMAIN_LAND" FormationClass="FORMATION_CLASS_CIVILIAN" AdvisorType="ADVISOR_GENERIC" Name="LOC_UNIT_BUILDER_NAME" Description="LOC_UNIT_BUILDER_DESCRIPTION" CanCapture="False" CostProgressionModel="COST_PROGRESSION_PREVIOUS_COPIES" CostProgressionParam1="4" PurchaseYield="YIELD_GOLD" BuildCharges="3"/>
            <Row UnitType="UNIT_TRADER" Cost="40" BaseMoves="2" BaseSightRange="2" ZoneOfControl="false" Domain="DOMAIN_LAND" FormationClass="FORMATION_CLASS_CIVILIAN" AdvisorType="ADVISOR_GENERIC" Name="LOC_UNIT_TRADER_NAME" Description="LOC_UNIT_TRADER_DESCRIPTION" CanCapture="False" CostProgressionModel="COST_PROGRESSION_GAME_PROGRESS" CostProgressionParam1="400" PurchaseYield="YIELD_GOLD" PseudoYieldType="PSEUDOYIELD_UNIT_TRADE" IgnoreMoves="true" MakeTradeRoute="true" PrereqCivic="CIVIC_FOREIGN_TRADE"/>
    
    Output:
    Code:
    ...
    INSERT INTO TypeTags (Type, Tag) VALUES ('UNIT_BRAZILIAN_MINAS_GERAES', 'CLASS_ANTI_AIR');
    INSERT INTO TypeTags (Type, Tag) VALUES ('UNIT_ANTIAIR_GUN', 'CLASS_ANTI_AIR');
    INSERT INTO TypeTags (Type, Tag) VALUES ('UNIT_MOBILE_SAM', 'CLASS_ANTI_AIR');
    --Units
    INSERT INTO Units (UnitType, Cost, BaseMoves, BaseSightRange, ZoneOfControl, Domain, FormationClass, FoundCity, PopulationCost, PrereqPopulation, AdvisorType, Name, Description, CanCapture, CostProgressionModel, CostProgressionParam1, PurchaseYield, PseudoYieldType) VALUES ('UNIT_SETTLER', '80', '2', '3', 'false', 'DOMAIN_LAND', 'FORMATION_CLASS_CIVILIAN', 'true', '1', '2', 'ADVISOR_GENERIC', 'LOC_UNIT_SETTLER_NAME', 'LOC_UNIT_SETTLER_DESCRIPTION', 'False', 'COST_PROGRESSION_PREVIOUS_COPIES', '30', 'YIELD_GOLD', 'PSEUDOYIELD_UNIT_SETTLER');
    INSERT INTO Units (UnitType, Cost, BaseMoves, BaseSightRange, ZoneOfControl, Domain, FormationClass, AdvisorType, Name, Description, CanCapture, CostProgressionModel, CostProgressionParam1, PurchaseYield, BuildCharges) VALUES ('UNIT_BUILDER', '50', '2', '2', 'false', 'DOMAIN_LAND', 'FORMATION_CLASS_CIVILIAN', 'ADVISOR_GENERIC', 'LOC_UNIT_BUILDER_NAME', 'LOC_UNIT_BUILDER_DESCRIPTION', 'False', 'COST_PROGRESSION_PREVIOUS_COPIES', '4', 'YIELD_GOLD', '3');
    INSERT INTO Units (UnitType, Cost, BaseMoves, BaseSightRange, ZoneOfControl, Domain, FormationClass, AdvisorType, Name, Description, CanCapture, CostProgressionModel, CostProgressionParam1, PurchaseYield, PseudoYieldType, IgnoreMoves, MakeTradeRoute, PrereqCivic) VALUES ('UNIT_TRADER', '40', '2', '2', 'false', 'DOMAIN_LAND', 'FORMATION_CLASS_CIVILIAN', 'ADVISOR_GENERIC', 'LOC_UNIT_TRADER_NAME', 'LOC_UNIT_TRADER_DESCRIPTION', 'False', 'COST_PROGRESSION_GAME_PROGRESS', '400', 'YIELD_GOLD', 'PSEUDOYIELD_UNIT_TRADE', 'true', 'true', 'CIVIC_FOREIGN_TRADE');
    ...
    
    I think there is one issue with this in that you need to replace 'true' with the number 1 and 'false' with the number 0 once you are done. Haven't got around to doing this programatically yet.



     
    Angryr likes this.
  8. thecrazyscot

    thecrazyscot Spiffy

    Joined:
    Dec 27, 2012
    Messages:
    1,674
    Thanks! Yes, I'm doing it all in Python. I've done some preliminary looking into tkinter but haven't found the time yet to attempt some implementation. As these scripts are as much practice exercises for me as ways to reduce modding time, I will probably attempt it some point in the future (all my previous Python experience was very specialized in writing ArcGIS scripts, where UI was not a concern, whether in the script context or the program context in general!).

    Thanks for sharing these! Looking at scripts from far more experienced programmers is also helpful for me just in continuing to figure out more efficient ways of doing things. My earlier scripts (ModArt, for example) are far too manual, and while I'm improving I still have a long ways to go.
     
  9. poundjd

    poundjd Chieftain Supporter

    Joined:
    Dec 4, 2013
    Messages:
    108
    Gender:
    Male
    Location:
    USA PA 17222
    @thecrazyscot & @Deliverator,
    I can't use ModBuddy yet (ANY pointers to a really good guide would mean the most to me), but if these could be integrated into ModBuddy I am sure the community would appreciate it!
    -jeff
     
  10. Deliverator

    Deliverator Graphical Hackificator

    Joined:
    Feb 12, 2008
    Messages:
    3,990
    Location:
    London, UK
    @thecrazyscot There is a small issue with Mod.Art.xml generator in that if you actually have Clutter.artdef in your mod you end up with 2 duplicated rows like this:

    Code:
    <Element>
       <consumerName text="Clutter"/>
       <relativeArtDefPaths>
          <Element text="Clutter.artdef"/>
          <Element text="Clutter.artdef"/>
       </relativeArtDefPaths>
       <libraryDependencies>
          <Element text="Landmark"/>
       </libraryDependencies>
       <loadsLibraries>true</loadsLibraries>
    </Element>
    I should say that it's a really really useful tool that I use a lot!
     

Share This Page

Ebates: Get Paid to Shop