OrgrinRT's Learning Log AKA The pitch-in thread for helpful souls

[orgrinrt]

I thought I might share the working final bit in case somebody needs similiar functionality.

Spoiler :

-- UGPFunctions.lua
-- Author: Olritoim A.K.A OrgrinRT
-- DateCreated: 11/24/2016 9:42:58 PM
--------------------------------------------------------------

--------------------------------------------------------------
-- Big thanks to LeeS and Chrisy15 for doing the base for this and essentially making it work!
--------------------------------------------------------------
local iPromotionUnitBorn = GameInfoTypes["PROMOTION_CREATED"]
local civilizationID = GameInfoTypes["CIVILIZATION_FINNS"]
local tGreatPeopleClassesToProcess = {
    [GameInfoTypes["UNITCLASS_SCIENTIST"]] = true,
    [GameInfoTypes["UNITCLASS_ENGINEER"]] = true,
    [GameInfoTypes["UNITCLASS_MERCHANT"]] = true,
    [GameInfoTypes["UNITCLASS_ARTIST"]] = true,
    [GameInfoTypes["UNITCLASS_WRITER"]] = true,
    [GameInfoTypes["UNITCLASS_MUSICIAN"]] = true,
    [GameInfoTypes["UNITCLASS_PROPHET"]] = true,
    [GameInfoTypes["UNITCLASS_GREAT_GENERAL"]] = true,
    [GameInfoTypes["UNITCLASS_GREAT_ADMIRAL"]] = true,
    [GameInfoTypes["UNITCLASS_SCOUT"]] = true,
    [GameInfoTypes["UNITCLASS_JFD_GREAT_SCOUT"]] = true,
    [GameInfoTypes["UNITCLASS_JFD_ADVENTURER"]] = true,
    [GameInfoTypes["UNITCLASS_JFD_GREAT_ADVENTURER"]] = true,
    [GameInfoTypes["UNITCLASS_JFD_EXPLORER"]] = true,
    [GameInfoTypes["UNITCLASS_JFD_GREAT_EXPLORER"]] = true,
    [GameInfoTypes["UNITCLASS_JFD_GREAT_DIGNITARY"]] = true,
    [GameInfoTypes["UNITCLASS_JFD_GREAT_MAGISTRATE"]] = true,
    }

-- JFD_IsCivilisationActive
function JFD_IsCivilisationActive(civilisationID)
    for iSlot = 0, GameDefines.MAX_MAJOR_CIVS-1, 1 do
        local slotStatus = PreGame.GetSlotStatus(iSlot)
        if (slotStatus == SlotStatus["SS_TAKEN"] or slotStatus == SlotStatus["SS_COMPUTER"]) then
            if PreGame.GetCivilization(iSlot) == civilisationID then
                return true
                end
        end
    end
    return false
end

-- JFD_GetRandom
function JFD_GetRandom(lower, upper)
    return Game.Rand((upper + 1) - lower, "") + lower
end

-- This bit is based on LeeS's sample code
function GetNameForGreatPerson(pUnit)
    if pUnit ~= nil then
        local sUnitType = GameInfo.Units[pUnit:GetUnitType()].Type
        local playerID = pUnit:GetOwner()
        local pPlayer = Players[playerID]
            local tCorrectUnitNames = {}
            for row in DB.Query("SELECT UnitType, UniqueName FROM Unit_UniqueNames WHERE UnitType = \'" .. sUnitType .. "\' AND CivilizationType = 'CIVILIZATION_FINNS'") do
                table.insert(tCorrectUnitNames, row.UniqueName)
            end
                if #tCorrectUnitNames > 0 then
                    if #tCorrectUnitNames == 1 then
                    return tCorrectUnitNames[1]
                        else
                        return tCorrectUnitNames[JFD_GetRandom(1, #tCorrectUnitNames)]
                    end
                end
    end
    return "Tuntematon Sotilas"
end

--This bit is based on Chrisy15's sample
function C15_GPBorn(playerID, unitID)
    local pPlayer = Players[playerID]
    if pPlayer:GetCivilizationType() == civilizationID then
        local pUnit = pPlayer:GetUnitByID(unitID)
        if pUnit then
            if tGreatPeopleClassesToProcess[pUnit:GetUnitClassType()] then
                local sSelectedName = GetNameForGreatPerson(pUnit)
                if sSelectedName then -- If the civ has no entries, then this will be nil; we must check for this
                    pUnit:SetName(sSelectedName)
                    Events.UnitNameChanged(playerID, unitID)
                end
            end
        end
    end
end
--if JFD_IsCivilisationActive(civilizationID) then
    LuaEvents.SerialEventUnitCreatedGood.Add(C15_GPBorn)
--end

It's a simple thing that basically just renames each born great person with a civilization-specific name. All it needs to function is Machiavelli's SerialEventCreatedGood, the names added in the table, a xml file that has the text strings and a sql file to add the CivilizationType column in the Unit_UniqueNames in case the user doesn't have LeeS's mod.

Thanks a bunch to everyone involved! It feels truly amazing to have something like this done, eventhough it's essentially a product of your labor, not mine. Either way, it's alive and working, and I can continue to next bits with a heart full of excitement.

Next thing I need to figure out is how to have era-specific UnitSelect and UnitOrder sounds. But that's probably going to be its own mod, so I think I'll just finish with this package, apply it to some civs and then see if it's doable for me. I'll probably keep bugging you guys, if you don't mind

EDIT: By the way, it's actually pretty neat it only took 3 days to complete it. The involvement from you, the community, is pretty damn amazing and quick! I've only been learning this for 5 days, so this whole thing is pretty unimaginable without this community being so awesome

EDIT2: Though I definitely suggest everyone to use LeeS's mod, it's way better and more complete with the table saving functionality from TSL. He's also made it very well-documented and added the functionality to also rename all the GP, not only GGs or GAs.

While we're at the subject of saving the table - I might want to see how to apply the SaveUtils here. I'm thinking it might be faster to save the GP names table and have it load it from there?

 

[Chrisy15]

While we're at the subject of saving the table - I might want to see how to apply the SaveUtils here. I'm thinking it might be faster to save the GP names table and have it load it from there?

I doubt there'd be any noticeable difference tbh. If you really wanted to optimise, you could try filling tCorrectUnitNames outside the function at the start of the file, and then checking through that when needed; I don't think it'd really be necessary tho.

 

[LeeS]

It should only make a difference really for xml-tables that have a lot of rows in them. In this case caching the data in an lua table or a variable (if you are only interested in one piece of info) is better.

If Great General & Admiral Names is active you would be processing through 1200+ names if you did not have the DB.Query presort working in your favor of only looking at the rows that match to the current UnitType and CivilzationType you are specifically interested in.

That having been said I generally advocate for caching everything you will need out of the game's database when the game is loaded, and storing the results in an lua table or lua variable as needed for the sort of information you are looking at. So a Civ's ID# you just directly place into a variable, but information about all units that are "X" + "Y" you would normally store into an lua table.

Another consideration is how often you are likely to be accessing the information. If you are going to access it two or three times per game session then it does not really matter all that much whether you cache the info into lua tables or variables instead of looking it up directly those two or three times you need it. But if you expect your code will need to make reference to the same information dozens of times per-turn, then you probably need to cache instead of looking through the database every time you need this information.

 

[orgrinrt]

Yeah, with the database query I might be good without actually caching them for now. Maybe that'd be getting in over my head.

Anyway, I had to make kind of a detour. Seeing as I'm planning on having multiple civs there to bundle together, I decided that instead of copypasting that code in each civ's folder, I might make the core modular enough that every civ can run the same functions from there and just have the civ variables defined in their own files.

Well, turns out I don't really understand that concept either. So, at the same time, the thing is now completely broken again Yeah, this'll take a bit more than a couple of days. At least I know what I'm going to be doing.


EDIT:

ehh... so the problem was I was getting a flood of nil values.

Guess what? I forgot to add the new lua file into the content. Yes! I did it again. I've now edited the original bit to an unrecognizable form, and I bet the first version worked just fine if I had just remembered to add it to the content. fudge me.

 

[orgrinrt]

Today's been all about Finnish unique great people. 170 names researched and added to the database. Still need to add another similiar bunch more, but it's underway at least.

I also thought it'd be best to seperate the mods I'm working with here, so I started to make the UGP module more... well, modular, so that I can use it with my future civs. If I ever get around making any future civs. I had to completely rework the structure, and where it does make sense now, I somehow broke it in the process. I'm still a little bit unaware as to why that is, but I guess that's going to dictate how I'm going to spend the night.

Anyway, as I'm nearly finished with the Hyperion patch and have everything I need to add some more civs in, I'd like to know if someone's abandoned a civ and doesn't want to make it up-to-date? I'm willing to patch it all up and nice if there's anyone out there like that. The thing is, I figured I shouldn't start just meddling with things I'm not sure I have the "right" to meddle with, so I'll just leave it up to you.

On that note, I'm also going to compile a few new civs together. Since my knowledge unfortunately limits me to only really understand the northern european and similiar cultures, I'll go and find some contacts around different cultures and have some dialogue there, maybe I can then decide if there's a civ I can work with.

For now, I have a small list, and I'm only probably gonna do one or two civs and that with minor focus, so chances are I'll abandon this whole thing before I finish anything

But yeah. The upcoming weeks I'm gonna spend some more time researching for great people for the finnish civs, fix the UGP, see if the era mod is doable and start investigating on which civs I might do. This is more of a log for myself, so I don't completely forget what I've been planning here.

Spoiler :

• The Karelian Republic
  
• Ingria
• The Kvens
• Estonia (if there's no existing one that's up-to-date and otherwise decent and balanced)
• Latvia (probably not)
• Åland (check the historical name, and if it's associated with Gotland and the Swedes in general = some sort of independent kingdom far back?)

EDIT:


Aight, Kekkonen has got himself a brand new all-around support for JFD's mods. I was surprised that it only had veery old culdiv support. Well, now he can go all mad with making Finland a soviet state with his reform preferences

I'm still debating whether or not I should just have all the same names used for governments et such for both of the Finlands. I'm thinking changing at least Mannerheim's "Marsalkka" title would be required. It'd be neat to make Kekkonen the General Secretary of the Central Committee if the player chooses dictatorship ^



EDIT2:

Probably getting a little overboard with these, but we started working on the other Finland's dom narrative with Mr. Harris anyway, so we made some edits on Mannerheim's bit as well. It now, at least in my opinion, compliments the atmosphere a little better.

So, both doms will be finalized within a few days, and then there's not much to do with this patch.

I still haven't figured out what went wrong with me breaking all the different pieces apart and into their own mods, but I bet it's something completely dumb that I'm just overlooking.

The saga continues...

 

[orgrinrt]

So Kekkonen is now working and unique great people applies to both correctly.

I added YNAEMP support to make use of the v24 maps as well as added Kekkonen's city-states TSL support. I somehow succeeded at breaking the YNAEMP loading process. I'm guessing there's some very obvious typo somewhere, but haven't been able to locate it just yet. The only civs YNAEMP setup screen now recognizes are the two Finnish civs. Other mapscripts work correctly.

For some reason, YNAEMP now also shows oil, if the resource is within starting city's view range. Which shouldn't be possible I take, at least I haven't witnessed this before in the numerous games I've played in YNAEMP maps.

So I guess I know what I'm going to do today. I thought I had this thing together, but obviously I don't.

EDIT: Scratch that, it seems that the problem isn't with this thing but with the vanilla civs. Other custom civs show correctly in the YNAEMP setup screen. Now that I think, I think I had this before, I just had such a huge list of custom civs that the list was plentiful anyway.

What comes to the oil "problem", I actually just now realized I started in industrial age and I'm pretty sure it's supposed to show at that point.

So... no problems after all, it's working! Though I'd really like to make the vanilla civs work with TSL. I can get them show in the list if I choose TSL and random, but when I choose TSL, I only get custom civs.


EDIT2: Hmm. So turns out, there's no sql to add in the tsl for yagem. Some maps have a huge amount of startinpositions, like europe huge, including civs I have never installed, but most maps have zero TSL updated into the database. I'm guessing this isn't an error in my end really, I'll try and go through the YNAEMP files to see if there should be support for starting locations for at least the vanilla civs.



EDIT3: Got it. Only JFD's additions are correctly updated into the database, and it doesn't include yagem or yahem etc. Pretty weird! I'll continue my search, but I'm thinking the v24 of ynaemp just doesn't have the file that adds the TSL for yagem or yahem.


EDIT4: Yeah the xml which I presume is the "original" starting position file, doesn't have yagem or yahem in them. That explains a bunch. So note to self: when testing, use some other maps.


EDIT5: Right, finally understood the root of this problem. The reference with YNAEMP was missing one digit in the modID, so it wasn't the first mod to create the ynaemp etc. map tables, and since it wants to write, not merely update, it just error'd and decided it will not write the names in the table.



EDIT6: Wow, such edits.

Anyway, the above didn't fix the problem. The problem was with Hypereon's Finland (Kekkonen). It didn't have a reference to YNAEMP, and I guess for some reason in the load order it always loaded before ynaemp did, and created the tables before ynaemp's files could and ynaemp then always went and error'd on me. I added the reference to it, but I'm not sure at all how to add that into this external patch mod..

How does one rewrite files? I guess this would have to be done via lua, and I'm not even sure if it has the capabilities or rights to rewrite files that way. The other option would be to replace the file completely with the patched one, but again, not sure how to do this with an external mod. I'd love to be able to patch all these problems with this one mod, instead of everyone having to go and edit the files seperately. I guess I'll have to see if this is doable.

 

[orgrinrt]

In addition to finally fixing the obviously dumb problem I had with the ynaemp, it all seems to be completely working now. Apart from having to replace or rewrite the .modinfo file on Hypereon's Kekkonen. Which I'll get to at some other time, for now it works for me and that's the base of this whole project; making it work for me.

On that note though, I started to work a little bit more on the little UGP mod. Since I have some civs planned that don't necessarily need their own specific great people, but can use a mix of a couple of packages, I thought I'd go and associate the great people names with JFD's CulDiv groups. So the GP names are now associated with that, and all the relevant civs that need to have their unique great people, will now be added to a new table that just contains the culturetype and the info that it is indeed a relevant civ.

I still haven't figured out how to exactly implement this succesfully though. For now, I'm only able to fetch the relevant civs from the table and make sure that the name changing stuff only happens if one of them is active. It also now fetches the names with the CivilizationType tag in the relevan civs table. But what I haven't figured out, is how to group multiple tags into one and only use that (i.e the CultureType tags instead of CivilizationType). I guess I'm going to have to figure this out inside lua, which is still pretty unknown to me as it is. I'll see to it.

 

[Chrisy15]

How does one rewrite files? I guess this would have to be done via lua, and I'm not even sure if it has the capabilities or rights to rewrite files that way. The other option would be to replace the file completely with the patched one, but again, not sure how to do this with an external mod. I'd love to be able to patch all these problems with this one mod, instead of everyone having to go and edit the files seperately. I guess I'll have to see if this is doable.

Yeah, you're probably not gonna be able to edit the .modinfo from an external mod. Easier just to add a note on Hype's thread so that people can fix it themselves.

On that note though, I started to work a little bit more on the little UGP mod. Since I have some civs planned that don't necessarily need their own specific great people, but can use a mix of a couple of packages, I thought I'd go and associate the great people names with JFD's CulDiv groups. So the GP names are now associated with that, and all the relevant civs that need to have their unique great people, will now be added to a new table that just contains the culturetype and the info that it is indeed a relevant civ.

I still haven't figured out how to exactly implement this succesfully though. For now, I'm only able to fetch the relevant civs from the table and make sure that the name changing stuff only happens if one of them is active. It also now fetches the names with the CivilizationType tag in the relevan civs table. But what I haven't figured out, is how to group multiple tags into one and only use that (i.e the CultureType tags instead of CivilizationType). I guess I'm going to have to figure this out inside lua, which is still pretty unknown to me as it is. I'll see to it.

Unsure what you're trying to do here. Are you trying to replace the civ-specific GPs with culture-specific GPs? Or are you aiming to have both?

 

[orgrinrt]

The aim is to have both. So let me explain it a little bit:

The "library" of civ-specific GP names consists of those for specific civs, obviously. So every list is tagged with civtype. For example:

• UGP_FINNS.sql would add names to Unit_UniqueNames with CIVILIZATION_FINNS
• UGP_SWEDES.sql would add names to Unit_UniqueNames with CIVILIZATION_SWEDES
  
• UGP_RUSKIS.sql would add names to Unit_UniqueNames with CIVILIZATION_RUSKIS etc. etc.

What I want to do, to further it, is create a method of assigning a combination of multiple of those to one civilization. For example

• UGP_INGRIA would be a combination of UGP_FINNS and UGP_RUSKIS and it would draw the random name from a combined list
• UGP_KVENNS would be a combination of UGP_FINNS, UGP_NORSKIS and UGP_SWEDES etc. etc.

I feel like the solution is obvious, but I haven't got around to it just yet, since I'm trying to make some things more modular and streamlined.

Basically I'd just have to create a hash of sorts, I guess. I think this could be handled easily in the database by making a new table for the hashes and they'd just duplicate all the names with a simple sql bit, but the problem is that would just needlessly flood the database; there'd have to be a better way to do it. And my thinking is, it's probably pretty easy to just create a hash within lua with the lua tables, but since I'm still very unfamiliar with the whole programming thing, it intimidated me enough for me to just focus on streamlining the existing parts for now, instead of attempting to actually do it.

I'll get to it eventually, though, and I'd rather have myself figure it out instead of getting a free sample code this time. But any vague pointers would be more than welcome though!

EDIT: Actually I think I cracked the nut there myself; just need to create a table in lua and fill it with all the names the current civ needs. Haha. Dumb me.

Also I'm not 100% sure that hyp's Kekkonen was the problem, but adding in the reference did fix it for me. I'm cautious in thinking that was the problem though, because I did do a bunch of potential fixes between the tests so it actually could have been a number of other things I meddled with.


EDIT2:

Seems like the Unique Cultural Influence was also set to JFD_Turks for some reason in Mannerheim's files. The text was finnish tho, so no idea what was going on in there. Also fixed some typos that were throwing errors in the xml.log for Mannerheim. Added UCI support to Kekkonen.

 

[orgrinrt]

I'm doing something wrong, and I can't understand what.

Lua log doesn't print any errors, so I'm pretty sure the whole thing in itself technically works, but it's unable to succesfully detect the things I need detected. Here's the problematic part:

Spoiler :

-- This bit (as well!) is based on LeeS's sample code
--local tTableOfRelevantCivs = { [GameInfoTypes.CIVILIZATION_FINNS] = true,
--[GameInfoTypes.CIVILIZATION_HYP_FINLAND] = true }
local tTableOfRelevantCivs = {}
    for row in DB.Query("SELECT CivilizationType FROM Civilization_ORT_UniqueGreatPeople WHERE IsRelevant = '1'") do
                tTableOfRelevantCivs[row.CivilizationType] = true
    end

The commented-out part is what works. The part below it is the part I'm trying to get working. The database query is correct and gives correct results when executed in SQLlite spy. But for the life of me, I can't figure out what could be wrong with such a simple, short bit of code. To me, it looks like it should be working correctly.

Any ideas anyone?


EDIT:

I actually got it now. The GameInfoTypes.CIVILIZATION_X gives a ID value. This actually makes sense and I feel stupid for not realizing it before this, since Chrisy's sample code further down in the bit uses playerID. God this took me longer than it should have I guess there's two things I could do now; either join the Civilizations table and my relevant civs table on the ID part and just leave the code as it is, or alter the code to use CivilizationType instead of ID. I think the former will be easier for me, since I don't know all that much about lua, so I'll just do that. Dunno which one would be faster, but I'm guessing the latter. Oh well.

 

[orgrinrt]

The namemixes are now implemented. Sort of.

I still have to build a couple of namesets to be able to test if it's working correctly, but how I see it, it should. But I'm not all that great with lua, so probably something's a little off, but generally, I think it's there.

I decided to take the bull from the horns and do it with lua. I'll now start doing the boring spellcheck and correct all the syntax errors and then I'll just have to add in a couple of namesets to put it in action.

What a long and hard day!


EDIT: To elaborate a little:

The script now checks on the event of unit created if the unit is in the units-to-process table. If it is, it proceeds to check if there are names available to apply. It first checks if there's a nameset for CivilizationType. If there is, it picks a random one. If there isn't it checks if there's a NameMixType associated to the civ. If there is, it picks a random one. If there isn't, it then finally checks if there is a nameset available for the CultureType tag that is automatically fetched from JFD's CulDiv if it is enabled. If there is, it picks a random one.

I ran the colossus the first time and there's a bunch of errors Expected.

Let the spellchecking begin!


EDIT2:

So... As of now, it's officially finished. It now performs as it should, though it's not optimized in any way. Maybe I'll get to that when I know if it's really needed, but GP in general shouldn't be a too frequent occurance so I don't think this will slow the game down at all. I will be adding some support to some civs once I get around to add more names, but in the name of practicality I'm not gonna go and research too many names myself. If somebody's nice enough to give me some names, I'd be more than glad to add those in, and make it work for the vanilla civs as well as the custom civs I can find with a quick search. I'm thinking this isn't worth a release yet, so I'll just wait until I have a good collection of names. I'll also make it work with JFD's CulDiv eventually. The base work is done, all I need to do is to tag every nameset I get done / submitted with a CulDiv CultureType and the lua will check it if no other tag yields results.

All I now need to do is to gather a huge enormous massive bunch of names and fill in the database with them... yeah, that'll be fun.


Anyway, I'll probably continue with my more grander plans for now, and return to the civs I'm working on when I'm on travels or something like that, since they're easier to do without instant check if it works. Most of it is just adding text, really. So there will not be any civs any time soon, but I'd like to officially "lay claim" to the following ones:

• The Kvens
• The Karelians
• Ingrians
• The Pohjola
• Terra Mariana

Make of that what you will. I wouldn't like to waste anybody's time, so I think putting this out here might be of some use if someone's thinking of making one of those civs. I'm going to make a top-notch product that will be amazing .......probably.

 

[orgrinrt]

Uh.. Well, you know. I quickly realized I don't want to be manually editing every namemix into the lua code, so I had to figure a way for it to reference the database at every point possible, so that everything is handled automatically.

Well, definitely not my piece of cake, as I only know the little I do about lua.

Here's a small cut from the previously working bit to show how it was structured:

Spoiler :

local tUGP_FINSWENOR_Civs = {}
    for row in DB.Query("SELECT CivilizationID, CivilizationType FROM Civilization_ORT_UniqueGreatPeople WHERE NameMixType = 'UGP_FIN_SWE_NOR'") do
                tUGP_FINSWENOR_Civs[row.CivilizationID] = true
    end

local tUGP_FINNS_Civs = {}
    for row in DB.Query("SELECT CivilizationID, CivilizationType FROM Civilization_ORT_UniqueGreatPeople WHERE NameMixType = 'UGP_FINNS'") do
                tUGP_FINNS_Civs[row.CivilizationID] = true
    end

function GetPlayerNameMixType(civilizationID)
    if civilizationID == tUGP_FINSWENOR_Civs[CivilizationType] then
            return UGP_FIN_SWE_NOR
    end

    if civilizationID == tUGP_FINNS_Civs[CivilizationType] then
            return UGP_FINNS
    end
end

And my millionth attempt at making it completely flexible was this (and still not working correctly):

Spoiler :

function GetPlayerNameMixType(civilizationID)
    if civilizationID == tNameMixTypes[CivilizationType] then
    local pNameMixType = tActiveNameMixType[1]
    local tActiveNameMixType = {}
        for row in DB.Query("SELECT NameMixType FROM Civilization_ORT_UniqueGreatPeople WHERE CivilizationType = ".. civilizationID .."") do
            tActiveNameMixType[row.NameMixType] = true
        end
    end
    if pNameMixType ~= nil then
    return pNameMixType or tActiveNameMixType[2]
    end
end

There's probably something very obvious wrong with it, just can't figure out what. Thought I'd leave it here in case somebody stumbled upon this thread and knows the answer. Meanwhile, I'll continue trying to tinker things.


EDIT:

HA! Don't you worry. I got it. Finally. It. Is. Done. At least I think so.

Now I need to add in all the actual unique names.

While collecting those (I got a bunch earlier today, need to add those in before anything else) I'll focus on next adventures. I've got something preeetty interesting on the doings