Éa III, Sword & Sorcery: Developer's Thread

[Pazyryk]

I'm starting this thread to reduce code techno-babble in the mod's thread and the size of developer Inboxes. If you are not interested in Éa mod development, please pardon our construction noise...

Right now, ls612 is the main contributor to code, along with myself. Please post if you see something in the To Do list below that you think you might want to contribute to.

If you are an Artist, please follow the 2nd link in my sig. If you know how to add icons to mods, then look under "Art grunt work" below. If you like doing creative work, look under "Content finishing for phase 3" and also "Table grunt work". Most of the rest of the items are coding. (And no, I'm not calling you a grunt. That's just my name for work that feels like a pain in the behind to me.)

A running build can be assembled from my GitHub repositories:
https://github.com/Pazyryk/EaDLL
https://github.com/Pazyryk/Ea_III_Sword_and_Sorcery
You also need the current Ea Media Pack here.

Current working Game Manual is now online!

PHASE 3 RELEASE TO DO LIST: ONGOING WORK LIST:

New DLL functionality to remove Lua/SQL kludges: These will help me remove >100 hidden policies/buildings/etc., remove dependency on dangerous Events, or improve code in other ways.

1. NaturalWonderDiscoverHappiness in Features table; default 1 and allowing +/- values --Done
2. GameEvents.CityCanAcquirePlot() CallTestAll. Follow whoward69's example here. --Done
3. player:SetLeaderYieldBoost(yieldTypeID, percentBoost) and corresponding Get. This is a global percent modifier that needs to work for all yield types, and persisted. --Done
4. city:SetCityResidentYieldBoost(yieldTypeID, percentBoost) and corresponding Get. This is a city percent modifier that needs to work for all yield types, and persisted. --Done
5. Minor quest Lua support: GameEvents.AllowMinorQuest(iMajor, iMinor, questID) CallTestAll. This is to prevent quests that don't make any sense in the mod. A Lua method to add a specific quest will be needed too (just port whatever dll method does this).
6. New Promotions tag: "Slaver". This is needed by full civ or city state to capture civilians; otherwise, they are killed. Note: BARBARIAN_PLAYER should capture without this promotion, but ANIMAL_PLAYER should not. Don't rely on isBarbarian() because it returns true for both of these players. Note2: Any player should still be able to recapture a civilian that was originally theirs (I'm not sure if this is handled by same dll methods or different). --Never mind. Will do this differently.
7. New UnitCaptured Hook to allow slavery mechanics. --Done
8. New Builds tags: DisallowTech, PrereqPolicy, DisallowPolicy. Look for where PrereqTech is implemented and add extra checks there. --Done

Hard coded stuff in dll that probably messes up the mod (particularly AI):

1. 4 Builds (the feature-remove ones) are hardcoded by table ID (literally 14, 15, 16 and 17 ) --Fixed by re-hard-coding for mod-specific type strings, with logic to decide whether to "chop" or "slash-burn" where needed (not great but better than before)
2. Add FEATURE_BLIGHT enum and use it where FEATURE_FALLOUT is now used (blight can be removed in the mod, fallout can't) --Done
3. "UNIT_WORKER" is referenced in many places (e.g., AI for scrapping) by text string. That's a problem for us because we have 6 different kinds of worker (3 races each with worker and slave). I'll add Units tag "IsWorker" in next non-dll commit. On dll side recode all occurrences of "UNIT_WORKER" to use this with new isWorker() method (i or I, what's the pattern Firaxis?). (Skip the steam update method CvCity::IncrementUnitStatCount.) --Done

Additional DLL (or DLL plus Lua/SQL modding):

1. Rivers make internal city connections with Fishing tech (or with building enabled by tech). whoward69 is working on this so wait and copy his solution. --Done
2. (Complex; probably after release) Slaver CSs are supposed to go on slave raids occasionally. This could either be a non-war state where units are allowed to attack without war (like hidden nationality but not hidden) or a brief 2-3 turn war after which they sue for peace. It's a complex problem either way. The attack is based purely on opportunity: e.g., the CS sees an easy unguarded worker; or CS has a mass of units and can likely capture a couple nearby units (their units have promotion allowing them to capture other units). After the raid, previous relations resume (raids are just their way of saying "hello neighbor"). Some but not complete protection for friends and allies.

AI observation in autoplay: The game works now in long autoplay sessions (even up to turn 400 depending on the specific build). I've modified UI so you can see what all players' cities are building as observer. A very useful thing is watching AI players in these sessions and debugging stupidity. Base AI already has plenty of stupidity, but new mod features can add to these. Note that all AI choices in policy and tech progression and Great Person movement and actions are controlled by my own Lua code. Everything else is base AI. Important Note: To Autoplay, you need to use the mod's own Autoplay method. Select "EaMain" state and type "Autoplay(<turns>)". Or stop while running with "Autoplay(1)"

1. Why are AI workers not building some improvements???!!! Workers often just sit there in city while a valuable resource 2 plots away remains unimproved, even with no nearby threat. (I've checked obvious stuff, like ability to build and flavors.) --Mostly solved I think
2. There are two Builds for most improvements, one that removes features and one that does not. Only one is available to a particular player (depending on whether they have the Pantheism policy). The 2nd set is not being performed by AI even though it works for human player. I haven't looked yet but I'm suspicious that the AI code expects a 1-to-1 relationship between Builds and Improvements in the table (it is 1-to-1 from the point of view of player). --Solved but needs testing still
3. AI doesn't know about the mod's Living Terrain system (well, how could it?). Non-Pantheistic civs need to clear out Forest, Jungle and Marsh much more aggressively. --Working solution in next commit; needs extensive autoplay observation to see if enough or too much removal now
4. AI civs are not attacking each other as much as they should when they have a massive/overpowering military buildup. This could be due to all Leader flavors set at 5 right now, or due to global strategy foul-up (see next point) causing AI civs to never seek conquest victory.
5. The four Global Strategies are mostly (or maybe all) fouled up due to the mod. Their selection depends on the existence of certain things like space ship parts or specific victory conditions that don't exist. I need to either alter selection code or design new global strategies. Mostly this affects endgame but doesn't matter in early to mid-game.

Content finishing for phase 3: These are areas where I've done some work but there is still missing content. For each item, we should make sure the working Game Manual is filled in first. Then add to game in existing tables. Avoid new mechanisms as much as possible (until after release). We just need to make sure all civs have something.

1. Favored Techs. All civs should have ~5 - 8 of these (it gives a discount on research cost). --almost done
2. Enabled Policies. A list of ~5-7 for each civ. These are taken from a common list, but each civ has its own different list (with 1 or 2 unique or not widely shared). See branch in manual. My strategy here is to add "empty policy names" here first, then give them effects (next item). --Delayed until after release
3. Enabled Policy effects. Give actual effects to the currently "empty" policies. (Keep these easy to implement so we can finish! We can always alter to do something more complex later.) --Delayed until after release
4. One special "trait effect" for each civ. This could include UBs or UUs if we want. Avoid things that don't matter to players, like late-game UUs (unless it's balanced out by something that does matter). --Done

Table grunt work:

1. Adjust leader personalities. Since any GP can become a leader in the mod, I've put all the personality values into the EaPeople table in EaPeople.sql. (Some SQL magic automatically generates entries for Leaders and Leader subtables from data in EaPeople.) They are mostly all set to 5 right now. They need to be set to give variable but appropriate values for each individual "person". The two considerations are: a) Likely class/subclass. Each person can spawn as more than one class/subclass, but they have strong biases listed in the table. Personality as leader should match these class/subclass (e.g., Warriors should be warlike; Sages should favor peaceful research; Merchants should like to bribe CSs; etc.). The other source for inspiration is the portraits in Media Pack. You have to open the named dds portraits there to do this. It's totally subjective, but some just look cowardly, others deceptive, and so on...
2. Help texts. Write these from manual. (We'll check manual = game too.)
3. Pedia. As above. Note that some Pedia pages don't work properly for mod. This is more of a coding issue (base coding for Civilopeadia is a total mess, so hard to mod).

Art grunt work:

1. Icons! We could probably get to 80% completion (for techs & policies at least) by recycling existing icons and grabbing from the forum. This is just grindingly slow work for me, and the very last thing I will bother with (I don't mind releasing before this is done). The easiest way to see what's missing is to start a game and look around. Lots needed for techs, policies, promotions, etc.

UI work:

1. UI for "plot effects": Glyphs, Runes and Wards --Done
2. Build separate UI for Spells above the current Builds panel. There is already a tag in EaActions for this (overridden for now so spells appear in Builds panel). --Done

Complex Lua/SQL coding:

1. About 20 or so spells listed in Manual need to be added. The spell system is complex but very modular with spell-specific functions for target testing, human UI, AI valuation, etc. Follow existing examples. [Paz note: Most likely I will do all of these.] -- ~10 done, ~10 to go...

Available units to be added: (links here)

1. Giant Spider
   [*]Archdemon (Great Unclean One)
   [*]Demons (Tyranids)
   [*]Heaven's Guard (Angel Spearman)
   [*]Tree-Ent
   [*]Skeletons
   [*]Griffons
   [*]Kraken
   [*]Scorpions
   [*]Goblins (with new Goblin Encampment)
   [*]Nagas (with new Naga Encampment in desert)
   [*]Drakes - black, green, red, white
   [*]Storm Giant
   [*]Eagles --added to here

Leave it for me:

1. Reminder to self: Explosive Runes still broken for AI players --Done
2. Fix soundtrack. Was added as replacement files when that was the only way. Need to add in proper modding way now. --Done
3. Add 7 Holy Temples and 9 Unholy Temples --Done
4. Add Pantheistic Temple for each god --Done
5. Lone animals in GP layer
6. Warrior Challenge action
7. Add natural wonder yields and effects
8. finish tweaking CivAI (add Enabled Policies to EaCivPlans in EaCivAI.sql)
9. Add 3 more Tomes (Corpus Hermeticum, Necronomicon, Book of Eibon)
10. Add two Cults for new gods (desert themed and earth/wealth themed) --Done
11. Move existing buildings to balance out tech tree (add something easy if needed for blank tech)
12. Add Merchant espionage actions
13. Few more magic buildings
14. Final checks to make sure Game Manual = Game. Edit manual or update game to make it so.
15. Some techs will be disabled/hidden before release if their content isn't added yet, which is likely. These are all "branch tip" techs that lead to no other techs, including Lycanthropy, Vampirism and Song of Leviathan.

 

[Pazyryk]

@ls612,

whoward69 did CityCanAcquirePlot in his tutorial here. Just search for it on the linked page.

 

[ls612]

@ls612,

whoward69 did CityCanAcquirePlot in his tutorial here. Just search for it on the linked page.

I have seen that and mostly implemented it as written. However, testing it's efficacy is a bit hard, and I have been getting inconsistent results with both lua event handlers and a hardcoding test in the Dll, so more debugging is needed. If you want to see, uncomment the new #define, and remove the hardcoding test in CvCity::EaCanAcquirePlot (something I was using to check my work).

 

[Pazyryk]

Added my current working Game Manual in the OP.

 

[ls612]

@Pazyryk: I have written the C++ code the way Whoward's tutorial described, but I am not good with lua and can't really test if it is working. I have sent a pull request with my current code, could you please test it with lua to see if it is working or not? My tests have failed so far, but I'm thinking that is because I can't write lua, as hardcoding tests in the DLL worked.

 

[Pazyryk]

Lua's already there for you in EaMain/EaPlots.lua. Just disable the first function below (and the Events line) and enable the second. (Don't worry about "hidden player" stuff: that's all obsolete.)

Spoiler :

--DEPRECIATE: Replace with function below when CityCanAcquirePlot comes on line
local function ListenerSerialEventHexCultureChanged(hexX, hexY, iPlayer, bUnknown)	--fires for all owned plots at game init too
	--print("ListenerSerialEventHexCultureChanged ", hexX, hexY, iPlayer, bUnknown)
	if bHidden[iPlayer] then	--these only ever own city plot
		local x, y = ToGridFromHex( hexX, hexY )
		local capital = Players[iPlayer]:GetCapitalCity()
		if x ~= capital:GetX() or y ~= capital:GetY() then
			Dprint("Cancelling hidden civ plot ownership")
			local plot = GetPlotFromXY(x,y)
			plot:SetOwner(-1, -1)
		end	
	elseif bInitialized then
		Dprint(string.format("Hex ownership change at hex coordinates: %d, %d for player: %d", hexX, hexY, iPlayer))
		if iPlayer ~= -1 then
			local x, y = ToGridFromHex( hexX, hexY )
			local plot = GetPlotFromXY(x,y)
			if plot:IsWater() and gg_bPreventWaterOwnership then			--hills & mountains?
				if plot:GetResourceType(-1) == -1 then	--allow if resource (can't seem to stop spread to these when city-adjacent)
					--print("Cancelling water ownership")
					plot:SetOwner(-1, -1)
				end
				gg_bPreventWaterOwnership = true
			end
		end
	end
end
Events.SerialEventHexCultureChanged.Add(ListenerSerialEventHexCultureChanged)


-- GameEvents

--[[	In preparation for CityCanAcquirePlot
local function OnCityCanAcquirePlot(iPlayer, iCity, x, y)
	print("OnCityCanAcquirePlot ", iPlayer, iCity, x, y)
	local plot = GetPlotFromXY(x,y)
	if plot:IsWater() then return false end
	if plot:IsMountain() then return false end
	return true
end
GameEvents.CityCanAcquirePlot.Add(OnCityCanAcquirePlot)
]]

If you're wondering, GetPlotFromXY is just a localized variable holding Map.GetPlot.

Edit:
Btw, where is EA_DEBUG printing? Does it go to a text document somewhere?

 

[ls612]

Btw, where is EA_DEBUG printing? Does it go to a text document somewhere?

Right now it prints to the Output window in Visual Studio. I intend to enhance it in the future by adding another parameter which can direct it to a log file, as well as log which class the log is coming from.

 

[Pazyryk]

Might be nice to have it go to the Fire Tuner, so it can be seen in context of Lua code running at the same time. The prompt could indicate C++/Class (the prompt now tells you the "Lua state" where the print statement originated).

If we decide to do this, the easiest way I can think of is to simply create a new GameEvents hook, and use it to push text to the Lua side. Then it could be passed to Fire Tuner by print (or disabled if we want). There might be a more direct route to Fire Tuner. It's written in Lua so I'll look and see if I can determine how it gets text from the game.

 

[Gilgamesch]

Right now it prints to the Output window in Visual Studio. I intend to enhance it in the future by adding another parameter which can direct it to a log file, as well as log which class the log is coming from.

Will you share this knowlegde to other dumb peoples here, would be nice for the future to know how to link it to he database.
Thanks and good luck with Ea i will play it for sure.

 

[ls612]

Will you share this knowlegde to other dumb peoples here, would be nice for the future to know how to link it to he database.
Thanks and good luck with Ea i will play it for sure.

It is on my Github fork for the Ea DLL. Look for CvGlobals::EA_DEBUG(). Right now it is a glorified wrapper for the C-style output formatting methods, but I intend to add more functionality when I need it.

 

[Pazyryk]

@ls612, Let me know if you're doing the Lua change I mentioned in post #6, or if you want to leave that for me. (I don't mind either way, just want to reduce merge confusion.)

Never mind. I'll enable the dll change and do the Lua side.

Edit2: Fixed a pPlot/pLoopPlot error in city init code that was passing to the EaCanAcquirePlot. It all works now. I'll add to commit soon.

Edit3: Commits up for dll and Lua. Working great in autoplay...

 

[Pazyryk]

Attached dll for Krajzen (since I can't PM attachment). This is Ea dll compiled with changes that add Animal player (id 62) and nothing else.

Spoiler :

It's just the Ea dll with only these two defines set and all others disabled:

#define EA_ANIMAL_PLAYER					// Add as player 62 (define ANIMAL_PLAYER, ANIMAL_TEAM enums); all spawning logic on Lua side						
#define EA_ANIMAL_BEHAVIOR					// Don't enter enemy borders; don't pillage trade route

This can only be used as BNW dll. Add it to mod at top level (ie, not in a folder),with VFS=false. Then go into your mod properties, Actions, Add, OnGetDLLPath, SetDLLPath - CvGameCore_Ea.dll.

You can verify that it's loaded with: print(GameDefines.ANIMAL_PLAYER)

 

[whoward69]

• player:SetLeaderYieldBoost(yieldTypeID, percentBoost) and corresponding Get. This is a global percent modifier that needs to work for all yield types, and persisted.
• city:SetCityResidentYieldBoost(yieldTypeID, percentBoost) and corresponding Get. This is a city percent modifier that needs to work for all yield types, and persisted.

Which yields? Food, Production, Gold, Culture and Faith only or also the pseudo-yields of Tourism, points per GP type, Happiness (local and global) and Golden Age points as well?

As there is no unified approach, even to the 5 base yields, tracking these through the code base is a real PITA - for example, Faith seems to be "simple" once you've found the CvPlayer::GetTotalFaithPerTurn() and CvCity::GetFaithPerTurn() methods. But what about faith from goody huts, city state meetings and kills?

 

[ls612]

@ls612, Let me know if you're doing the Lua change I mentioned in post #6, or if you want to leave that for me. (I don't mind either way, just want to reduce merge confusion.)

Never mind. I'll enable the dll change and do the Lua side.

Edit2: Fixed a pPlot/pLoopPlot error in city init code that was passing to the EaCanAcquirePlot. It all works now. I'll add to commit soon.

Edit3: Commits up for dll and Lua. Working great in autoplay...

It figures what was confusing me was something so simple. Thanks for merging.

@Whoward: Yeah, that looks like a pain. I'm almost tempted to write wrapper functions for all of the yields and hide all of that mess behind a CvPlayer::GetYield(YIELDTYPE type) kind of function. Would that be a good idea?

 

[whoward69]

@Whoward: Yeah, that looks like a pain. I'm almost tempted to write wrapper functions for all of the yields and hide all of that mess behind a CvPlayer::GetYield(YIELDTYPE type) kind of function. Would that be a good idea?

You'll still have to find them all to wrap them.

The methods for culture are CvPlayer::GetTotalJONSCulturePerTurn() and CvCity::getJONSCulturePerTurn() [[what is it with Firaxis' inability to decide if it is "get" or "Get"???]] and the exception list is

• Culture from goody huts
• Culture from kills
• Culture from WB scenario (probably not relevant!)
• Culture from advanced starts
• Culture from GP special ability
• Culture from researching techs
• Culture from archaeological digs
• Culture from acquiring cities

Edit: The patterns for searching the code for faith is (Set|Change)Faith[^a-z] and that for culture is (set|change)JONSCulture[^a-z]

 

[Pazyryk]

To clarify, city:SetCityResidentYieldBoost and player:SetLeaderYieldBoost affect only the base 5 yields: Food, Production, Gold, Culture and Faith (the ones with yieldTypeID).

The main concept for both of these is to modify "per turn" yields, so ignore goody huts, kills, etc. That narrows down whoward69's list quite a bit.

For city:SetCityResidentYieldBoost, only need to modify city yields.

For player:SetLeaderYieldBoost, this also acts "locally" on city yields (additive with above but acting in all cities) and it should also affect other "per turn" civ-wide yields (from trade, beliefs, whatever else that doesn't show up as a city yield).

Yeah, I know these get complicated and that the existing methods don't all work for all 5 in many cases.


@ls612, for the sake of making progress and keeping focus on the mod (and ease of dealing with updates later), I wouldn't spend too much time trying to repair badly coded stuff. Where there's ad hoc code, it's usually fastest to add your own ad hoc solutions. (I'm not saying that's always the best solution. Just saying that you need to pick your battles with some care.)

 

[whoward69]

To clarify, city:SetCityResidentYieldBoost and player:SetLeaderYieldBoost affect only the base 5 yields: Food, Production, Gold, Culture and Faith (the ones with yieldTypeID).

The main concept for both of these is to modify "per turn" yields, so ignore goody huts, kills, etc.

For city:SetCityResidentYieldBoost, only need to modify city yields.

For player:SetLeaderYieldBoost, this also acts "locally" on city yields (additive with above but acting in all cities) and it should also affect other "per turn" civ-wide yields (trade, beliefs, whatever else).

Yeah, I know these get complicated and that the existing methods don't all work for all 5 in many cases.

That makes finding the places to add the multipliers much easier

 

[whoward69]

The first method is where the LeaderYieldBoost multiplier needs to go, the second for the SetCityResidentYieldBoost multiplier

YIELD_FAITH
  CvPlayer::GetTotalFaithPerTurn()
  CvCity::GetFaithPerTurn()

YIELD_CULTURE
  CvPlayer::GetTotalJONSCulturePerTurn()
  CvCity::getJONSCulturePerTurn()

YIELD_GOLD
  CvPlayer::calculateGoldRateTimes100
  CvCity::getYieldRateTimes100(YIELD_GOLD)
  
YIELD_SCIENCE
  CvPlayer::GetScienceTimes100()
  CvCity::getYieldRateTimes100(YIELD_SCIENCE)
  
YIELD_FOOD
  CvCity::foodDifferenceTimes100
  CvCity::getYieldRateTimes100(YIELD_FOOD)
  
YIELD_PRODUCTION
  CvCity::getProductionDifferenceTimes100 (and possibly CvCity::getProductionDifference)
  CvCity::GetBaseYieldRate(YIELD_PRODUCTION)

Note: Due to inconsistencies in usage you can't just multiply every return from CvCity::getYieldRateTimes100

HTH

W

 

[ls612]

The first method is where the LeaderYieldBoost multiplier needs to go, the second for the SetCityResidentYieldBoost multiplier

YIELD_FAITH
  CvPlayer::GetTotalFaithPerTurn()
  CvCity::GetFaithPerTurn()

YIELD_CULTURE
  CvPlayer::GetTotalJONSCulturePerTurn()
  CvCity::getJONSCulturePerTurn()

YIELD_GOLD
  CvPlayer::calculateGoldRateTimes100
  CvCity::getYieldRateTimes100(YIELD_GOLD)
  
YIELD_SCIENCE
  CvPlayer::GetScienceTimes100()
  CvCity::getYieldRateTimes100(YIELD_SCIENCE)
  
YIELD_FOOD
  CvCity::foodDifferenceTimes100
  CvCity::getYieldRateTimes100(YIELD_FOOD)
  
YIELD_PRODUCTION
  CvCity::getProductionDifferenceTimes100 (and possibly CvCity::getProductionDifference)
  CvCity::GetBaseYieldRate(YIELD_PRODUCTION)

Note: Due to inconsistencies in usage you can't just multiply every return from CvCity::getYieldRateTimes100

HTH

W

Thank you very much for that, it is making implementation of these two methods much easier.

 

[ls612]

@Pazyryk: Two questions

1. Why do you want the getter for the city and civ-wide yield percent modifiers to be sent to lua? I am already processing the effects of that method in the DLL.

2. Do you have sample Lua code I could use to test my addition of these methods?