Gedemon's Civilization, development thread

[Gedemon]

Hey @Gedemon I think turn times suddenly got a lot worse after the latest update. Would you mind posting, either here or on the github, something that explains which scripts run when? (centralized in one place to make the script files easier to fathom). I might be able to help you optimize which parts can and can't get cached

Most runs at the beginning of each player's turns (called from GCO_PlayerScript.lua), and the map update is done at the beginning of a new turn. It's a bit of a mess I'm afraid, I wasn't expecting to use Lua only for so long and yes there is some more caching that could be done.

I've also noted that I get warnings from my timers even at the beginning of a game on a large map, which wasn't happening so soon before. The lua.log shows those for function that takes too much time to process like here:

Spoiler :

So there is this in GCO_PlayerScript.lua, called before each player's turn

DoPlayerTurn( playerID )

which call, in GCO_CityScript.lua

DoCitiesTurn( playerID )

and in GCO_UnitScript.lua

DoUnitsTurn( playerID )

Then there is this in GCO_GameScript.lua, at the beginning of each game turn

InitializeNewTurn()

And finally this in GCO_PlotScript.lua, at the beginning of each game turn. That's the one causing the few seconds freeze just before the beginning of the human's player turn, pondering to move it in a coroutine, but not sure how to handle it for MP sync as I'd like some actions (like moving an unit) to affect the culture map.

OnNewTurn()

 

[CalebR_300]

Okay, been doing some playtesting... gone through two games, and I've got a problem in the second.

First game seemed alright, first city in floodplains, developed it, made a second city, everything seemed fine. Made a mistake with a builder (thought I had 2 charges, but only had one), so tried to boot an autosave (GCO version), but the save was seemingly corrupt. Didn't make a manual save... so RIP first game.

Second game is where I had an issue. I built my first city (no floodplains), slowly developed the city, got all the basic material producers and food buildings, but my city look a very long time to grow, so much so manpower was a major issue in the early game (on my third unit). After a while, the population finally started to increase (up to 5 pop, around 100,000 people), manpower was restored, and everything seemed to be on the up and up. But then my population started going backwards big time for seemingly no reason, and my urban workforce dried up, manufacturing dropped big time, and now my city is back to one pop (65,000). Growth is still dropping. Birth/Death rate is positive.

I've tried to puzzle out my food situation. If I'm reading this right, my people eat 5.86 food/turn.

But I'm having a hard time trying to determine how much is coming in... If I'm reading this table right, is it 42 (22 meat, 3 grain, 17 misc food)?

Either way, I'm rather confused as to why my population is dropping like a brick, although I suspect it's the immigration system. In this stunted state, other civs are swarming my single city, and I think I'm due to live life as a city state.
By the way, hovering over my city tile shows that 100% of the population is independent... that doesn't seem right.

 

[Gedemon]

Okay, been doing some playtesting... gone through two games, and I've got a problem in the second.

First game seemed alright, first city in floodplains, developed it, made a second city, everything seemed fine. Made a mistake with a builder (thought I had 2 charges, but only had one), so tried to boot an autosave (GCO version), but the save was seemingly corrupt. Didn't make a manual save... so RIP first game.

Second game is where I had an issue. I built my first city (no floodplains), slowly developed the city, got all the basic material producers and food buildings, but my city look a very long time to grow, so much so manpower was a major issue in the early game (on my third unit). After a while, the population finally started to increase (up to 5 pop, around 100,000 people), manpower was restored, and everything seemed to be on the up and up. But then my population started going backwards big time for seemingly no reason, and my urban workforce dried up, manufacturing dropped big time, and now my city is back to one pop (65,000). Growth is still dropping. Birth/Death rate is positive.

I've tried to puzzle out my food situation. If I'm reading this right, my people eat 5.86 food/turn.
View attachment 554416


But I'm having a hard time trying to determine how much is coming in... If I'm reading this table right, is it 42 (22 meat, 3 grain, 17 misc food)?
View attachment 554417

Either way, I'm rather confused as to why my population is dropping like a brick, although I suspect it's the immigration system. In this stunted state, other civs are swarming my single city, and I think I'm due to live life as a city state.
By the way, hovering over my city tile shows that 100% of the population is independent... that doesn't seem right.
View attachment 554419

Looks like you downloaded the version I posted before the hotfix, this was caused by a divide by 0.

Both save corruption and full migration of your culture group were caused by that, try to update.

 

[Gedemon]

well, just got a save corruption too, looking at it.

 

[CalebR_300]

Can confirm I'm using the hot-patched version of the game, and can also confirm that manual saves are also corrupted.
Oof. I was actually having some success boosting my population by annexing my neighbours (population started rising once I annexed a city on floodplains with some decent farm tiles) .

 

[Gedemon]

Doing a review of divide operations in city and plots scripts now.

 

[Gedemon]

Update

- restore the Error message on screen, please post your Lua.log when you get one
- add a check to City and Plots script to display an Error message on divide by 0
- add a new check to detect corrupted tables on saving (and loading) and display a message on screen, auto-saves from the turn before the error message should be safe to use, once the reason of the corruption is patched of course
- change the rounding methods
- remove most output to the lua.log, keep only turn counter, warnings and errors
- add China and Korea City Names
- add Korea Colors
- disable Harbor's Projects

 

[DemonEmperor]

So I added my own debugging prints:

GCO_PlotScript: Attempt to deserialize:
return { ["2129"] = {  }, ["2262"] = { ["99"] = { ["2177"] = { ["Total"] = 75, ["Migrants"] = 1 }, ["2261"] = { ["Total"] = -192, ["Migrants"] = -3 }, ["2178"] = { ["Total"] = 875, ["Migrants"] = 2 } }, ["101"] = { ["2177"] = { ["Total"] = 77, ["Migrants"] = 1 }, ["2261"] = { ["Total"] = -198, ["Migrants"] = -3 }, ["2178"] = { ["Total"] = 879, ["Migrants"] = 2 } }, ["100"] = { ["2177"] = { ["Total"] = 76, ["Migrants"] = 1 }, ["2261"] = { ["Total"] = -195, ["Migrants"] = -3 }, ["2178"] = { ["Total"] = 877, ["Migrants"] = 2 } } }, ["3131"] = { ["99"] = { ["3047"] = { ["Total"] = 948, ["Migrants"] = -1 }, ["3048"] = { ["Total"] = 29, ["Migrants"] = 4 }, ["3215"] = { ["Total"] = -889, ["Migrants"] = 1 }, ["3132"] = { ["Total"] = -81, ["Migrants"] = 5 }, ["3216"] = { ["Total"] = -1427, ["Migrants"] = -1 } }, ["101"] = { ["3047"] = { ["Total"] = 933, ["Migrants"] = -8 }, ["3048"] = { ["Total"] = 44, ["Migrants"] = 8 }, ["3215"] = { ["Total"] = -887, ["Migrants"] = 1 }, ["3132"] = { ["Total"] = -73, ["Migrants"] = 2 }, ["3216"] = { ["Total"] = -1428, ["Migrants"] = -1 } }, ["100"] = { ["3047"] = { ["Total"] = 941, ["Migrants"] = -7 }, ["3048"] = { ["Total"] = 36, ["Migrants"] = 7 }, ["3215"] = { ["Total"] = -888, ["Migrants"] = 1 }, ["3132"] = { ["Total"] = -75, ["Migrants"] = 6 }, ["3216"] = { ["Total"] = -1427, ["Migrants"] = 0 } } }, ["734"] = { ["99"] = { ["735"] = { ["Total"] = -1033, ["Migrants"] = 0 }, ["817"] = { ["Total"] = -1277, ["Migrants"] = 1 }, ["818"] = { ["Total"] = -1226, ["Migrants"] = 4 }, ["650"] = { ["Total"] = -1266, ["Migrants"] = 1 } }, ["101"] = { ["735"] = { ["Total"] = -1033, ["Migrants"] = 0 }, ["817"] = { ["Total"] = -1276, ["Migrants"] = 0 }, ["818"] = { ["Total"] = -1218, ["Migrants"] = 4 }, ["650"] = { ["Total"] = -1266, ["Migrants"] = 0 } }, ["100"] = { ["735"] = { ["Total"] = -1033, ["Migrants"] = 0 }, ["817"] = { ["Total"] = -1276, ["Migrants"] = 1 }, ["818"] = { ["Total"] = -1222, ["Migrants"] = 4 }, ["650"] = { ["Total"] = -1266, ["Migrants"
GCO_PlotScript: It says it's successful
GCO_PlotScript: function: 0000000112438E40
GCO_PlotScript: nil
Runtime Error: [string "return { ["2129"] = {  }, ["2262"] = { ["99"] = { ["2177"] = { ["Total"] = 75, ["Migrants"] = 1 }, ["2261"] = { ["Total"] = -192, ["Migrants"] = -3 }, ["2178"] = { ["Total"] = 875, ["Migrants"] = 2 } }, ["101"] = { ["2177"] = { ["Total"] = 77, ["Migrants"] = 1 }, ["2261"] = { ["Total"] = -198, ["Migrants"] = -3 }, ["2178"] = { ["Total"] = 879, ["Migrants"] = 2 } }, ["100"] = { ["2177"] = { ["Total"] = 76, ["Migrants"] = 1 }, ["2261"] = { ["Total"] = -195, ["Migrants"] = -3 }, ["2178"] = { [..."]:1function expected instead of nil
stack traceback:
    [string "return { ["2129"] = {  }, ["2262"] = { ["99"] = { ["2177"] = { ["Total"] = 75, ["Migrants"] = 1 }, ["2261"] = { ["Total"] = -192, ["Migrants"] = -3 }, ["2178"] = { ["Total"] = 875, ["Migrants"] = 2 } }, ["101"] = { ["2177"] = { ["Total"] = 77, ["Migrants"] = 1 }, ["2261"] = { ["Total"] = -198, ["Migrants"] = -3 }, ["2178"] = { ["Total"] = 879, ["Migrants"] = 2 } }, ["100"] = { ["2177"] = { ["Total"] = 76, ["Migrants"] = 1 }, ["2261"] = { ["Total"] = -195, ["Migrants"] = -3 }, ["2178"] = { [..."]1: in function '(main chunk)'

fsr, that one table isn't deserializing correctly. using loadstring returns a function, but trying to call that function fails, because lua thinks it's a nil? weird

it *might* be because that specific table is > 1 MB large, but that sounds unlikely

 

[Gedemon]

When it happens it's caused by trying to save a nand/inf value after a divide by 0

The last update has solved a few, and should detect when it happens on save (by trying to deserialize immediately the saved string)

edit: but yes, it's weird, I was looking at the wrong part of the code for a while

 

[DemonEmperor]

@Gedemon So the wonders not appearing issue is still here for me. Applying a similar fix as last time works.

current if-statement around line 2046 in file UI/ProductionPanel.lua

if (bCanShow or bCanStart) and not (row.IsWonder and (not bCanStart)) then

bCanStart is still undefined, so that simplifies to:

(bCanShow or nil) and not (row.IsWonder and (not nil))
= bCanShow and not (row.IsWonder and true)
= bCanShow and not row.IsWonder

recommend replacing it with:

if bCanShow and not isDisabled then

 

[CalebR_300]

Booted a new game with the latest patch. Seems to be working. Saves are loading, and so far nothing untoward has occurred.
Had an error message pop up that 'player is nil in GetPlayer for playerID# -1'... something glitching with a barbarian I suppose. Dropping the lua.log just in case, but it's probably nothing.

 

[Gedemon]

@Gedemon So the wonders not appearing issue is still here for me. Applying a similar fix as last time works.

current if-statement around line 2046 in file UI/ProductionPanel.lua

if (bCanShow or bCanStart) and not (row.IsWonder and (not bCanStart)) then

bCanStart is still undefined, so that simplifies to:

(bCanShow or nil) and not (row.IsWonder and (not nil))
= bCanShow and not (row.IsWonder and true)
= bCanShow and not row.IsWonder

recommend replacing it with:

if bCanShow and not isDisabled then

Thanks, either a typo on my side or they changed bCanStart to isCanStart in a patch, should be this:

if (bCanShow or isCanStart) and not (row.IsWonder and (not isCanStart)) then

We need to see disabled buildings (well not all those we see ATM, we need to decide a little something for Religious buildings until a real Religion overhaul), at least the "Recruitment Centre" so the user can see why it's disabled, same for those disabled because of a lack of resources.

Wonders should also be shown.

Booted a new game with the latest patch. Seems to be working. Saves are loading, and so far nothing untoward has occurred.
Had an error message pop up that 'player is nil in GetPlayer for playerID# -1'... something glitching with a barbarian I suppose. Dropping the lua.log just in case, but it's probably nothing.

Thanks, can ignore that one, someone pillaged something outside any player's territory, will add a check.

 

[DemonEmperor]

@Gedemon cool. will note that that logically simplifies to:

if (bCanShow and not row.IsWonder) or isCanStart then

but that's not a major concern

More important is that "ERROR : __orderedIndex already exists on orderedNext first call" is getting thrown, with accompanying "ERROR : CITIES TURN UNFINISHED AT TURN # 36 FOR PLAYER #5 ( UZBEKISTAN )" for the AI.
attached lua log below

Edit: oops, attached it twice

 

[Gedemon]

and small update

UI : restore Wonders on production Panel, thanks to DemonEmperor
balance: bigger housing effect on higher class population Birthrate
balance: smoother Social Stratification
bug fix: partial lack of luxuries was applied as a bonus for Upper Class

City should not grow as much as before when housing is a limiting factor, let me now.

 

[Gedemon]

@Gedemon cool. will note that that logically simplifies to:

if (bCanShow and not row.IsWonder) or isCanStart then

but that's not a major concern

More important is that "ERROR : __orderedIndex already exists on orderedNext first call" is getting thrown, with accompanying "ERROR : CITIES TURN UNFINISHED AT TURN # 36 FOR PLAYER #5 ( UZBEKISTAN )" for the AI.
attached lua log below

Edit: oops, attached it twice

Thanks I'll have a closer look tomorrow, time to sleep now.

 

[CalebR_300]

Blooming heck, another update... I haven't even finished play-testing the last build!
No matter, here's some more feedback from the previous build (only 137 turns in, mind you).

• Governmental stability system seems to be working. No too difficult to maintain 100% empire stability provided you make the effort to place forums down. Though to be fair, I haven't played as expansionist this time round.
• Housing is a lot more important to growth now, so that seems to be working as intended; although I'll probably need a longer play-through to make sure, as the issues of overpopulation usually occurred later in the game.
• Urban employment is now a factor that needs to be considered when adding new production type buildings. I like it, as you need to balance your industry to your population - or suffer the consequence of partly functional buildings.
• No more sudden population drops, so the immigration system seems to be doing it's thing now.

In short, the recent fixes seem to have worked. Now for some more general feedback:

• In all the games I've played with this mod, the AI doesn't seem to have built up much of a military. Compounded with walls not providing 'city attacks', it's pretty easy to siege up cities - food and troop health (sickness, not HP) being more of a threat than the defending army most of the time.
• Similar to above, the AI doesn't seem to deal with barbarians too well - a barb camp nearby to a civ can keep it stunted for an extended period. The past 50 turns of this game I've been watching my northern neighbours get overrun, with only one warrior levy showing up long enough to get chased around and destroyed. This could be a vanilla thing though.
• More of a general thought, most resource outputs scale over time (wood, sheep, etc), but generic food output does not (apart from tech bonuses). Generally speaking, a standard farm will peak around 45 food by Replaceable parts (assuming being on floodplains, next to a river, and surrounded by other farms), whereas I have a rice farm generating that much food (37 rice + 8 base) in the Classical era. I mean bonus resources are bonuses, but when clay/stone outputs on generic farms can out-scale the food output, something seems off.
• With wonders back on the menu (cheers DemonEmperor! ), I wonder if the wonder effects might need to be adjusted to better fit in with the mod. I'm thinking something that ties into the 'inspiration' system for techs. But that's one of those 'down the line' things.
• In regards to the health (sickness) system, units seem to suffer a fair bit of penalty owing to adjacent tiles (jungle/marsh, etc) - I think there should be heavier weighting to the current tile they're on (unless maybe, the food/water they require is on that adjacent tile). Once again, a little thing.

And that's about all I can think of for now. Time to play-test yet another game, to see what I can see. Cheers again for all the hard work, I want to see this project to the end.

 

[CalebR_300]

Alright, playing the latest build. After an epic siege (AI was actually doing a decent job defending this time), I got the dreaded divide by zero error upon conquering the city.
Posting the lua log. Apart from that, all was going well.

Edit: Maybe a false alarm - save game loaded fine after error, and after a few turns everything is still running fine.

Edit 2: Just found some sugar cane growing on a mountain... this normal?

Edit 3: The Great Pyramids of Madrid, under the gaze of the sugarcane mountain. Feels good to have Wonders.

 

[Gedemon]

@Gedemon cool. will note that that logically simplifies to:

if (bCanShow and not row.IsWonder) or isCanStart then

but that's not a major concern

More important is that "ERROR : __orderedIndex already exists on orderedNext first call" is getting thrown, with accompanying "ERROR : CITIES TURN UNFINISHED AT TURN # 36 FOR PLAYER #5 ( UZBEKISTAN )" for the AI.
attached lua log below

Edit: oops, attached it twice

not sure what's going on, I'll add more output to the orderedNext debug log, and a way to monitor a specific function.

I output an error when _orderedIndex already exists because it means that the previous call to pairs (replaced by orderedPairs in scripts for MP sync) didn't exit cleanly and a real error was not caught (or a function used a break in a pairs call without manually removing the __orderedIndex)

So I'll add a log from the previous call in the output.

and thanks for the simplification, added.

 

[Gedemon]

@CalebR_300 a few quick answers, and thanks for the feedback, help to understand what need to change in the UI.

• tile size and production:

works a bit like the correspondance between "size" and real "population" values for cities, there is also a max value for employment (based on the number and types of resources on the tile with a multiplier applied if the tile is improved for a resource) and a production factor.

population growth with time, following migration (working a plot accelerate migration from a city to that plot), and so the output rise.

mechanism is the same for all tiles, but some have a higher base value.

It will need tweaking of course, and the UI will change. I'm pondering a small population icon on each tile to split the information from the main plot tool tip, there is already a lot in it.

• builder using material:

IIRC I can't prevent the worker to be killed on using the last charge. Could be implemented anyway, afraid of the (even worst) imbalance for the AI.

• Food output:

We should be able to add bonus resource mid-game at some point in development, the base output will just be a bonus then, but not the main source of food

• Wonders:

We'll do a pass to adapt them to the mod's mechanism, some will be stacked back in cities, but when it makes sense, (yes, like Pyramids) they will stay on the map.

More generic things like Policies, Governments or Buildings take priority.

 

[Gedemon]

Forgot one, it's some of the military techs (bottom row) that unlock new military organisation level (more soldiers, strength bonus, more reserve, longer supply line, faster "healing" from reserve to front line)

There is an "Elite Military" policy (unlock at Military Tradition, which also unlock the first military organisation level) that keep them a bit smaller in size to help with you with the support cost.