Barbarian Great Library bug: investigation

tupi

Warlord
Joined
Jun 25, 2011
Messages
214
Location
Russia
Probably many of you know that when the Great Library is destroyed (or captured by Barbarians) game goes crazy in various ways. It can crash randomly, it writes some (it seems) random data at (it seems) some random places.

Why?

At the first glance, we can see that game checks only that city of Great Library is not -1, i.e. that this wonder was built once in some city. It's ok.

But there's no check that the wonder was destroyed (city is 128 in this case). Nor here, nor in the function "check that wonder's in city of our civ and it is not obsoleted". This is already bad, because "128th city" is beyond city array and at the place of "city civ" of "city number 128" there can be some abnormal data (not only some random civ from 0 to 7, but also any number more than 7 (game reads 1 byte here) - and then game will write "tech" data in the wrong place in memory).

But what is in memory at this address? Actually, "land mass" of generated world (at least in v05), which can be 0, 1 or 2 (normally 1, if you did not customize your world). Little-endian way: 0000, 0100, 0200. "city civ" of "city number 128" is the second byte of this variable, which is always 0 (barbarians). So not a big deal!

So problem is not here...

Problem is in the other place, and this problem is severe. When the game calls "does this civ know this advance?" function, it always returns 0 for barbarians. And here lies the source of this nasty bug. You see, when some civ gets a new tech and conditions for Great Library "steal" are ok, the game recursively calls "get new tech" again, but for civ with the Great Library this time... But because barbarians "failed to learn anything" (actually they DID learn, and this is only place where this happened, I think - but function forcibly returns 0 for them anyway)... Because of that, the "get new tech" function is called again, because the game thinks Barbarians still do not have this tech - this time, when "get new tech" function is called for barbarians themselves, when they should get a tech from GL. And then it's recursively called again and again, to infinity - or, actually, until stack overflows into main, heap memory and game either crashes or continues its flow somehow because of... reasons. Result can be very unpredictable and disastrous.

In the case when you play as barbarians, with destroyed or captured Great Library you will get endless loop (recursion, actually): message "Barbarians acquired tech from the GL!" -> Sid discovery screen/newspaper -> Civilopedia tech reference screen etc.. Stack overflow error happens at about 25-27 time with message:
run-time error R6000
- stack overflow
and game crashes.

Maybe sometimes game crashes immediately with AI, but in my tests after about 90-100 times of calling "get new tech" function (I added embassy to Barbarians to count) game continues with corrupted main memory (in my case: until Egyptians data, Germans and data before are not corrupted), with some random tech in my (English) list and some random diplomacy statuses... I've got some random diplomacy flags, but embassy with Barbarians is still here - so this number (90-100) of recursive calls is correct...

I don't know why game does not crash with stack overflow error in this case (or in some cases) and instead it continues to write stack data into main memory.

And finally, how can we fix that? Probably the simplest solution is to allow "does this civ know this advance?" return actual data for barbarians. Because they (as far as I understand) never can get any new tech anyway, so this check is redundant. The only exceptions are: well, barbarian Great Library - and the case when human Attila obtains some tech from diplomacy talks with AI civ (if you reject peace and AI then offers something; exchange with you as Barbarians is not possible, because, again, game always think that you don't have any tech even when you actually do have some, so nothing to exchange).
But this way, it will be possible to actually steal some tech from Barbs after they got Great Library...

Without changing number of bytes, we can (probably?) change "great library city is not -1" to "first bit of the right byte of library city number is 0". This way, we exclude both -1 (0xFFFF) and 128 (0x0080) cities, so at least destroyed GL will not cause the bug. Cases when Barbarians actually capture the GL city are much, much rarer. I should to see how it should look in machine code, and then we can fix it.
Original bytes here are:
1ade:2234 a1 b2 6e MOV AX ,[GREAT_LIB_CITY ] 1ade:2237 89 46 f8 MOV word ptr [BP + loc_gl_city ],AX 1ade:223a 3d ff ff CMP AX ,-1 1ade:223d 74 70 JZ LAB_func_return

If something like TEST AX, 0x80 JNZ LAB_func_return will use same number of bytes (or less), then we can at least partially fix this bug.
We should probably change it to:
a8 80 TEST AL,0x80 90 NOP ;free space... 75 70 JNZ LAB_func_return

This fix is not neat and kinda ugly, because it's clearly looking like a hack (for me, good hex edit is the edit which style looks like it's a result of source code editing). And there's still a problem when barbarians capture GL city... Anyway, should test this fix now...
 
Last edited:

tupi

Warlord
Joined
Jun 25, 2011
Messages
214
Location
Russia
So, first fix (v05):
search for
3d ff ff 74 70
in civ.exe (address: 0xd006)

and change it to:
a8 80 90 75 70
EDIT: I made a mistake: not a8 f0 90 75 70, but a8 80 90 75 70 - of course, we should check only most significant bit and not a whole high nibble.

after that, barbarians will not get a tech from destroyed Great Library (in other versions of the game civ in this case can be not barbarians and not even within civ number range 0..7, so more correct description is "destroyed Great Library will not longer work")

Second (undesired! full consequences are unknown!) fix:
search for (v05)
83 7e 06 00 74 f6
in civ.exe (address: 0xd09d)

and replace it with NOPs:
90 90 90 90 90 90
OR change civ number here (00) to some value beyond 0..7 range, for example, 8:
83 7e 06 08 74 f6

After this, barbarians will became kinda more like full-fledged civ: if they do have some tech, you (as Attila) can exchange it with other civs, you (as Attila) can change your government if you know a corresponding tech, default production in your new-founded barbarian city can be Musketeers or Riflemen, it will be possible for other civs to steal some tech from barbarians (with "civ developed tech" message instead if "civ acquired tech from civ2" message) etc.. The only normal way for AI barbarians to get some tech still will be to get it from the Great Library. I also think that as a result of this second hack, if AI barbarians obtain some tech from GL that allows to build some wonder, it will even be possible for them to build that wonder.

If you want to use this second hack, use it along with the first one. If you will use only the second hack, barbarians (barbarians at least in v05, but in general case, some undefined civ from 0..255 range) will get civ. advances from destroyed GL. But maybe you want such behavior, in this case use 2nd hack without the first one.
 
Last edited:

divec

Chieftain
Joined
May 17, 2019
Messages
16
Amazing work! One of the most interesting unintended consequences of the game rules, IMHO.

This fix in not neat and kinda ugly, because it's clearly looking like a hack (for me, good hex edit is the edit which style looks like it's a result of source code editing). And there's still a problem when barbarians capture GL city... Anyway, should test this fix now..
Your standards are way too high 😜 Patching three-decade old compiled code with a working fix the exact same byte length is a minor miracle in itself!
 
Top Bottom