No I meant NOPing 5 bytes
. It didn't mess up anything just did some not pretty animations. I think what I would like to achieve is making it draw every 2nd, 3rd or maybe even 4th frame of animation. Now it just looks ugly. I should look somewhat deeper into it..
Ok so did you NOP them in memory while CIV was running? Because even if you NOP the 5 bytes in the cold file CIV.EXE before running, for example you overwrite
9A 12 04 E8 1E with
90 90 90 90 90, the position of the last two bytes is
still recorded in the relocation table... And when loading CIV.EXE, it would typically become
90 90 90 32 92 (9090 + 1A2 = 9232), which stands for "db 32; xchg ax, dx;"...
Anyway, if it works, good for you
Side remark/observation - in Warlords 2 I patched some code in a way I presumed would crash the game immediately. I needed more bytes to code mov instruction with immediate value and not register:
mov [addr],al => mov [addr], imVal
So I "swallowed" pop or push instruction before. I thought it would break everything right away (messing up stack should have disastrous and immediate results), but it didn't; I don't exactly know why.
Can't tell you why either, without more details
At this level, every bit counts, and some changes may have little to no impact, while others will just break everything... Detailed analysis is usually inevitable.
Yes, BPLM command in DOSBOX gives proper results - breakpoint on write. But if my cheat engine points to some address in code it seems that I have to have correct (at runtime) number of CS and IP to set breakpoint properly. Although it is not a big problem because I do not search for code in Cheat Engine, only use it for "patching" code "on demand". But if I forget what the CS:IP address was, I cannot use any address which points to the same area in code - it must be the address.
Ok I see, I didn't use CheatEngine yet for CIV, so I didn't know its limitations.
Good to know. Where do you take your knowledge from? I must say I started my studies in the end of the nineties, so era of DOS and segment addressing were pretty much gone then.
In fact, the
short story is "me + Internet
research (+ thank you Gowron)"...
The
long story (be prepared, cause it's going completely overboard) is that I have always been more or less interested in programming: I started to play Civ as a kid (following my older brother), and used to fiddle around with it using hex editors and QBasic, for fun. I studied computer science in the early 2000's, and went for an IT job after that, but a little more on the business side of things. Still I always kept a an interest for software/hardware technology.
So I had some of that background, in late 2012, while I was losing some time playing a Civ game as Babylonian Prince, reaching uncontested dominance around 1750... I had been playing Civ once in a while since the early 90's, and used to abuse the settler hack for doing massive land improvements. Once again, I wanted to transform all land, but once again I grew tired of spending long turns clicking around tens of settlers, with no real challenge other than patience, just for the sake of improving all land. So I just started looking for ways to hack my savegame and do it more quickly... and that's how it all started.
From that point on, I spent a lot of time
researching: Dack's
TerraForm seemed to do the job, but its free version limitations and registration policy just seemed like a "hassle" at the time - I just wanted to modify my MAP after all, right here, right now, and not after waiting 5 days and posting 5 "meaningful" comments to the civfanatics forums (incidentaly, I got a TerraForm license much later, when the 5 days and 5 posts were already achieved for a long time).
So I tried to look at the binary MAP by myself, and discovered that a single change in the land (forest turned to plains) resulted in an almost completely different MAP binary - which sounded to me as "it's gonna be hard".
After some
research through the forums and the Internet, I found a seemingly inocuous comment by Gowron mentioning a fan-made (namely Joel McIntyre - I could never find a way to contact him) file viewer for MicroProse's Darklands "PIC" files, that was actually able to open Civ's
MAP files as pictures. For some reason, this file viewer could not open Civ's
PIC files though, only MAP (I discovered later on that it did not support Civ's EGA/VGA mixed palette format). Also it was just a viewer, not an editor or converter. But quite fortunately, the viewer came along with some C source code for parsing the PIC/MAP image format, together with some very valuable documentation about the PIC format.
After being unsuccessful in porting the C code to Java (which I was much more comfortable with), I pushed the
research further trying to clarify what Joel McIntyre just hinted at in his document: the compression format used in PIC files. A lot of
research, pen-and-paper, and programming trial and error lead me to strongly suspecting that it was using a variant of the [wiki]LZW[/wiki] algorithm (as suggested by McIntyre himself), together with the easy [wiki]RLE[/wiki].
I further
researched LZW and polished my experimental implementations until it reached something rather exciting - this is when I started this thread:
MAP, PIC and PAL formats figured out ! (almost).
At this point I could tweak my MAPs, but my expectations were surpassed by the ability to completely extract all CIV art from PIC files... My brain started buzzing into many directions: why only tweak MAPs and not SVEs, too? What about modding CIV art? Or actually modding Civ completely? Or actually make a tool to mod Civ completely?
I pursued those endeavours by creating small tools to extract PICs (and to some extent create PICs) as well as to make basic editions to MAPs and SVEs. Those were later merged as what is now
JCivED.
While progressing on JCivED, I felt urges to push it to the end: I wanted a complete understanding of the SVE and MAP contents. Also, I wanted completely understanding of all Civ resources files, including fonts, music and sounds. The fonts (in FONTS.CV) were relatively easy to hack out, although it was quite an interesting experiment in pure pattern analysis and reverse-engineering - it obviously helped that FONTS.CV is not compressed...
But for the SVE and MAP contents, it wasn't that easy: while working through major portions of the SVE format (followed up in this thread, together with
massive contributions from Gowron, Dack, Renergy, Whelkman, and others:
SVE file format) even with a lot of tweak-and-play sessions, some parts remained very obscure. In particular, the
land value algorithm was seemingly impossible to figure out, in spite of considerable effort by Dack, simonnomis and others (I myself wrote pieces code that would make batch screenshot recognition and statistical analysis on terrain / land value... but in vain).
I got intrigued by the extent of Gowron's contribution, and how he could figure out things by working directly with Civ's disassembly in IDA. I had given IDA a try, but always considered assembly as something beyond my reach, so I didn't even consider mastering it in any way. However, I must - as always - credit
Gowron for giving my the crucial kickstart that changed everything: the "BreakPeaceDialog" from part 1 of the tutorial actually comes from his first message to me for getting familiar with IDA!
Soon enough, I discovered the existence of overlays: Civ disassembly in IDA is full of "INT 3Fh", with which IDA has a lot of trouble to deal. While
researching the Internet for the logic of "INT 3Fh" and overlays, I realized the overlays were right inside CIV.EXE! As well,
research proved fruitful in understanding MS-DOS interrupts, and how Civ (as most DOS programs) would configure its own interrupt handlers upon start-up - not only for 3Fh, by the way... Basically, the overlay loading code is right there within CIV.EXE, so all that remained was to understand it.
Days of
research later (...) I realized it was possible to add more code to an IDA project, and could import manually all overlays into IDA. At the same time I was playing with IDA scripting, and managed to write a script that would automatically replace all the troublesome "INT 3Fh" with equivalent "far calls" into imported overlays.
Not long after, I found out the "Load/Save SVE" sub-routine that Gowron had mentioned, inside an overaly... This proved a dramatic progress for mapping the CIV data in the disassembly, and general understanding of CIV disassembly altogether... This also made it possible to finally decipher the SVE format (well not
really, since some bits and bytes within some data structures remain unclear...)
That's it for my knowledge on
overlays Of course there's more to tell down the road, how DOSBox Debug mode was crucial for understanding
land value calculation for example... As of right now, my further endeavours brought me to starting writing an embryo of IDA-inspired Java-based DOS disassembler (based on distorm3) and decompiler, just to save myself from the "hassle" of manually going through all CIV code - especially the bigger sub-routines... And this leads me to
researching things such as "static code analysis", including things in graph algebra, re-compiling DOS-era C decompiler source code, etc. No idea where this is all gonna end...
Re-reading the above, it seems I have my own concept of "hassle"... Also, I wanted to underline how
research is essential and usually fruitful with today's Internet: things are not necessarily easy to find, but they can de found - thanks to the many anonymous (or not) people who put them out there.
And a special mention (again) to Randall Hyde's "The Art Of Assembly Programming", the 16-bit DOS version:
http://www.phatcode.net/res/223/files/html/toc.html
And also, if you've not seen it yet, a handy online disassembler for easily writing your own DOS bytes (chose 8086 architecture):
ODA - The Online Disassembler
One thing is really upsetting - I heard that even if one wanted to buy IDA (and spend 2000$ or so) they only sell it to trusted parties. In such cases I really strongly support piracy - it is not eating to their profits, since they are not selling it anyway, and I don't accept this infringement on my freedom to have tools.
Well that's a point of view
In that case, it shouldn't be a problem for you to dig out non-free versions of IDA lurking inside the confines of the Internet...