Civ3 Show-And-Tell

Yes, I know this is a Civ3 project, and yes I haven't done much in that section of the forums (yet). I do play Civilization 3, but to be honest I'm partial to Civ4. I am not sure if can be of much help right now so I will see how (and where) it goes, and if you do think of something I can help with let me know.
 
I started this project with a fear of spoiler info, but lately I've been thinking I'm pretty sure it's possible to see the known map for each civ, and there might be use cases for that. Is the multiplayer game tracker something that exists, or is it something you want but don't have? What type of map info would be most useful: whole map, known-map-by-civ or aggregate known world?

The PBEM tracker I was thinking of is here. An example of a game on it is here. I'm not sure if it actually supports Civ3 games? At any rate, the amount of info displayed is not great, and you have to be logged in to track games you don't have a direct link to. It's a promising idea, but to someone who's not already super-into PBEM, it's not great.

A Civ3 PBEM game that does make some attempt to show the status is this one (see the map in the first post). But it's really not enough to get any sense of the current situation - and that's with the benefit that it's an Earth map.

For me, as a lurker, the whole map would be most helpful. There would be issues of potential spoilerage, so it would have to be determined per-game to enable it. But, particularly with known maps like the Age of Imperialism one, and later-game PBEMs where the map is de facto known, I think it would make a big difference to lurker-friendliness. Although it'd probably be best to ask some of the people who actually play PBEMs to see whether they also think this would be good to have (or whether they wouldn't use it for fear of spoilers).
 
Ok, I've spent like 12 hours on this today. Wow. What did I accomplish? Online web app that can read arbitrary compressed or uncompressed C3C saves by URL to extract and display their maps.

Experimental web app where you can enter a URL to a save game

It's rudimentary. It's rough. It doesn't cache anything, and it won't restart itself if I lose power and the server reboots. But now you have some control over what it does. :cool:

Here are some URLs from this forum you can paste in there and then hit the button:
  • http://forums.civfanatics.com/attachment.php?attachmentid=361461&d=1379988203
  • http://forums.civfanatics.com/attachment.php?attachmentid=347664&d=1365461213
  • http://forums.civfanatics.com/attachment.php?attachmentid=365818&d=1386279052
  • http://forums.civfanatics.com/attachment.php?attachmentid=361905&d=1380603894
  • http://forums.civfanatics.com/attachment.php?attachmentid=373703&d=1397781891
  • http://forums.civfanatics.com/attachment.php?attachmentid=320057&d=1335676329

Edit: I see someone tried to paste a link to their local hard drive. This can't read from your PC unless you download it and run svg.py locally. The online version can only read URLs available on the Internet.
 
Ok, please try these. I made a slight change in the tag structure to more closely mirror the tiger.

Mac zoom test, svg as img
Mac zoom test, svg in DOM

When making this test I realized it's not just the big map that behaves differently depending on how I include the SVG. IE is faster than anything when I include the SVG in an img tag. Chrome seems to do better with the SVG in the DOM while IE pretty much chokes on it. I'll be doing more experimentation to see what the root cause is, but I'm more focused on other things so I'll just offer two viewers for now. Please also try two-finger-scrolling the Game of Thrones links above. They are mostly the same, but I removed a couple of constraints and wonder if it makes a difference.

Still doesn't work in chrome or FF unfortunately :(
 
Hmm. Perhaps they are using a different version of code on the tiger. I'll have to poke at it later.

Today I took a stab at the jungle tile. I'm not sure I like it, but it's better than "Jungle" spelled out on the map I suppose.

Evolution of the jungle tile colors in screenshots.
 

Attachments

  • 2014-05-25_14-36-17.png
    2014-05-25_14-36-17.png
    112 KB · Views: 204
  • 2014-05-25_14-38-16.png
    2014-05-25_14-38-16.png
    113.9 KB · Views: 144
  • 2014-05-25_14-39-07.png
    2014-05-25_14-39-07.png
    111.1 KB · Views: 129
  • 2014-05-25_14-46-43.png
    2014-05-25_14-46-43.png
    112.6 KB · Views: 124
  • 2014-05-25_14-49-32.png
    2014-05-25_14-49-32.png
    114.1 KB · Views: 248
Marshes. I really didn't like these at first, but they might grow on me. I'll move on to flood plains rather than try to tweak these now.

Edit: No, I'm going to punt on flood plains. It looks to me like the game displays the same desert tile but adds greenery to the edge of the river. I'll hunt for rivers next and perhaps give flood plain treatments in the river logic of my program.
 

Attachments

  • 2014-05-25_16-10-27.png
    2014-05-25_16-10-27.png
    53.1 KB · Views: 179
  • 2014-05-25_16-10-01.png
    2014-05-25_16-10-01.png
    77 KB · Views: 134
I updated the web app to draw the jungles and marshes, and instead of "Flood Plain" it now just puts "FP" on the flood plain tiles.

Another crazy idea occurred to me just now: an animated SVG replay viewer. It would require the sequence of save files; CivAssist can be made to collect them and can do this:


In theory I should be able to access those same save files and make the SVG map animated. I'm not exactly sure how yet, but I'm pretty sure it's doable. But that is still "crazy idea" phase. And even if I did it, you would have to have a series of save files saved to make it happen.

My current itches for this project are to find and add rivers, to find where it stores "is visible" masks for each civ, where to find the starting/home location for each civ, add more code to help me analyze the save data, and maybe identify some more of the available data.

I'm also thinking I might actually play the game today. Or maybe I could do some chores at home. Or maybe I could go outside and pretend like I have some sort of life away from the computer chair. :rotfl: Oh, that was a good one!

Edit: Oh, sometime in the past couple of days I made a workaround for the tile gaps. I am trying oversized tile shapes. At 1px (at 128px size) outside the box at each corner the faint gridlines can show up on big maps if you zoom out, and when you zoom way in on a small map you may notice the overlap, but the overlap and antialiasing makes it much smoother overall. It's very subtle, but the slight 1px-2px blur (from antialiasing) at the tile edges makes the color transitions much less in-your-face. My first idea was to stroke the tiles with the same color, but I may yet go back and do something besides named-color solid fills for base terrain, so the oversized tiles are better future-proofed. I briefly tried 2px oversize (at 128px x 64px size); I forget how the transitions looked, but the jagged tile overlaps were too noticable.
 
A quick jungle tweak. Which is better? ("None of the above" is assumed; which is the least harmful of these two?) The only difference is whether or not the trunks are obscured by the "leaves".

 

Attachments

  • 2014-05-25_18-20-07.png
    2014-05-25_18-20-07.png
    110.2 KB · Views: 312
A quick jungle tweak. Which is better? ("None of the above" is assumed; which is the least harmful of these two?) The only difference is whether or not the trunks are obscured by the "leaves".


I prefer the one on the left. I like the tiger one, too, but it probably would get old a lot more quickly than the other options.
 
Hmm, a split decision. I may try lowering where the trunks disappear into the leaves as a compromise between the two.

I discovered (and rediscovered) a bunch of stuff today while analyzing the save file. I found out where the non-spoiler improvements info for each civ is. i.e. I can choose to display the last-known improvement for known-but-not-currently-line-of-sight-visible tiles. Apparently forests appear and disappear in real time whether or not you have lin-of-sight. (Maybe I'll get around to game-test-verifying that someday.) I found where the actual improvements and all resources are. I found trade network IDs, tile visibility which oddly is divided into unit-visible *, culture-visible and colony-visible (?!). I found that there is a per-civ view of land trade networks. I found the world seed number. I found unit ids (presumably the top/showing unit ID for that tile), the city ID and the City ID indicating which city is actively working that tile. I found barb-associated info I haven't fully figured out yet but think it's related to the barb tribe. I found the player start locations.

You know what I didn't discover? Rivers. Which was my main hope. I did find a couple of spots of code I believe to be related to tile transitions/graphics. Unlike my map, the game map varies terrain graphics based on adjacent tiles, and I think I know where this is even if I don't understand it yet. Since rivers are on the edges, perhaps the river info is coded in these places. I'll go look at mod info for terrain graphics and maybe I can connect some dots as to what these numbers are.

* (I know barbs can spawn in sight of non-combat units but as far as I know not within sight of culture borders or combat units. Apparently there is some in-memory logic done that isn't represented in the save for that distinction.)

I think I'm about to burn out on this for a while. I have the info I need to add a bunch of stuff--except rivers :mad:--to the map, but I don't feel like finding or making the graphics for it all right now. Maybe I'll get text labels for non-strategic (because spoiler) resources on there before I burn out completely; we'll see.

Info I (re-)discovered in the past 24 hours or so:
Spoiler :
Code:
Assumptions:
	Barbs are first civ, then human player(s) then AI player(s)

Guesses:
	WRLD section data does not change from turn to turn
	Barbs are first civ so they can always ensure they're at war with everyone? And/or otherwise programatically differentiate themselves from other civs?

Thoughts:
	Are continents represented in WRLD? Guessing not since the sizes seem consistent
	Is there only one barb camp per civ? In my huge 2-player map I have reason to believe there are only two current barb camps on a huge continent.

WRLD
	2 bytes
	bit flags?
	No correlation to map size or player # found
	Not turn or date
	LSb is one for all 3 of my test saves
		gamesaves/unc-got-map.sav
		0000  4b 00                                             K.
		(75,)
		 1001011
		gamesaves/unc-test.sav
		0000  11 00                                             ..
		(17,)
		   10001
		gamesaves/unc-lk151-650ad.sav
		0000  6b 00                                             k.
		(107,)
		 1101011
WRLD (2nd instance)
	164 bytes
	looks like 41 integers
	has map width and height
	0x00 int? - Continent ID of the ocean (may be a short or even byte, but the high bytes have been 0 so far)
	0x04 int? - Map height / y rows
		4 ints? - No clue. Why the separation between rows and columns? These are small enough to be two x,y coordinates, but they aren't. Only lsB seems to be used..hmmm, bit flags?
		map data like arid/etc? Kinda doubt it.
		don't think they are continent IDs
		Hmm, last int at 0x14 always 0?
	0x18 int? - Map width / x columns
	0x1c - 0x9b - 32 * int - Index of home tile for each civ! -1/ffff for nonexistent civ or barbs
		guessing this is the start location and not necessarily the current home
		0x1c int - Barb start location: -1 / ffff because n/a
		0x20 int - Human player 1 start location (tile index, not x,y)
		... next human players and then AI players
	0x9c - int - World Seed
	0xa0 - int? - no clue - 5 seems to be a common value so far

WRLD (3rd instance)
	52 bytes
	looks like 13 integers
	1 is a common value, also 2. Occasionally 0 or -1/ffff . Nothing else seen so far

TILE (1st instance, aka "A"? aka Tile36)
	36 bytes
	think mostly ints but some shorts or bytes mixed in
	0x00 - no clue. Seems 0 for first byte, 0 or 1 for second byte
	0x02 - no clue. Both bytes used
	0x04 int - Resources! -1/ffffffff if no resource, so I can tell this is a 4-byte int
		0x00 - Horse
		0x01 - Iron
		0x02 - (unknown strat) (coal?)
		0x03 - (unknown strat) (aluminum?)
		0x02 - Saltpeter
		0x04 - (unknown strat) (oil?)
		0x05 - (unknown strat)
		0x06 - (unknown strat)
		0x08 - Wines
		0x09 - Furs
		0x0c - Spices
		0x0d - Ivory
		0x0a - Dyes
		0x0b - Incense
		0x0e - Silks
		0x0f - Gems
		0x10 - Whales
		0x11 - Deer
		0x12 - Fish
		0x13 - Cow
		0x14 - Wheat
		0x15 - Gold
		0x16 - Sugar
		0x17 - Tropical Fruit
		0x18 - Oasis
		0x19 - Tobacco
	0x08 int - definitely associated with units. Wild guess: visible/top unit ID?
	0x0c int - varied numbers, can't quite spot a pattern
	0x10 byte? or low nybble of 0x11
	0x11 byte? - Pre-Conquests terrain? All 0's in most samples, but LK's world map seems to have terrain info here
	0x12 short? - Pre-conquests flag of some sort? All 0's in most games, but some variation in LK's WM
	0x14 short? - Barb info. Do barbs have tribe IDs? They must I guess, but would that be here? LK's WM has repeated number; my maps are limited but seem to have unique IDs
		0x003e - Barb camp and/or barb warrior on my test map
		0x004b - ? - very common on LK's WM, incl one in water
		on isolated map, values of 0x0001, 0x0029 and 0x0011
	0x16 short? - City ID
	0x18 short? - Colony for me is 0x0000 ... Colony ID?
	0x1a short or byte - Continent ID
	0x1c short? - no clue, varies, big numbers
	0x1e short? - all -1/ffff
	0x20 short? - all zeroes
	0x22 short? - all zeroes

TILE (2nd instance aka Tile12)
	12 bytes
	0x00 byte? (int I think) - Improvements
		00000001 - Road
		00000010 - Rail?
		00000100 - Mine
		00001000 - Irrigation
		00010000 - Fortress?
		00100000 - Goody hut?
		01000000 - Pollution?
		10000000 - Barb camp
	0x01 - all zeroes. High byte of short? No, I think Improvements is one byte. Guessing byte padding
	0x02 - all zeroes - Guessing byte padding
	0x03 - all zeroes - Guessing byte padding, or maybe these 4 "0"-bytes are an unaligned integer?
	0x04 - part of Terrain? Not sure why lsB would be 0, but what else is this?
	0x05 byte - Terrain. Low-nybble base terrain, high-nybble overlay terrain
	beyond -- looks bit-flag-ish, but doesn't seem to be visibility
		By-civ culture visibility? Masking with 0x02 looks like it could be my civ's ...something. Culture vis isn't right.
	0x0a short? - 
		values 0x000a or 0x002a associated with starting positions ... 00001000 bit
		Think the high "2" bit 00000010 00000000 may be associated with human players because 0x002a is human starting position. But there are plenty of other values, and I don't know what this means.

TILE (3rd instance aka Tile4)
	4 bytes
	0x00 int - All zeroes?

TILE (4th instance aka "B"? aka Tile128)
	128 bytes
	0x00 int - bit mask of is visible to civ (fog of war). lsb is barbs, next is human player 1, etc..
	0x04 int - bit mask of is visible NOW for units only, does not seem to include city/culture visibility
	0x08 int - bit mask of visible to colonies?
	0x0b int - bit mask of city/culture visibility
	0x10 byte - tile transition values? Only values are at top row and bottom two rows of map
	0x11 byte - tile transition values? Seems mostly to left of land transitions, but not absolutely. Also at top and bottom of map
	0x12 byte - tile transition values? Seems to apply to most land transitions, but standalone and left-most hills and mountains seem to be -1/ff . Presume this controls how the overlays/transitions work to e.g. show continuous hills
	0x13 byte - 0 or 1. Flag of some sort; can't figure out what it means. Or maybe high byte of previous value?
	0x14 short - City ID of the city working this tile (includes city tile)
	0x16 - Guess: 32-short-int list of trade network IDs known to that civ? But not necessarily tradeable I think. Perhaps this is what trade routes are seen/known to that civ because of recency of tile improvement knowledge? Maybe both civs have to be able to see the same trade network ID to trade via that tile?
		0x16 short? - all 0xffff
		0x18 short? - Appears to be ID of road/land trade networks. Each isolated road network has its own ID
		0x20 short? - Also seems to be land trade network ID, but is 0xffff everywhere on my pre-conquest-win test save (no opponents left). Maybe this is used to determine which civs might trade with each other?
		0x22 short? - At a quick glance looks identical to 0x20
		...
	0x56 - Guess: 32-byte per-civ-knowledge improvements flag
	0x76 short? - can't determine what is is, but 1a1a values look familiar; maybe this is related to Tile36 0x06? Maybe this helps determine which graphic tile to use?
	0x78 short? - All zeroes. Maybe high-bytes of int?
	0x7a short? - All zeroes.
	0x7c short? - All zeroes.
	0x7e short? - All zeroes. Or maybe this has been byte padding
 
Ok, bonus and luxury resources are on the map as text. (Hmmm, what about bonus grasslands? I don't have those accounted for.) As are cities as generic "CITY" text.

I just realized that what I identified as "culture visibility" may be exactly equal to territory. Duh. I'll have to double-check that, but I may be able to color-shade territory, but since I currently don't have the ability to turn layers on and off I need to be careful about overlapping too much. It's already confusing with CITY on FP or a resource.

Sample map

The web app is also updated to show this info. I won't bother with a screenshot because it looks exactly like the earlier ones except it has additional text on it.

I noticed somebody got clever and put a save game on their Google drive to access it with my web app. Good thinking!

I'm really disappointed that I don't have rivers, but the map is currently pretty darn useful and doesn't look bad.
 

Attachments

  • cow.png
    cow.png
    60.9 KB · Views: 165
Ok, the cow is staying. I started this project with a jumble of ideas for how this work but not really how it will look. For me that's secondary. But it is starting to accidentally pull together in a somewhat cartoon-ey cohesive feel. I'm liking it because it's becoming more and more recognizable as an iconized civ map.

I added three more icons and made myself a quick spreadsheet to give me a really good starting point for scale and positioning to avoid trial-and-error in adapting each icon into the map. I did have to trial-and-error city placement a bit, but that's because I don't want the icon centered based on its outer borders; the steeple should stick farther up, so I had to make adjustments.

It's amazing how much more readable the map is with icons instead of "CITY" and "Cow". Adding icons is much faster now, but it still does take time, so that's it for tonight.

I updated the web app, I'll push to GitHub shortly, and here are today's links:

Standard size map
Huge map


Edit: It really bugs me that I don't have rivers figured out yet. And bonus grassland. I think those two things plus adding icons makes this map genuinely useful in discussing game situations. Sure, there are no units, improvement or strategic resources, but you have a really good lay of the land.
 
No updated graphics today, but I did look for rivers. Didn't find them. I'm reading in the continent sections now, not that it's any use. I found the BG flag and I think the "forest already chopped" flag. I found another flag that seems to be in all the tundra, none of the jungle but spread out around all other land terrain. I'm not sure what that's about. I found mountain snow caps. Aren't there snowy trees? I don't see any, so maybe I'm imagining I saw them, or maybe it was the all-winter terrain. There is also a flag that marks the "fat x" around each city whether or not the culture borders encompass or exceed it. I wonder why that's there?

I got clever and did a bitwise AND against several river tiles on values I thought are probably the ones that choose which tile graphic to use, but apparently those aren't rivers. :(

Maybe tomorrow I'll make a custom map that's almost all grassland with a little bit of river and see where the variations are.
 
Maybe tomorrow I'll make a custom map that's almost all grassland with a little bit of river and see where the variations are.

That helped. I think I found the rivers! It's the very first byte in the very first tile. I went past it quickly when I was picking out the easy stuff (e.g. "hey, 0x22 is always grass tile!") and didn't consider it again until I decided it had to be in one of the tile sections and found it via process of elimination and a big-ass all-grass map with some river in the middle.

But now I have to go pretend I have a life for a while.
 
Here's a peek at a little bit of what I did last night and what I may spend some time on now. The bonus grasslands are marked, some of the rivers are sort-of there and I have debug text on the tiles with the river "short" value. That may or may not be masked...I can apply bitmasks.

I haven't quite figured out what each bit means or if the bits even have a consistent meaning. I may end up with a series of "if it's this, this or that then river on NE edge."

There doesn't seem to be a single bit that says "this is a river tile that gets 1c bonus", at least that I've identified yet, although now I'm realizing my hand-drawn river on a test map may not be consistent with a generated river. (This screenshot is from a generated map.)

Edit: I just noticed I'm spoiling BGs under forests. I'll have to put a forest and/or overlay check in.

Edit 2: Got it. I made some wrong assumptions when looking at my hand-painted river map. If the river byte is nonzero then it's on a river. When I hand-drew the rivers the corners got bonuses and river values that wouldn't happen on a generated map, and that confused me.
 

Attachments

  • 2014-05-29_18-37-52.png
    2014-05-29_18-37-52.png
    179.1 KB · Views: 157
Top Bottom