[REFERENCE] How Unit/Tile Visibility Works in BTS 3.19

Merkava120

Oberleutnant
Joined
Feb 2, 2013
Messages
450
Location
Socially distant
Lately, as I've been searching around in the SDK trying to understand Civ4's visibility system, I have learned two things:
  1. The visibility system in Civ4 is brilliant, powerful, and elegantly designed.
  2. The methods and variables used to form this system are named in deceptive, misleading and confusing ways.
As a result, I suspect many SDK modders are like me, and have had to spend hours Ctrl+F-ing their way around the visibility code before finally understanding where to put their mods. So I thought I'd write down what I figured out, in case anyone else gets confused in the same way.

Visibility in BTS 3.19

There are three levels of visibility in Civ4:
  1. Being able to see a tile at all (instead of black) - this is referred to as revealed in the SDK. "Revealed" is also the word used for whether or not a city can be seen (as opposed to displaying terrain without the city), and the "RevealedOwner" is which border color is displayed on the tile.
  2. Being able to currently see a tile (instead of a darkened tile/the "fog of war") - this is referred to as visible in the SDK.
  3. Being able to see units of a certain invisible type (such as INVISIBLE_SUBMARINE) - this is referred to as invisiblevisible in the SDK.
Any method with "revealed" in the definition - for example, CvCity::setRevealed() - is talking about whether or not the thing should be displayed on the map at all.
Any method with "visibility" or "visible" in the definition - for example, CvPlot::isVisible() - is talking about whether or not the thing can currently be seen.
And any method with "invisiblevisibility" or "invisiblevisible" - for example, CvPlot::isInvisibleVisible() - is talking about whether or not a unit of a certain invisible type is visible.

All three layers of visibility are determined separately for each team. A tile can be revealed for one team, revealed and visible for another team, and neither for a third team.

In addition, the third "InvisibleVisible" layer is determined separately for each team AND invisible type. A submarine might be visible to a team - thus being InvisibleVisible - but the same team might not be able to spot a stealth bomber, even on the same tile.

Note: throughout this reference I include references to definitions in the SDK like CvPlot::getRevealed(). I only include variables inside the parentheses if I want to talk about them, so if you want to call these functions in your code you'll have to look at the definition yourself (visual studio should tell you what variables to include as soon as you type the name of the method).

First Level: REVEALED

A tile's revealed status is held in the SDK as a boolean. It's accessed through CvPlot::getRevealed() and set through CvPlot::setRevealed() - simple as that.

There are also a collection of methods that return the improvements, features, and so on that are revealed on the tile - for example, CvPlot::getRevealedImprovementType() will get the improvement that is visible on the tile at the current time, even if the tile is not visible. Note that the revealed improvement/feature/whatever may not be the same as the actual improvement/feature/whatever!


Edit: Two notes courtesy f1rpo - first, what things are actually revealed vs. not revealed?

Nitpick: Features are actually not affected by the fog of war, only improvements, routes, tiles owners and cities are. (Well, cities only so long as they have never been revealed; once revealed, razing and ownership changes can be observed.)

Second, note that BTS reveals a lot of map changes that players should not be able to see:

There is also some BtS UI code that calls getRouteType or getImprovementType when it should really call the getRevealed... functions. For example, the pathfinder waypoints give away fogged routes. Not terribly important, but, ideally, modders should be careful about using getImprovementType, getRouteType and getOwner; there's a decent chance that a getRevealed... function should be used instead. Generally, observability checks are easy to forget.

setRevealed() is called in four places:
  • CvUnit::changeVisibilityCount() - see below for details on this one.
  • CvMap::setRevealedPlots() - this function sets every plot's revealed status to TRUE, and is only used in two places as far as I can tell - techs that reveal the entire map (through the xml tag bMapVisible), and in CvGame::regenerateMap().
  • CvDeal::startTrade() - when a map trade is made, every tile that is revealed to the map owner is set to revealed for the map receiver.
  • CvPlayerAI::AI_doAdvancedStart() has a small section after the comment "if you get to here, advanced start is broken" where it sets a new starting location for the player and reveals tiles around that location.
The first one is the interesting one if you want to get into modding visibility and sight - which brings us to the second level.

Second Level: VISIBLE

A tile's visible status is controlled through a count, not a boolean! In other words, every unit that can see a tile adds one to the count, and if the count is greater than 0, then the tile is counted as "visible".


All of these counts are held in an array that is accessed through CvPlot::getVisibilityCount(team) and CvPlot::changeVisibilityCount(team, amount, SeeInvisibleType). The array has one value for each team - so passing "team" to each of those methods is basically passing an index to the array.

There is also a method called CvPlot::isInvisible(team). This simply returns whether or not the count for that team for that tile is greater than 0 - OR whether or not the stolen visibility count for that team is greater than 0.

StolenVisibility is identical to visibility and uses all the same methods - just with StolenVisibility in the definition instead of Visibility - except that it's used for civs that are being allowed to spot tiles through another team's eyes instead of actually spotting the tiles themselves. This happens when you have a lot of espionage points against a team.

Where are these methods used? See below - this can get confusing unless you fully understand all the levels of invisibility. (It'll also explain why there's a SeeInvisibleType passed to the changeVisibilityCount method.)

Third Level: INVISIBLEVISIBLE

A tile's visibility status is handled through a count, not a boolean - similar to the above, except it's through a two-dimensional array instead of a one-dimensional array. This array contains one "row" for each team, with one spot in that row for each invisibility type. If any spot is greater than 0, that means there is some unit somewhere belonging to the team for that row, who can see that particular invisible type, on that tile. This array is accessed through CvPlot::getInvisibleVisibilityCount


Why is this useful? Say you have a stack of submarines and stealth bombers and infantry in the same tile. An enemy destroyer approaches and gets within visibility range of the tile. This destroyer can see submarines, but not stealth bombers. So CvPlot::getInvisibleVisibilityCount(destroyer's team, invisible type submarine) returns greater than 0, and CvPlot::getInvisibleVisibilityCount(destroyer's team, invisible type stealth) returns 0. The submarines are spotted, but the stealth bombers remain invisible, thanks to the array being 2d instead of 1d.

How to tell if something is visible

Units: There are actually two ways. First, you ask CvPlot::isInvisibleVisible(spotting team, invisible type of the unit you care about). That method just looks at the array and returns TRUE if the count is greater than 1 for your spot. Second, you can call CvUnit::isInvisible(), which will check 1) if the unit is always invisible (which is an xml tag; spies use this) and 2) if isInvisibleVisible returns false for the tile. If neither of those are true, and the unit is not cargo, the method returns false.


Tiles: CvPlot::isVisible(). This returns true if CvPlot::getVisibilityCount(team) is greater than 0.

Units have no isVisible method and tiles have no isInvisible method - don't mix those up.

So where does the game check visibility?

If you're like me, you started getting into the visibility system by doing a Ctrl+F on "visibility", and that takes you to the second two levels. But those levels are actually called within each other and in so many other places that it gets confusing very quickly. Here's the key:


First, and most importantly, all of these methods are checked in basically one place, all at once: CvPlot::changeAdjacentSight(). This method is somewhat misnamed, because it actually changes sight for all tiles within a given range of the center tile, not just the adjacent ones. It's the main method the game uses for updating sight, and with only a few exceptions (listed below), every time anything needs to know if visibility changed anywhere, this method is called.

The method takes a team, a range, a boolean called bIncrement, a unit object, and a boolean called bUpdatePlotGroups which doesn't have anything to do with spotting and will be ignored by me for now. changeAdjacentSight() looks at the unit, then at every tile within range, and sees if those tiles are blocked by anything (by calling canSeeDisplacementPlot). After that, it calls changeVisibilityCount() with 1 if bIncrement is true and -1 if bIncrement is false. (In other words, bIncrement decides whether to add 1 to the visibility count or subtract 1 from the visibility count.)

Before we go on, three interesting side notes:
  • For each plot, before calling CvPlot::canSeeDisplacementPlot() to see if line of sight is blocked, this method actually checks CvPlot::shouldProcessDisplacementPlot() to see if the unit should be looking at the tile at all. shouldProcessBlah takes the unit's facing direction and determines whether or not tiles are too far away from that direction. This could be easily modified to make units see only in the direction they are facing, if you wanted to.
  • If a unit is "aerial", then the method doesn't care about line of sight or anything like that. Right here aerial just means DOMAIN_AIR, but this could (very, very easily) be modified to include, say, helicopters.
  • No matter what direction the unit is facing, as long as it is not NO_DIRECTION, changeVisibilityCount is called for every adjacent tile - twice, once with +1 and once with -1. I suspect this is just to flag the other methods called by changeVisibilityCount (see below) without actually affecting the total count (since the count might have increased already above) but I'm not positive.
Next we turn to changeVisibilityCount(), which is called for each tile within the range passed to changeAdjacentSight(). We already mentioned this one, but there's something I skipped: changeVisibilityCount() doesn't just adjust the counts in the visibility array - it also calls changeInvisibleVisibilityCount and setRevealed!

So what's going on here? Here's the big picture:
  1. Something requires the visibility to be checked around a UNIT, and calls CvPlot::changeAdjacentSight() for that UNIT and a spotting team, passing along a range to check and whether sight should be set to "more true" (bIncrement = true) or to "less true" (bIncrement = false).
  2. changeAdjacentSight() looks at each tile within range and line of sight of the UNIT, and calls changeVisibilityCount() for each one, passing along the SeeInvisible types of the unit (there can be multiple see invisible types).
  3. changeVisibilityCount() does what its name says.
  4. changeVisibilityCount() calls changeInvisibleVisibilityCount to see if the see invisible status changed for the spotting team. This does nothing on its own, it just updates the invisibility so that other places will know if the units on that tile are invisible or not.
  5. changeVisibilityCount() then calls setRevealed if the status of the plot changed from not visible to visible. setRevealed checks if the plot was already revealed, and if not, reveals it. Additionally, setRevealed updates symbols, fog and visibility of graphics on the tile if the spotting team is the active team (i.e. the dude watching all this from the real world). This is also where bUpdatePlotGroups actually gets used if you're curious about what that does - it's rather technical and involves trade routes.
  6. changeVisibilityCount() then calls CvTeam::meet() for teams on the tile if the tile went from not visible to visible. meet() is a tiny function that just checks if the two teams have met, and if not, makes them meet each other (i.e. "stick your head on a pole" blah blah blah).
  7. Lastly, changeVisibilityCount() does a few miscellaneous things - "CvCity::setInfoDirty()", changeStolenVisibilityCount (for teams that can spy through the spotting team's visibility), and then updates the fog, minimap color and center unit if the spotting team is the active player (the guy from the real world again).
That's a lot of stuff packed into one place! Basically, any time you want to check visibility because of something your mod did, you just need to call changeAdjacentSight() and all the other stuff will follow.

For that reason, almost every time the game needs to check visibility, it calls changeAdjacentSight().


So what if you want to check/update visibility without relying on units?

And this is where the game doesn't call changeAdjacentSight() - instead, it calls the specific methods needed: changeVisibilityCount, changeInvisibleVisibilityCount, or setRevealed().

For a good example of how this is used, see CvTeam::addTeam(), which sets up the initial visibility settings for the added team for every tile in the game. Other examples include setFeatureType() and similar methods that change things about a tile.

What happens when units are "spotted"?

This is a natural question for those wanting to do mods that involve the spotting system. What if you want a unit to gasp embarrassingly when it is spotted by an enemy?


Unfortunately, there is no direct "onUnitSpotted" section of the SDK to go stick things in. However, there is one place that deals with the only thing that happens when a team goes from spotted to unspotted: CvTeam::meet().

Any time meet() is called, that means somebody just got spotted. In fact, the SDK doesn't even check if the teams have met already before calling meet() - that's done inside meet() itself. So if you want to do something when units are spotted, you could either put it in meet(), or find all the places meet() is called (the most important one is probably in CvUnit::setXY()) and add your stuff there.

And what determines visibility range?

I left this until the end because Civ4 actually doesn't care all that much about visibility range - it's mostly seen as a constant between units. But there's some really interesting stuff with elevation here, and many mods might try to add visibility range to units in ways besides promotions, so here's a rundown:

  • CvUnit::visibilityRange() returns the global define UNIT_VISIBILITY_RANGE, plus getExtraVisibilityRange().
  • CvUnit::getExtraVisibilityRange() returns the extra visibility range...
  • ...which is set in CvUnit::changeExtraVisibilityRange(). This method just adds the extra visibility range to the unit - very straightforward - but also calls our old friend changeAdjacentSight() before and after doing so. (So if a unit is given extra sight at any point, its new visibility range is applied immediately.)
  • Visibility range from promotions is applied in CvUnit::setHasPromotion(), which calls changeExtraVisibilityRange() to do it (either increasing for gaining the promotion or decreasing for losing the promotion).
  • Finally, CvPlot::canSeeDisplacementPlot() checks line of sight between this plot and other plots. It doesn't actually use visibilityRange at all, it just checks the plots passed to it - but I'm putting it here because this is where the seeFrom and seeThrough values for features/hills/etc. are used. They don't actually change visibility range at all - they are just used as sort of "elevation" levels. Hypothetically, you could add a whole bunch of seeThrough and seeFrom levels from 1 to 10 and this method would work fine with it. (seeThrough = height of thing you're looking which might be blocking the thing you're looking at, seeFrom = height of thing you're looking from, and is also used as the height of the thing you're looking at itself.)
    • However, note that the seeFrom and seeThrough values for hills and peaks come from GlobalDefines.xml, not CIV4TerrainInfos.xml.
  • The extra visibility with hills and peaks actually comes from a line in changeAdjacentSight() that adds one to the range of a unit before calling canSeeDisplacementPlot() for every tile in that range. The game considers those extra tiles the outerRing, and if a tile is in the outerRing, it's visible IF
    • That plot is higher than the first plot looking toward it (this is why you can see hills from further away), OR
    • The plot(s) in between are all lower than the plot looking toward it (this is why hills can see +1 further than plains; hill->plains->thing means that "thing" is not blocked even if view range is 1).
    • Forests can't see extra tiles because even though their seeThrough is 1, their seeFrom is 0, which means they block sight but don't get "extra sight".
    • I haven't worked out the specifics, but this is applied to water tiles, too, involving CvTeam::changeExtraWaterSeeFromCount(). That's how some techs can give you extra view range over the water - again, not actually modifying visibility range, just changing the effective "heights" of different tiles.
Edit: a very interesting observation by f1rpo -

About CvPlot::shouldProcessDisplacementPlot, it seems worth noting that NO_DIRECTION gets passed to that function unless the observing unit has bLineOfSight. That Unit XML tag, along with the shouldProcess... function, was added by the BtS expansion, presumably just for mods, in particular Firaxis's own Afterworld mod.

If you haven't played Afterworld, in that mod units can only see tiles in the direction they're actually facing. So it seems that bLineOfSight and the shouldProcessDisplacementPlot() function are used together to make units see in one direction rather than the regular omni-directional sight - very interesting tool for modders who want something more tactical.

Other Interesting Methods
  • CvUnit::willRevealByMove(plot) sees if any plots will be revealed (changed from dark to displayed) if the unit moves to plot, and returns true if that's the case. (There is a setting somewhere in the xml to prevent units from revealing new tiles; this is how that tells when to prevent a unit from moving.) This method checks tiles within the visibility range of the unit plus 1.
  • CvPlot::updateSeeFromSight() is another interesting one which loops through plots near the focus plot, within a range that is taken to be either the global define UNIT_VISIBILITY_RANGE + 1 + the total sum of visibility range that can be given by promotions, or the global define RECON_VISIBILITY_RANGE + 1, whichever is higher. (RECON_VISIBILITY_RANGE is used for air recon missions.) updateSeeFromSight() then calls CvPlot::updateSight() for each plot in this range, which calls - you guessed it - changeAdjacentSight(), passing the global define PLOT_VISIBILITY_RANGE as the range there. updateSeeFromSight() is used whenever the feature, plot or terrain of a tile is used, and nowhere else that I can tell. updateSight is used when the entire map is revealed all at once, independently of updateSeeFromSight.
TLDR / Takeaways

  1. If you want something to happen right when a unit is spotted, look in setXY() where CvTeam::meet() is called - that's where you'll want to insert stuff.
  2. If you want to change visibility range of units, your target is probably CvUnit::getExtraVisibilityRange(), but keep in mind that the "heights" (seeFrom/seeThrough values) of hills and peaks need to be considered because they'll mess with your visibility range independently - and also remember that these are set up in GlobalDefines instead of CIV4TerrainInfos.
  3. If you want to change how tiles are spotted - for example, through random invisibility - your best bet is to modify changeAdjacentSight(). (That's actually how the Random Invisibility mod does it - that whole mod is like four lines.)
  4. If you really need to change visibility stuff even if units aren't involved, the important methods are setRevealed(), changeVisibilityCount(), and changeInvisibleVisibilityCount(), and to see how this is done, you might check out CvTeam::addTeam(). This only matters for changing the contents of a tile in a way that changes visibility - for example, when planting a forest or something.
  5. If you want to do some interesting stuff like fooling a Civ or player to think something is there when it is actually not, you can use setRevealedFeatureType(), updateRevealedOwner, and similar methods with "revealed" in the definition. This could, theoretically, be used to make "fake map trading" a thing.

Thanks for reading, and I hope this was useful.

If I missed anything or if you find any errors, post about them below!
 
Last edited:

f1rpo

plastics
Joined
May 22, 2014
Messages
1,552
Location
Germany
Thanks for sharing.

About CvPlot::shouldProcessDisplacementPlot, it seems worth noting that NO_DIRECTION gets passed to that function unless the observing unit has bLineOfSight. That Unit XML tag, along with the shouldProcess... function, was added by the BtS expansion, presumably just for mods, in particular Firaxis's own Afterworld mod.
If you want to do some interesting stuff like fooling a Civ or player to think something is there when it is actually not, you can use setRevealedFeatureType(), updateRevealedOwner, and similar methods with "revealed" in the definition.
Nitpick: Features are actually not affected by the fog of war, only improvements, routes, tiles owners and cities are. (Well, cities only so long as they have never been revealed; once revealed, razing and ownership changes can be observed.) Features and, for that matter, buildings and bonus resources arguably should've been given m_aRevealed... arrays too. As it is, disappearing features can give away AI cities in the fog of war, and, in the spirit of the BUG mod, it would be fair game to implement an alert notifying the player of any buildings completed in fogged rival cities. There is also some BtS UI code that calls getRouteType or getImprovementType when it should really call the getRevealed... functions. For example, the pathfinder waypoints give away fogged routes. Not terribly important, but, ideally, modders should be careful about using getImprovementType, getRouteType and getOwner; there's a decent chance that a getRevealed... function should be used instead. Generally, observability checks are easy to forget.
 

Merkava120

Oberleutnant
Joined
Feb 2, 2013
Messages
450
Location
Socially distant
Changes added! Thanks for the good information. I couldn't figure out what CvPlot::shouldProcessDisplacementPlot() was actually used for, thanks especially for pointing that one out. Might be fun to give every unit bLineOfSight sometime, haha...
 

pecheneg

Prince
Joined
Mar 28, 2021
Messages
571
Thank you very much. There is a Big Black Hole in the modiki regarding invisibility and review.

This could, theoretically, be used to make "fake map trading" a thing.

Hmm. If we consider only the Second World War, then this is already a full-scale "mock-up war".
It may be unrealizable, but I have my own idea-fix.
In vanilla and current mods, ground-based "reconnaissance" after the advent of aviation are banal combat units. However, historically they have been a necessary addition to aviation intelligence, which has been deceived (and deceived now) often and successfully. Including various imitators. The same applies to spies, who are more likely saboteurs. If it is possible to build "fake" improvements that deceive (depending on the level) certain means of intelligence, it will already be very good. Well, if manage to get closer to the real scale and significance of such games, then it's just fine. For example, in PIE mod, gold is spent on remote attacks. Actually, the "theft" of this component will dramatically increase the importance of intelligence and mock-up warfare.
 

Eusebio

Chieftain
Joined
Dec 8, 2020
Messages
21
Do you know if is it possible to "link' the visibility update mechanic with the discovery of some research? I would like to see if something like the idea suggested here, on the "Map Exploration Mechanics" could work. The idea is that before the discovery of a given research ("Exploration", for example), every plot on the map that has the "revealed" level but not the "visible" level (in other words, every tile obscured by the fog of war) should be converted back to non-revealed level. In other words, before the research of this technology, you'll only be able to see the tiles within the visibility of your units and cities; everything else returns to black.
 

Merkava120

Oberleutnant
Joined
Feb 2, 2013
Messages
450
Location
Socially distant
Do you know if is it possible to "link' the visibility update mechanic with the discovery of some research?

Absolutely! In CvPlot.cpp, in CvPlot::changeVisibilityCount() there should be the following -
Code:
if(bOldVisible == isVisible(eTeam))
     return;

After this it does stuff reserved only for tiles that are currently visible. You should see a call to setRevealed(eTeam, true, false, NO_TEAM, bUpatePlotGroups) just below there. The first bool "true" is the new status of revealed.

That if statement about OldVisible is where the game would handle stuff that happens to a tile that was visible, but now is not. Replace "return" with whatever you need to check, for example techs, and then put setRevealed(eTeam, false, etc.) at the end if players do not meet the requirements.

Note that you'll want to add an xml tag to TechInfos governing this if you want to be not annoying to future modders. But that's like the easiest thing to do in sdk thanks to this tutorial.

You might also consider just having all ancient units bNoRevealMap tag set to 1. That prevents the player from being annoyed at having to re-reveal everything, but would prevent the units from moving anywhere for a long time.
 

Eusebio

Chieftain
Joined
Dec 8, 2020
Messages
21
Absolutely! In CvPlot.cpp, in CvPlot::changeVisibilityCount() there should be the following -
Code:
if(bOldVisible == isVisible(eTeam))
     return;

After this it does stuff reserved only for tiles that are currently visible. You should see a call to setRevealed(eTeam, true, false, NO_TEAM, bUpatePlotGroups) just below there. The first bool "true" is the new status of revealed.

That if statement about OldVisible is where the game would handle stuff that happens to a tile that was visible, but now is not. Replace "return" with whatever you need to check, for example techs, and then put setRevealed(eTeam, false, etc.) at the end if players do not meet the requirements.

Note that you'll want to add an xml tag to TechInfos governing this if you want to be not annoying to future modders. But that's like the easiest thing to do in sdk thanks to this tutorial.

You might also consider just having all ancient units bNoRevealMap tag set to 1. That prevents the player from being annoyed at having to re-reveal everything, but would prevent the units from moving anywhere for a long time.

Thanks for the answer and the link, they will help me a lot! :goodjob::goodjob:
 
Top Bottom