Trade Route Screens, Tutorials, Questions, etc.

[Trade Winds]

This assert has already been posted at the bug thread, you can find the bug thread here, it will tell you which ones have already been found http://forums.civfanatics.com/showthread.php?t=505306.

Thank you. I'll check that thread before posting new bugs.

 

[Kailric]

Another update, I "seemingly" have the three main trade screens in M:C working through XML setting now. There is still several things left to do and fix up. I have another issue to deal with and that is Screens that allow you to sail from, say, East and or West. Since each screen only has one Europe associated with it now, we have thus removed the EUROPE_WEST and there is no current code to differentiate between , say, Spice Route on the west side or Spice Route on the east side. So, I am thinking that along with plots having a "getEurope" variable it also has a "getEuropeDirection" variable. So that when you assign a Europe to a plot you also assign its direction.

We "could" do all this with the same mathematical formulas that assigned the Europe to start with, but each time a direction is needed we would have to first calculate what Europe Direction the unit is already in, and then calculate its opposite direction. So, I just think it would be easier to add another variable. Any other ideas on this, (Nightinggale)?

For the tutorial I will probably add one extra Generic screen, like I mentioned before, in order to go through all the steps it takes to add a screen. I want to add an additional Trade Route to M:C anyway, so this will also be in preparation of that.

EDIT: I updated the first post with my To Do list.

 

[Nightinggale]

The first thing, which comes to my mind is to make a loop of access points and then add a button in Europe for each available access point. The player can then click the point he wants the unit to arrive in. However that would result in 6 buttons if we use both east and west and we split directions into 8. That might result in a graphical issue, but at least the player can control where the unit arrives. We can also make a button for "the plot it left from".

Maybe this needs a new variable and maybe it doesn't. I seem to remember something about the game setting the plot pointer right away. Once the travel counter is 0, then the unit is added to the plot it is pointing to. I wonder what would happen if you place a pirate on that plot while a ship is in transit.

I think we should postpone adding buttons in Europe for a future commit. You are affecting a huge amount of code already and tracking issues in it could be tricky. It will be ok to make a version where we for a while lacks proper control of where the unit arrives. One possible issue, which comes to mind with buttons like that is that the button clicked should be transmitted on network if network games should ever work. This reason alone is enough to make buttons a separate commit. Perhaps adding the buttons in one commit and then make them network safe in the next would also be a good idea because then people can look up that single commit and see the changes done to make it work in network games.

 

[Kailric]

I have encountered another issue (so soon) when looking at the canAutoCrossEurope function. We use that function to set a unit traveling towards its destination Europe Plot. In that function it makes a check for "Plot->isEuropeAccessable()" which basically checks the distance to the Nearest Europe plot. But sense we have multiple Europes, aka Trade Screens, we need a way to differentiate between the Europes when making this check. Again, I can add a new int Array to Plots that records the distance to each Europe when this calculation is made. I don't think a bitmap array can be used for this right? I don't know much about bitmap arrays, I seen your post (Nightinggale) about that and it looks promising, but I would need you to set it up.

 

[Kailric]

The first thing, which comes to my mind is to make a loop of access points and then add a button in Europe for each available access point. The player can then click the point he wants the unit to arrive in. However that would result in 6 buttons if we use both east and west and we split directions into 8. That might result in a graphical issue, but at least the player can control where the unit arrives. We can also make a button for "the plot it left from".

I thought about this too for games like 2071 where you should be able to leave from one Space Port and arrive at any other you have discovered. On Medieval Earth your options would be more limited. To deal with graphical issues we can add in a "Select Destination Port" button where it gives you a choice of Ports to travel too.

Maybe this needs a new variable and maybe it doesn't. I seem to remember something about the game setting the plot pointer right away. Once the travel counter is 0, then the unit is added to the plot it is pointing to. I wonder what would happen if you place a pirate on that plot while a ship is in transit.

Well, the way I understand it, when the unit leaves the Map to travel to Europe it is still on the same plot, just not visible on the map. If you have units in Europe and open World Builder it shows you where they are on the map with colored plots. If you select to Travel to the West it then has to change the Units plot. It dose this by looping through all plots to find a West one, then it finds the best West plot that is closest to one of your cities, then it moves the Unit to that plot. It checks for things like Pirates. Not sure what it would do exactly if you filled the whole side of the map with Pirtes, that might be fun to watch , but I am sure that if a Pirate is sitting on the Units current plot and you command it to return home it will auto move it to a safer plot when it arrives.

I think we should postpone adding buttons in Europe for a future commit. You are affecting a huge amount of code already and tracking issues in it could be tricky. It will be ok to make a version where we for a while lacks proper control of where the unit arrives. One possible issue, which comes to mind with buttons like that is that the button clicked should be transmitted on network if network games should ever work. This reason alone is enough to make buttons a separate commit. Perhaps adding the buttons in one commit and then make them network safe in the next would also be a good idea because then people can look up that single commit and see the changes done to make it work in network games.

You maybe right, and the game is plenty functional with out it for the moment. I will concentrate on the AutoCrossEurope commands and the issue I posted above.

 

[Nightinggale]

I have encountered another issue (so soon) when looking at the canAutoCrossEurope function. We use that function to set a unit traveling towards its destination Europe Plot. In that function it makes a check for "Plot->isEuropeAccessable()" which basically checks the distance to the Nearest Europe plot. But sense we have multiple Europes, aka Trade Screens, we need a way to differentiate between the Europes when making this check. Again, I can add a new int Array to Plots that records the distance to each Europe when this calculation is made. I don't think a bitmap array can be used for this right? I don't know much about bitmap arrays, I seen your post (Nightinggale) about that and it looks promising, but I would need you to set it up.

A bitmap uses bools and the code uses int. You are touching a piece of code, which uses the int as a bool, but I figure it would have been a bool if it is only used as a bool. I went hunting for a reason for using int and found CvGame::updateOceanDistances() and CvPlot::findNearbyOceanPlot(). The update function sets distance for all ocean plots to 0, then it spreads out and stores the distance to the nearest ocean for all water plots. Water in lakes are unreachable by this spread and will have the default value, which is the one you check for (the bool usage of the int).
CvPlot::findNearbyOceanPlot() finds the nearest plot by being on a plot, then jump to the lowest scored nearby plot in a loop until it reaches 0. This plot will be the nearest Europe access plot. In other words the distance int is a pathfinding cache and I think it is a powerful one. It greatly simplifies the calculations needed to find the nearest plot. However actually going there is still done by the normal pathfinder, which will avoid enemy presence etc. (I think)

From what I can tell, we really should have this cache for all trade screens. Bitmaps will not work. Instead we should have an int just-in-time array for trade screens in each plot. I don't mind coding it, but it mean you should commit your work first. Maybe we should consider a branch or we could simply say "master is work in process" until we have dealt with this. It would be correct to branch, but I have yet to write a guide on branching

I wonder what to do with land access plots. The code here appears to be hardcoded to water movement. In fact following the logics in this approach, the entire cache should be recalculated on an island/continent if a forest appears or is removed. I will postpone even looking at this issue until it works for all water based trade screens.

Note to self: finding the nearest plot should be done locally and then goto that plot should be done with doCommand(). This way only the sending player uses CPU time to find the plot in network games and not all players. This appears to be important for land based access plots, which may need to be found without the aid of a cache.

 

[Kailric]

From what I can tell, we really should have this cache for all trade screens. Bitmaps will not work. Instead we should have an int just-in-time array for trade screens in each plot. I don't mind coding it, but it mean you should commit your work first. Maybe we should consider a branch or we could simply say "master is work in process" until we have dealt with this. It would be correct to branch, but I have yet to write a guide on branching

Ok, I will work out the other parts of my to do list and leave these two above mentioned issues alone for now. Besides, once you know where the Europe Access is, you can manually move your unit there and then the command to Travel the Trade Route will appear. I still need to pull all the changes you have already made before I can push a commit. I also still need a couple or so days to finish up a few things then we can talk more about what to do with the commit.

 

[Nightinggale]

Posting here because I'm not really sure where to put it (we should organise better)

I just committed a change in plots. The storage of which team has revealed the plot has been changed into a bitmap. This should reduce memory I/O and memory usage. The problem is that it is linked to MAX_PLAYERS, which has a hard limit around 60. To get around that I introduced a new type of variables called PlayerBitmap. This is a magic type, which is 32 now, but it grows to 64 bit when we have more than 32 players (MAX_PLAYERS, not ingame count). This mean increasing MAX_PLAYERS shouldn't break anything here (I tested it).

In semi related news I made new functions regarding loading and saving for 64 bit ints. This is quite simple as it splits them into two 32 bit unsigned ints and save those. Still it mean we can write pStream->write(64 bit int) and it will do as expected.

I added HasBit(), SetBit() and ClrBit() functions to make working with bitmaps a lot easier. I will write a wiki page on working with bitmaps soon.

I figure that PlayerBitmap can be used everywhere the code says "new bool[MAX_PLAYERS]" or "new bool[MAX_TEAMS]", which is actually quite a lot of places. I'm not sure if I want to start of that right now.

Savegames are modified (naturally), but I wrote conversion code to read old savegames.

 

[Kailric]

I have pushed my first changes to a new branch in github called Europe_CleanUp. I am not sure how someone would go about downloading it and testing it out, however, so maybe Nightinggale can fill us in. He has probably already posted that somewhere though. I have explained in Civ4EuropeInfo.xml what the new XML tags do and I list it below. This push is for testing only and does not even include any resent additions like plotgroups. Also note I have only tested the defualt M:C Trade Screens and have not tested all the new tags.

Edit: I just now realized that the AI uses any Europe plot it can get access to in order to travel to "Europe", so EUROPE_EAST isn't actually used. There maybe code that still uses the info however so it needs to remain in mods for now. The AI does not use any new trade routes yet but only the default Europe Screen, but like I said it can access it from any Europe plot it can travel too. I tested the AI with my new changes and they seem to be using it as before.

<EuropeInfo>
      <Type>TRADE_SCREEN_SPICE_ROUTE_MARKET</Type>
      <!-- This is the Command Title -->
      <Description>TXT_KEY_COMMAND_SAIL_SPICE_ROUTE</Description>
      <!-- This is the Command Help Text -->
      <Strategy>TXT_KEY_AUTO_SELL_SPICE_ROUTE_HELP</Strategy>
      <!-- This is the short name for price lists -->
      <Help>Spice Route</Help>
      <!-- No longer used -->
      <Direction>east</Direction>
      <!-- This sets the direction. Eample: If you want the Whole East side you must include E, NE, SE directions or East will only fill half the map-->
      <DirectionArrays>
        <DirectionArray>
          <Direction>DIRECTION_EAST</Direction>
          <bValue>1</bValue>
        </DirectionArray>
        <DirectionArray>
          <Direction>DIRECTION_WEST</Direction>
          <bValue>1</bValue>
        </DirectionArray>
        <DirectionArray>
          <Direction>DIRECTION_NORTHEAST</Direction>
          <bValue>1</bValue>
        </DirectionArray>
        <DirectionArray>
          <Direction>DIRECTION_NORTHWEST</Direction>
          <bValue>1</bValue>
        </DirectionArray>
        <DirectionArray>
          <Direction>DIRECTION_SOUTHEAST</Direction>
          <bValue>1</bValue>
        </DirectionArray>
        <DirectionArray>
          <Direction>DIRECTION_SOUTHWEST</Direction>
          <bValue>1</bValue>
        </DirectionArray>
      </DirectionArrays>
      <!-- If this Europe is used for Units to start from -->
      <bStart>1</bStart>
      <!-- 1 Means this route is only used by AI -->
      <bAIRoute>0</bAIRoute>
      <!-- Length of the Unit's Trip to the Trade Screen -->
      <iTripLength>3</iTripLength>
      <!-- Land must be this far away to apply the Europe. For Water Routes only -->
      <iMinLandDistance>4</iMinLandDistance>
      <!-- Used to set Land Based Routes max distance from Edge of map that land tiles will be used. If > 0 this route will be considered a Land Route-->
      <iMaxLandCoverage>0</iMaxLandCoverage>
      <!-- Sets what percent of map is used for Europe Tiles for each Direction -->
      <iWidthPercent>20</iWidthPercent>
      <!-- This Europe Requires a Tech in order to be research. Use CIV4CivicInfos.XML to set up the tech -->
      <bRequiresTech>0</bRequiresTech>
      <!-- If using one of the Below methods of Travel this must be set to 1 as Plots are not assigned a Europe with those methods -->
      <bNoEuropePlot>0</bNoEuropePlot>
      <!-- Sets where the Units can leave from -->
      <bLeaveFromBarbarianCity>0</bLeaveFromBarbarianCity>
      <bLeaveFromForeignCity>0</bLeaveFromForeignCity>
      <bLeaveFromOwnedCity>0</bLeaveFromOwnedCity>
      <bLeaveFromAnyCity>0</bLeaveFromAnyCity>
      <!-- This building is required in the City in order to Travel to Trade Screen -->
      <!-- <CityRequiredBuildingClass></CityRequiredBuildingClass>-->
      <!-- TradeScreenTypes is Currently Not Used -->
      <TradeScreenTypes/>
      <!-- This sets what screen is popped up when a unit arrives at the Trade Screen. Get the character string from CvScreensInterface.py-->
      <PythonTradeScreen>showSpiceRouteScreen</PythonTradeScreen>
      <!-- Sets the color of the trade route. The first Water route is always the players color, first land route is Purple, the rest must be set here -->
      <!-- <DefaultColor></DefaultColor> -->
      <!-- Sets the Command Button graphic -->
      <Button>Art/Interface/Game Hud/Actions/travelSpiceRoute.dds</Button>
    </EuropeInfo>

 

[Nightinggale]

I added PlayerBitmap to cities and teams. There are a few remaining places in the code, where it can be used, but I will not bother as it appears to be rarely used and memory usage doesn't really matter. I pushed this change to the main branch.

I have pushed my first changes to a new branch in github called Europe_CleanUp. I am not sure how someone would go about downloading it and testing it out, however, so maybe Nightinggale can fill us in. He has probably already posted that somewhere though.

Git guide: https://github.com/Nightinggale/Medieval_Tech/wiki/GIT-start-guide

Once you have the master branch it is just a matter of opening origin in branches (bottom left) and double click Europe_CleanUp. When asked if you want to make a local branch, select yes and yes to tracking. Switching branch might mean changing a lot of local files, which mean it isn't instantaneous.

This push is for testing only and does not even include any resent additions like plotgroups. Also note I have only tested the defualt M:C Trade Screens and have not tested all the new tags.

I consider merging master branch into Europe_CleanUp. This will add PlotGroups, but possibly more important the bitmap functions. I recall something about something, which could be a bitmap. Also merging from time to time mean merging back into master branch is likely to cause less problems as there will be less different lines.

I checked out Europe_CleanUp and it says that I have 8 modified files even though inspecting those files fail to reveal any local modifications to them. A closer look at the log reveals that you branched the 29th of december and I fixed the last line endings at the 4th of January. A merge should fix this issue. This is yet another reason for merging into Europe_CleanUp.

 

[Kailric]

A merge should fix this issue. This is yet another reason for merging into Europe_CleanUp.

Sounds good to me. I haven't made any new changes. If you merge it all then I can download the update and work from the master again.

 

[Kailric]

I recall something about something, which could be a bitmap.

Yeah, we needed a bit map for an array on plots for Europe Infos, so that we can have a function like, hasTradeScreenAccess(eTradeScreen).

I also thought of another Boolean for the XML. "bAnyEurope" so that the unit can leave from any Europe tile.

 

[Nightinggale]

Sounds good to me. I haven't made any new changes. If you merge it all then I can download the update and work from the master again.

Third time is the charm. It turned out to be tricky because you branched off from master before the line endings were fixed. Because of this a number of files had conflicts, which needed reviewing to resolve. I screwed that up two times and started over meaning I spent quite a lot of time doing that merging. However I think I have myself to blame for that one as I think I was the one screwing up line endings in the first place

Either way europe cleanup is now merged into master branch.

 

[Nightinggale]

A quick update. I added comments in the XML load functions (CvXMLLoadUtilitySet.cpp) telling which order they are called by the exe. Note that #6 and #7 is loaded AFTER main menu.

While this is a fairly small change in the source file and no change in the code itself, it did take some detective work to figure out the order which the exe use to call these functions. Hopefully we will no longer have problems with load order of XML (wishful thinking). I wonder what would happen if all XML files are loaded in #1. I wanted to simplify loading XML files, which turned out to take more planning than I first expected. I still plan to do something eventually.

 

[Kailric]

It turned out to be tricky because you branched off from master before the line endings were fixed.

Hmm, I could have sworn I had the fixed master installed before I started coding? Anyway, I just pulled the master again. I have a couple or so more tags to add for the Europe XML then I'll write a tutorial on adding new trade screens. I need to check and see if the normal Europe screen will work for the player. I believe it works for the AI. Also, if you allow the AI to autoplay you can always check their Europe screen by shift clicking the Immigration Button.

 

[Kailric]

I have noticed some issues with the current EuropeInfos system while writing the tutorial. Because of an error it is currently hardcoded at a maximum of 4 Europe types. I have fixed this but I have noticed another issue with the getStartingTradeRoutePlot function. I can't remember exactly what this does but I know it's important So, I will have to set it up to use EuropeInfo types instead of the old way. Not that big of an issue to do, however...

Edit: I think it has something to do with when you Summon a unit from a trade screen there is no predefined plot for that unit so the code uses getStartingTradeRoutePlot in order to assign a plot for the unit. So, I bet Summoning units doesn't work correctly at the moment, I do remember not testing that now.

I also notice while writing the Tutorial that we are still not out of having to edit the DLL in order to add a new trade screen. You still have to add a new CIV4ControlInfos to the xml as CvMainInterface.py uses those ControlInfos to set what happens when you click the new trade screen button.

Another problem is that all CIV4ControlInfos have an HotKey assigned to it as well. I would have to somehow work it out so that you can set the HotKey in the EuropeInfos, if that's possible. You wouldn't necessarily need a HotKey as you could just click the button.

However, it looks like it isn't possible to pass an argument from Python to the CvGame::doControl function in order to tell the DLL what EuropeInfo you are referring too, that way you would only need one Control type for all the EuropeInfos. So, Nightinggale, can we add arguments to the CvGame::doControl? I'm sure we can but just not exactly sure how to go about it.

 

[Kailric]

I just made another push to Europe_CleanUp and if you (Nightinggale) could have a look at my issue I am having that would be great.

I am attempting to fix the getStartingTradeRoutePlot functions but when I add in GC.getNumEuropeInfos() to m_aiTradeRouteStartingPlotX it causes the game to crash before the world can be built. There is some issue there that I am unaware of. I change the GC.getNumEuropeInfos() to NUM_YIELD_TYPES just to test things and the game will start like that so it's an issue with the GC.getNumEuropeInfos() being used there obviously. I didn't have an issue with TradeRouteTypes just when I went to fix the starting plots.

m_aiTradeRouteStartingPlotX = new int[NUM_YIELD_TYPES];
m_aiTradeRouteStartingPlotY = new int[NUM_YIELD_TYPES];
m_abTradeRouteTypes = new bool[GC.getNumEuropeInfos()];

 

[Nightinggale]

I also notice while writing the Tutorial that we are still not out of having to edit the DLL in order to add a new trade screen. You still have to add a new CIV4ControlInfos to the xml as CvMainInterface.py uses those ControlInfos to set what happens when you click the new trade screen button.

I need to take a deeper look at this. However I think we can avoid hardcoding.

Another problem is that all CIV4ControlInfos have an HotKey assigned to it as well. I would have to somehow work it out so that you can set the HotKey in the EuropeInfos, if that's possible. You wouldn't necessarily need a HotKey as you could just click the button.

I think CvMainInterface.py an be set to add hotkeys from XML. I will look into it eventually as I have an idea of how it might work. I think we should deal with hotkeys last as we can make a working solution without them. There are more critical parts to attend to.

However, it looks like it isn't possible to pass an argument from Python to the CvGame::doControl function in order to tell the DLL what EuropeInfo you are referring too, that way you would only need one Control type for all the EuropeInfos. So, Nightinggale, can we add arguments to the CvGame::doControl? I'm sure we can but just not exactly sure how to go about it.

We can provide the info without adding an argument. The code is made to forward eControl, which is technically an int and all we need is an int if we are a bit creative.

CONTROL_EUROPE_SCREEN,
CONTROL_EUROPE_SCREEN_MAX = CONTROL_EUROPE_SCREEN + 31,

This will give us 32 trade screens.

int iIndex = 0;
if (eControl >= CONTROL_EUROPE_SCREEN && eControl <= CONTROL_EUROPE_SCREEN_MAX)
{

iIndex = eControl - CONTROL_EUROPE_SCREEN;
eControl = CONTROL_EUROPE_SCREEN;​

}

This way we can call the function with the argument (CONTROL_EUROPE_SCREEN + index) and split it again without messing with number of arguments. We will hardcode to max 32 screens, but we will do that anyway if we make bitmap(s) elsewhere to keep track of screens. I think 32 should be enough for any sane game setup.

We might get an issue with NUM_CONTROL_TYPES though. It depends on how it's used. Another potiential issue is that some enums are hardcoded in the EXE and we can't change the content, but we can add to the end.

I just made another push to Europe_CleanUp and if you (Nightinggale) could have a look at my issue I am having that would be great.

I am attempting to fix the getStartingTradeRoutePlot functions but when I add in GC.getNumEuropeInfos() to m_aiTradeRouteStartingPlotX it causes the game to crash before the world can be built. There is some issue there that I am unaware of. I change the GC.getNumEuropeInfos() to NUM_YIELD_TYPES just to test things and the game will start like that so it's an issue with the GC.getNumEuropeInfos() being used there obviously. I didn't have an issue with TradeRouteTypes just when I went to fix the starting plots.

My guess is that GC.getNumEuropeInfos() returns 0 because the map is made before all XML files are loaded. I'm currently working on getting ALL XML files to load before main menu. In fact I want to load them before players are allocated. This turned out to be more tricky that I though, but I'm not giving up just yet.

Try to add FAssert(GC.getNumEuropeInfos() > 0) and see if it asserts.

 

[Kailric]

This way we can call the function with the argument (CONTROL_EUROPE_SCREEN + index) and split it again without messing with number of arguments. We will hardcode to max 32 screens, but we will do that anyway if we make bitmap(s) elsewhere to keep track of screens. I think 32 should be enough for any sane game setup.

I've already added one new control type for the Tutorial I wrote, still not sure about the method you mention. If we did as you mentioned then in python when you do the "WidgetTypes.WIDGET_ACTION, gc.getControlInfo(ControlTypes.CONTROL_EUROPE_SCREEN).getActionInfoIndex(), -1 )", what would you put for the ControlType?

My guess is that GC.getNumEuropeInfos() returns 0 because the map is made before all XML files are loaded. I'm currently working on getting ALL XML files to load before main menu. In fact I want to load them before players are allocated. This turned out to be more tricky that I though, but I'm not giving up just yet.

Try to add FAssert(GC.getNumEuropeInfos() > 0) and see if it asserts.

I seemed to have worked it out in my latest push to Europe_CleanUp, as it's all seemingly working now. You can summon and buy ships now and they appear on the map where they should.

 

[Nightinggale]

I rewrote how XML files are read. It now works as follows:
Read order 1: (referring to the 7 XML read functions)
All XML files are read. However since the exe is not yet ready to handle everything, only BasicInfo is read. This mean we have the vectors full of mostly garbage.

Between read order 2 and 3:
exe allocates memory for players. Just-in-time arrays are now allocated correctly as functions like GC.getNumUnitInfos() returns the correct number. The content can't be used, but allocating memory for the correct length works.

Read order 5:
All XML files are now read again. This time everything is read. As the index of the read data is already known, it overwrites the data read in read order 1. We had a bug with a unitInfo being overwritten once, but now we do the same intentionally.

The neat trick with this approach is that we use pXML->FindInInfoClass(szTextVal). I used to think it could only be used on data already being read. This is not true. It can be used on everything where BasicInfo is read while reading the rest of the data doesn't matter. This mean by the time we start to use pXML->FindInInfoClass(szTextVal), we are in step 5 and after step 1 we have access to everything. Because of this XML file order should no longer matter. There are a few exceptions, but odds are that we will never run into those.

Another neat thing is that now all XML files are loaded before main menu. This gives the option to provide a better cache for getDefineINT() as it can always be set before it is used. The concept of having some being set after main menu hurt quite a lot when setting up the cache.

I tested it a bit and pushed it. Please see if I broke anything.

This wasn't that hard to code, but it was a huge detective task to figure out how it works and come up with a plan to interact with it. One major issue was that players are allocated before hotkeys can be set. This mean the order in the exe clashes big time with my intend to get the needed sizes for the arrays before allocating players.

Now I should be free to look into what Kailric is doing.