1. We have added the ability to collapse/expand forum categories and widgets on forum home.
    Dismiss Notice
  2. All Civ avatars are brought back and available for selection in the Avatar Gallery! There are 945 avatars total.
    Dismiss Notice
  3. To make the site more secure, we have installed SSL certificates and enabled HTTPS for both the main site and forums.
    Dismiss Notice
  4. Civ6 is released! Order now! (Amazon US | Amazon UK | Amazon CA | Amazon DE | Amazon FR)
    Dismiss Notice
  5. Dismiss Notice
  6. Forum account upgrades are available for ad-free browsing.
    Dismiss Notice

Roadmap to Preview Version .2

Discussion in 'Gedemon's Civilization, a total overhaul project' started by Gedemon, Dec 11, 2017.

  1. Gedemon

    Gedemon Modder Moderator

    Joined:
    Oct 4, 2004
    Messages:
    7,489
    Location:
    France
    1 route per type, regardless of destination

    The routes efficiency is dependent of the type, and the code set the most efficient route available, but the problem is that road and river routes are much faster to get than sea routes, if we allow 3 undefined routes per city, all coastal cities will use 3 sea routes as they are far more efficient.

    I'm already keeping track of the current routes, so it's only the length of the route that need to be tested for enemy units before recalculating, but keeping track of the most efficient route between two cities is a good idea, and try to update a current route to that one first (after being changed because of enemy units) will make things faster. Still have to be updated from time to time for sea routes (because of canal cities that can be added and, obviously, roads)


    We'll need the routes to/from capitals for stability at one point or another. A centralized network was my first idea, we could go back to that. I'd need a step by step description for multiple center in the same civilization.

    Not sure that the calculation are causing the slowdown, but I'm approaching turn 250 on my current Enormous map, I'll try to put more timers in sub-functions to determine where it is exactly.

    And I would have to re-code/re-balance the whole resource system, so I hope it's not that.

    AFAIK automated units are impossible with the current modding access.

    Some computers won't like to process every players at once at then end of the game's turn, better to process each player during their turn, start is easier from a coding perspective with the available events.

    no automated units, sadly, we could do that with civ5 without the DLL (my convoy units in the WWII mod were exactly that, but also automated for the player, you had to protect/escort them with your other units)
     
    Knasp likes this.
  2. Knasp

    Knasp Chieftain

    Joined:
    Sep 10, 2011
    Messages:
    119
    Excellent, that would make coastal cities far more important, which makes sense since they're having access to more cities and probably access to more unique/exclusive goods*. You could choose to prioritise river routes over coastal/oceanic (I don't know which of these are most efficient really). If you want to, you could let each city start with 1 land trade route, connected to the capital city, or the closest friendly city.

    Instead of a limit of 1 trade route per type, or 3 per city, why not just add "trade buildings" and award 1 trade route of a specific type, per building?

    For e.g:
    • Riverine traders (requires city to be adjacent to river) = Gives 1 river trade route
    • Maritime traders (requires coastal city/harbour) = Gives 1 coastal/ocean trade route
    • Caravansery (requires adjacent land tile or city?) = Gives 1 land trade route
    Just like housing, you'd be able to construct a higher tier trade building to get an additional trade route of the same type. However, it shouldn't be more expensive to get an additional trade route of the same type, instead of a different type. Land routes are inherently less effective, so you wouldn't want to build a caravansery unless it gave access to a unique/special goods that are unobtainable from naval routes. Of course, you could always limit the number of trade routes (and trade route buildings) by the city size.

    On the other hand, a possible benefit of land routes would be to get roads, which would make it easier to establish effective systems of couriers (messengers on horseback), since such couriers could be faster than ships.

    If possible, it would be good if river trade routes could be combined with naval routes, meaning that you could trade between cities on different rivers, connected by a common sea. But since you say that sea routes take longer time to get, this suggestion would require even more processing. Also, this depends on the ships used, since only flat-bottomed ships with a low draft can (safely) navigate shallow rivers, while they would be disadvantaged on the sea where you'd want more freeboard. One solution to this would be to only let the city that is furthest downriver trace coastal routes from the mouth of the river to coastal cities/harbours.
    Yes, I suppose that new sea trade routes would need to be updated every time a new coastal city / harbour is founded. When it comes to choosing which city to trade with, for coastal/riverine trade routes at least, proximity isn't as big of a deal as it is for land routes. Rather, it is the resources that your city demands, and the resources that the other city wants, that should determine the destination.
    This would depend on whether you choose to introduce trade buildings (suggested above), and if so you could simply start with the city that has the most trade routes (being a commercial center), and if tied, you can go by total stock, then by population size, and so on. I'm not sure I can suggest something that would be easy to code or more efficient to execute than the current code, but I'll share my thoughts anyway:

    You could either restrict trading to internal trade first, and then do a second run with external trade (like the current system), or you could resolve all trade simultaneously. For each turn you'd need to check which cities that are connected to the trade network, and exclude those cities whose access are dependent on a blocked route. The excluded cities would form a separate trade network, until the blockade ends in a following turn. Within each network (whether internal cities first, or all cities combined) you could use one of the two following ways of matchmaking:
    Either go by:
    a) available goods (ability to supply) or,
    b) purchasing power (ability to consume).

    a) Ability to supply:
    1. Start by summing up every city's total stock, all goods combined. At the same time, sum up the total stock of goods that the cities demand*, i.e. the amount of goods that each city has capacity to store and use.
    2. Start with the city that has the highest total stock, all goods combined, resolve ties by starting with the city that have the least goods demanded*, otherwise pick the one with largest total population.
    3. Match the current city with the other city that have the greatest demand* of its goods. And mark those types of goods to be transferred as "locked" for further trade in both cities. (E.g. Wheat is locked in city A and B)
    4. Continue matching with other cities until all goods of the city are "locked" or can't be traded.
    5. Then go on to the next city, that has the second highest total stock (minus locked goods), and continue matching cities.
    6. Once all cities and their goods are matched, or no trade routes are available, then:
    7. Transfer goods and update stock.

    b) Ability to consume:
    1. Start by summing up every city's total stock, all goods combined. At the same time, sum up the total stock of goods that the cities demand*, i.e. the amount of goods that each city has capacity to store and use.
    2. Start with the city that has the highest total demand, all goods combined. Resolve ties by starting with the city that have the highest population or goods available for trade.
    3. Match the current city with the other city that have the greatest supply of goods demanded. And mark those types of goods to be transferred as "locked" for further trade. (E.g. Wheat is locked in city A and B)
    4. Continue matching with other cities until the city's type of goods are all "locked", unavailable for trade OR if the total demand of goods is now lower than the next city in line (OR lower than the city that started out with the lowest demand).
    5. Then go on to the next city, that has the second highest total demand (minus locked goods), and continue matching cities.
    6. Once all cities and their goods are matched, or no trade routes are available, then:
    7. Transfer resources and update stock.

    So trader units can't be modified? That sucks, but then we can disregard my suggestions regarding automatic units. Less possibilities make for easier choices.

    Good to know! I'm so used to playing with simultaneous turns in multiplayer, and I thought that maybe all CPUs did their turns simultaneously or something.

    *Goods. I use the word goods for the items that a city have in stock, or which they can stock should they acquire them through trade. I avoid using the word resources to avoid confusion with resources gained from tiles.
    *Demand. I use the word demand to mean items that a city have capacity to store and use, minus the current stock. E.g. if the stock of Item A = 20/100. Then demand of item A = 100-20 = 80.
     
    Last edited: Feb 26, 2018
    Gedemon likes this.
  3. Gedemon

    Gedemon Modder Moderator

    Joined:
    Oct 4, 2004
    Messages:
    7,489
    Location:
    France
    Thanks, I'll give some thoughts at all that.

    Some time ago, I was pondering a new improvement outside civs territory: a Trading Post, build with two restrictions: on coast and on river, that would be a waypoint for trade routes.

    I've started logging the timer, here are the total time used by the pathfinding functions for all civs on one turn (Enormous Terra Map, 30-40 civs):
    Code:
    ---------------------------------------------------------------------------
    Turn = 283
    IsPlotConnectedCoastal timer = 0.85699892044067 seconds
    GetRiverPath timer = 0.25099873542786 seconds
    IsPlotConnectedRoad timer = 0.88199973106384 seconds
    IsPlotConnectedOcean timer = 19.434997081757 seconds
    IsPlotConnectedLand timer = 10.66600227356 seconds
    
    ---------------------------------------------------------------------------
    Turn = 284
    IsPlotConnectedCoastal timer = 1.17999958992 seconds
    GetRiverPath timer = 0.28900003433228 seconds
    IsPlotConnectedRoad timer = 1.2270045280457 seconds
    IsPlotConnectedOcean timer = 20.577000379562 seconds
    IsPlotConnectedLand timer = 11.589004039764 seconds
    
    ---------------------------------------------------------------------------
    Turn = 285
    IsPlotConnectedCoastal timer = 0.65200018882751 seconds
    GetRiverPath timer = 0.44500041007996 seconds
    IsPlotConnectedRoad timer = 1.129002571106 seconds
    IsPlotConnectedOcean timer = 25.346997737885 seconds
    IsPlotConnectedLand timer = 11.008998632431 seconds
    
    ---------------------------------------------------------------------------
    Turn = 286
    IsPlotConnectedCoastal timer = 0.8240008354187 seconds
    GetRiverPath timer = 0.29300117492676 seconds
    IsPlotConnectedRoad timer = 0.95999574661255 seconds
    IsPlotConnectedOcean timer = 25.783000707626 seconds
    IsPlotConnectedLand timer = 11.871002674103 seconds
    
    ---------------------------------------------------------------------------
    Turn = 287
    IsPlotConnectedCoastal timer = 0.44700145721436 seconds
    GetRiverPath timer = 0.24899959564209 seconds
    IsPlotConnectedRoad timer = 0.87500047683716 seconds
    IsPlotConnectedOcean timer = 29.584996700287 seconds
    IsPlotConnectedLand timer = 11.215998411179 seconds
    
    Land is the supply path for units, which is not saved/cached
    Coastal is the sea trade routes for civs that haven't researched Cartography yet

    For a start, I'll try to limit the pathfinders to the tiles that are in range of a trade routes, instead of the whole map (I mean the code don't try to find a trade route between two plots that are outside that range, but for plots that are in range without a direct path, the pathfinders may actually try to go through plots that are outside the maximum range of the trade route)
     
    Knasp likes this.
  4. Gedemon

    Gedemon Modder Moderator

    Joined:
    Oct 4, 2004
    Messages:
    7,489
    Location:
    France
    Well, that escalated quickly !

    But now I have a base saved game to test optimizations on.

    Code:
    ---------------------------------------------------------------------------
    Turn = 292
    IsPlotConnectedCoastal timer = 0.091000080108643 seconds
    GetRiverPath timer = 0.52599859237671 seconds
    IsPlotConnectedRoad timer = 1.6469964981079 seconds
    IsPlotConnectedOcean timer = 112.94199919701 seconds
    IsPlotConnectedLand timer = 15.778000354767 seconds
    
     
    Knasp likes this.
  5. Knasp

    Knasp Chieftain

    Joined:
    Sep 10, 2011
    Messages:
    119
    Yes, letting the player construct trading posts would be accurate and would let you decrease the initial length of naval routes (having to rely on the posts for extending your trade enetwork, enabling longer routes). It could also give some use to those tiny 1 tile islands that you find. Though trade posts should be useable by all friendly nations (giving you a share of the profits for those trade routes).
    Escalating updates due to on and off blockades and sensitivity to other cities routes?
    Oceanic routes really seem to be the big problem, and I would guess that having so many interconnected cities, with so many routes that can be blocked, will lead to having to check new paths every turn. A city whose route is blocked could choose a different destination. While the city who was previously recieving that trade now is forced/enabled to check the route to the city that stopped its route. If the blocking unit moved, then the new route may be established, but then it could be blocked by the same unit in the following turn which would force a new update. And so I can imagine constant updating and path finding being forced upon many naval cities every turn.

    That's why I believe in doing path checking only when a city is founded (always tracing the shortest/most efficient path regardless of temporary obstacles). Then keep track of those paths and only modify/check for new paths when a new city/new tech/new policy or other event requires it.

    Naval supply lines?
    Do the timers above (isplotconnectedocean) include supply lines to naval units?

    Oceanic routes blocked on coastal tiles?
    Oceanic routes will still go through coastal tiles if it leads to the shortest path, right?
    Any I would assume that any route following the shoreline would easily be blocked.
    Can only enemy units (like barbarians) block paths?
    Whenever a path is blocked, does the code recheck paths to all other cities or only to the city whose path was blocked?

    Avoiding coastal tiles?
    Can the oceanic path finder be instructed to always prefer oceanic tiles as far as possible?
    Let the path finding start by checking the path from the coastal city to nearest ocean tile and likewise for the destination city. Then try to find the shortest path between those 2 oceanic plots, preferring oceanic tiles. That should avoid coastal tile blockades more often.

    2nd pass, global tables or a table for each city?
    Reading the "second pass" bit I'd like to to ask for some clarifications (if you have time):
    Does every city keep its own table with distances, paths and efficiency? Or is there a shared table of all cities?
    Is the out of reach table a global table, being updated whenever a city rechecks it's paths? Or is there an OOR table for every city?
     
    Last edited: Feb 27, 2018
  6. Gedemon

    Gedemon Modder Moderator

    Joined:
    Oct 4, 2004
    Messages:
    7,489
    Location:
    France
    Each city has it's own table (in a global table) same for the out of reach table.

    I've done two pass, first implement A* (GetPathToPlot functions in the table below) and compare with the old pathfinder:
    Code:
    ---------------------------------------------------------------------------
    Turn = 295
    
    IsPlotConnectedLand timer = 12.657998323441 seconds
    GetPathToPlotLand timer = 0.55500364303589 seconds
    IsPlotConnectedLand tested plots count = 1624020
    GetPathToPlotLand tested plots count = 95202
    
    IsPlotConnectedRoad timer = 1.5800004005432 seconds
    GetPathToPlotRoad timer = 0.55299925804138 seconds
    IsPlotConnectedRoad tested plots count = 265704
    GetPathToPlotRoad tested plots count = 115458
    
    IsPlotConnectedCoastal timer = 0.12600040435791 seconds
    GetPathToPlotCoastal timer = 0.002000093460083 seconds
    IsPlotConnectedCoastal tested plots count = 22308
    GetPathToPlotCoastal tested plots count = 192
    
    IsPlotConnectedOcean timer = 67.616000652313 seconds
    GetPathToPlotOcean timer = 15.197998762131 seconds
    IsPlotConnectedOcean tested plots count = 6336216
    GetPathToPlotOcean tested plots count = 1973436
    ---------------------------------------------------------------------------
    then added a check to not add plots to the possible path if they are at a greater distance of any end of the path than the MaxRange of the route
    Code:
    ---------------------------------------------------------------------------
    Turn = 295
    
    IsPlotConnectedLand timer = 11.91999745369 seconds
    GetPathToPlotLand timer = 0.26800131797791 seconds
    IsPlotConnectedLand tested plots count = 1500774
    GetPathToPlotLand tested plots count = 31830
    
    IsPlotConnectedOcean timer = 62.275001049042 seconds
    GetPathToPlotOcean timer = 7.7089986801147 seconds
    IsPlotConnectedOcean tested plots count = 5942496
    GetPathToPlotOcean tested plots count = 883314
    
    IsPlotConnectedCoastal timer = 0.10700106620789 seconds
    GetPathToPlotCoastal timer = 0.00099992752075195 seconds
    IsPlotConnectedCoastal tested plots count = 19518
    GetPathToPlotCoastal tested plots count = 174
    
    IsPlotConnectedRoad timer = 1.5199999809265 seconds
    GetPathToPlotRoad timer = 0.53000140190125 seconds
    IsPlotConnectedRoad tested plots count = 250170
    GetPathToPlotRoad tested plots count = 84636
    ---------------------------------------------------------------------------
    Now I need to check if I still get the correct routes...
     
    Last edited: Feb 27, 2018
    Knasp likes this.
  7. Gedemon

    Gedemon Modder Moderator

    Joined:
    Oct 4, 2004
    Messages:
    7,489
    Location:
    France
    I think I'm going to do the following:

    - Set specific buildings (Shipyards) to be able to stock and build naval equipment (stock a lot of "woods", makes/stock some "wooden hull parts" then makes/stock a few Galley/Trireme,...)
    - Mark naval equipment to be only stocked in those buildings and not available for trade, that will prevent over-production and waste, with maybe units-specific shipyards for the late game, only able to stock one unit of final equipment.
    - Use the "final naval equipment" (ie completed ships) as the other equipment for new units, reinforcements, upgrade...
     
    Knasp likes this.
  8. Gedemon

    Gedemon Modder Moderator

    Joined:
    Oct 4, 2004
    Messages:
    7,489
    Location:
    France
    Small update on my progress, the code for the naval upgrade seems to be working, but it has highlighted an issue with the global upgrade system, that is also visible when moving from chariot to riders for example.

    When an unit has an upgrade that use an equipment class with a different personnel/equipment ratio (for example, there is one chariot for 4 personnel in a chariot unit and 1 personnel for one war horse in a rider unit), that unit requires reinforcement in frontlines using only the ratio of its specific equipment.

    Which means that when upgrading to riders, there is not enough war horses to maintain the unit's current health and you end up with a low health unit.

    I need to change that, but it will require some important changes in the current code, and as that part of the code is quite complex, I will review it first and make a full description of an unit turn, as I've done for cities, that will allow me to have a clear understanding of my own code and what to change exactly... And how to change it because there is still the concern of keeping things fast, and using a different/variable equipment ratio for reinforcement may introduce a lot more checks each turn.
     
    Knasp likes this.
  9. heinous_hat

    heinous_hat Chieftain

    Joined:
    Sep 28, 2010
    Messages:
    576
    Location:
    under the weather
    Does the situation you describe account for the occasional event where all units are awoken from fortify/sentry status (with lowered health), requiring that you go around and reset all of them to a passive state?

    At least I think it's all units... definitely a lot of them.
     

Share This Page