Discussion: how should trade routes work?

Nightinggale

Deity
Joined
Feb 2, 2009
Messages
5,282
The current trade route system is a tweaked version of the vanilla trade routes. This means they are moving one type of yield from A to B and not really considering distance etc. It works ok for small players, but it doesn't scale well.

Now if we want to remake trade routes from scratch using new code, new GUI etc and we don't care about savegames (WTP 2.8 will break savegames anyway), then how should trade routes work?

We need something, which is easy and intuitive to set up for the player, but at the same time feels like the units move stuff around in a non-stupid way. Something like the perfect balance between fully automated, micro management and not being stupid like using a transport unit with 4-6 cargo slots to transport 5 tools.

A nice bonus would be to have exchange points like a small and slow transport can drop off in a hub colony where large transports with say 6 cargo slots fill up for long distance travel. Think 4 colonies feeding one colony, which then feeds to a far away port where the cargo is shipped to Europe. Ideally this should work in both directions as in cotton in one direction and tools in the other direction.

Any ideas on how this could work? Any ideas on what it should or shouldn't do?
 
A grieve that I have withl all atutomated moving units is that they move through yet undiscovered (black) lands, as -as far as I understood it- the undiscovered land was considered to have the lowest movement cost and so was considered to be an easy way compared to the already explored e.g. hills and forests.

Automated transports/wagon trains should only move through already explored areas and never try to move into or through undiscovered land.

And if we want to further limit it, require that the cities need to be connected by at least the lowest level of road to have fully automated transports to limit the amount of calculating the closest/shortest route.
 
A grieve that I have withl all atutomated moving units is that they move through yet undiscovered (black) lands, as -as far as I understood it- the undiscovered land was considered to have the lowest movement cost and so was considered to be an easy way compared to the already explored e.g. hills and forests.

Automated transports/wagon trains should only move through already explored areas and never try to move into or through undiscovered land.
Good point. I wrote a new ticket to remember a way to implement a way to control the movement cost of unexplored plots.

And if we want to further limit it, require that the cities need to be connected by at least the lowest level of road to have fully automated transports to limit the amount of calculating the closest/shortest route.
I thought about that and I wrote code in Medieval Conquest, which will ensure that colonies knows if they are connected by road even without actually using pathfinding. It's somewhat a copy of plotgroups from BTS, but heavily modded to match the needs of Colonization.

The problem with this approach is that it doesn't help to fix the problem that trade routes scale very badly with the number of colonies. If you have 15 colonies all connected by roads, they will be just as inefficient as they are right now. It won't try to fill the transports (it can, but it's not trying hard) and it's point to point while ignoring distances. Need 20 tools for your construction? The current code is perfectly happy picking up those tools 50 plots away while the colony 7 plots away has plenty of tools to export. Making sure colonies should be connected by road does nothing to fix that problem.
 
Here's my idea. Every city has an "upstream" city, which it uses to pick up goods and place "orders" for things it wants but which the upstream city doesn't have. A transport would be assigned to a single city, and would carry orders on behalf of the city it's assigned to. Each city keeps a list of goods it needs and wants, and these orders trickle up to your primary export city and the goods are carried back down to your destination cities. Here's some pseudocode; all the numbers are just guesses and would have to be tweaked for performance:

On game start, each city creates a list of "required," "desired," and "export" goods for itself. This is List A. Each city also performs the "find upstream city" routine below.

Every 5 turns each city decides which other city is "upstream" of it:

if I am a port city:
if there is another port city within 10 tiles, which is "larger" than me:
the largest of these is my upstream city;​
else:
There is no city further upstream; I am an "export to Europe" city;​
else (not a port city):
if there is another city within 10 tiles, which is "larger" than me:
the largest of these is my upstream city;
(increase to 20, then 30, etc., stopping at a value appropriate for the current map size, and redo this check each time);​
else (no larger city is found within a reasonable distance):
the nearest port city is my upstream city;​

Each turn the city does the following:

decrement my "dirtyness" counter by one;
if my dirtyness counter is <= 0:
create a list of my "required" and "desired" goods (this is List B);
find the difference between List B and List A;
if the difference is positive:
subtract half the difference from List A;​
else replace List A with List B; delete List B;​

if there is a transport unit currently within, and assigned to, me:
place any export goods on the transport;
"attach" my list of required and desired goods to the transport;
send the transport to my upstream city;​

Each turn, automated transports do the following:

if I'm in my assigned city:
unload all the goods I'm carrying;
receive the list of required and desired goods from my city;
go to my upstream city;​
if I'm in my destination city:
drop off any goods I'm carrying;
if the city has anything on my required goods list:
pick them up;​
else:
add them to this city's List A
reset this city's "dirtyness" counter​
do the same for desired goods
leave this city and go back to my assigned city​

The final product of this should be that upstream cities build up a list (which decays over time) of goods required by their downstream cities, without having to actually keep track of what's downstream of them and which specific city needs what -- they just go "I have recently been asked to provide X tools and Y stone, which I didn't have," and then send their transport to an upstream city and try to get those things. Alternately, a city downstream of them may be exporting these goods, in which case they'll just arrive and be picked up by other downstream cities. I think I may have missed a step in my algorithm that accounts for this: you'd have to compare your current inventory to your desired/required list and only request goods you don't actually have. Or maybe when a transport drops goods off, you compare the delivered goods to your list and if you don't need them, they get added to your export list so they'll be sent upstream.

The "upstream city" selection doesn't have to be done every turn, since the topography of the flow of goods doesn't change that much; cities added later seldom grow (in my experience) to be larger than cities that were founded earlier. It does happen sometimes though, so recompiling this list occasionally will be necessary.
 
Another way that doesn't involve upstream / downstream cities would be to build a queue (updated each turn by urgency, however you define that) of required goods in every city. When a transport becomes available, it pulls whatever's at the top of the queue and "accepts" that delivery. It then gets a list of all cities sorted by distance from the destination city which also export the required good, picks the nearest city that will have enough extra by the time the transport arrives, and goes there to pick up the goods for delivery. The delivery gets popped off the queue, the export city gets flagged as X of the goods in question being spoken for, and Bob's your uncle. Of course if the transport dies or gets stopped due to danger on the way, it has to push its delivery back onto the queue. This also requires a way to calculate future goods availability, which obviously may change for any number of reasons, but in 90% of cases it should be good enough and if the transport arrives and the goods aren't ready it can either wait if they'll be ready "soon" or redo the search to find a better export city.

Sorry if that's not clear, I'm super tired and can't be bothered to write out pseudocode right now.
 
I basically manually use a principle similar to Oberons suggestion, using hub cities.

However, larger cities are not always the better hubs. Especially when you have pirates afoot, your biggest port may be 9 spaces from high seas, leaving you exposed. Sometimes, depending on geography, my "export to europe" port never gets very big. May do nothing but feed itself and build warehousing/docks/defenses.

Honestly if I could just have the trade route system from original Col, adapted, I would love it and use the hell out of it. Only differences I'd want would be a "pick up X amt", and "always leave X amt", and "unload X amount", and "unload up to X amt in colony" function
 
Honestly if I could just have the trade route system from original Col, adapted, I would love it and use the hell out of it. Only differences I'd want would be a "pick up X amt", and "always leave X amt", and "unload X amount", and "unload up to X amt in colony" function
I have been thinking something similar. However it shouldn't be 100% like the classic Colonization. For starters, it shouldn't be limited to just 4 colonies.

It could be something like making trade route plans (or whatever we call them). The interface could be like the domestic advisor in the sense that it's a list of colonies. Maybe two, the top one being your plan, the bottom one being a list of your colonies. You can then alter the route with drag-n-drop of the city icons.

Reusing some DA code, the bottom would be showing the colony location on the minimap as well as what is in the warehouse.

In the simplest approach, a transport can be assigned to the plan and then it goes "now I'm leaving #2 in the list. Look up #3 and go there". Multiple units can be assigned to the same plan. Plans can be assigned a name.

The question is how to handle cargo. Drop off to fill warehouse to X. Pickup if warehouse has more than X. However that will fill up the transports with something, which has nowhere to drop off, or at least risk doing that. This means picking up something should loop through the list to figure out if there is somewhere to drop the cargo. Maybe add some on/off setting for "drop off excess here" would be nice, like some resource can be moved around, but if nobody request it and there is excess somewhere, drop it off at the port. If a colony request the cargo, do not drop it off at the port even if the transport visits the port first.

Look ahead. If the route goes A->B->C->B, then if the transport is in B and heading for C, only pick up stuff until it reaches B again meaning it won't pick up anything for A. There is no point in moving anything from B to A though C, particularly if it takes up room for anything going between B and C.

Make cargo groups. Allow the trade route to pick up "raw materials" and it will pick up tobacco, cotton etc. This will make the setup easier and if you pick up raw materials everywhere, the transports will automatically pick up the fur the natives gave you even though you didn't explicitly added it to the route because you produce nothing.

Add some sort of template colony settings. If you have multiple colonies following the same orders, you can make the settings once and then tell each colony which role it should play. You can then change the settings in one location and all your routes are updated. You can also copy those settings to your route and modify them or make them from scratch. It's a way of reducing micro management without removing the ability to configure everything as you would want it.

If the trade route plan decides what to load in each unit rather than leaving it up to the unit, maybe we can exploit knowing where the other transports are. If you need to pick up something from either A or D and drop it off at E, let the plan decide that the next unit to visit D will handle the task, not the unit arriving in A earlier the same turn.

If coded correctly, we can add all of this one step at a time and stop wherever we want when we feel "it's good enough now".

Any other ideas on how to manage this?
 
The system that you have proposed sounds really good. I really respect that you and the team, are willing to explain and discuss your reasoning and ideas.

My request would be for a clear ingame tutorial for automated trade, as at the moment the current system is a little opaque. I can imagine that some people who play the mod are either not aware or don't make use of it.
 
I completely forgot about the 4 city limit on the old system. You're right that's wont do lol. Other than that I really like what you've described.

And a tutorial would be great. I am pretty sure this is the most complex aspect of the game
 
The current trade route system is a tweaked version of the vanilla trade routes. This means they are moving one type of yield from A to B and not really considering distance etc. It works ok for small players, but it doesn't scale well.
Indeed, and you're right to point that out as I believe it's a game breaker. What I enjoy in RAR/WTP is to grow extensive and complex trading system, but if you're getting too ambitious, you reach a point where it just doesn't work anymore which just kills the point of continuing to play.

We need something, which is easy and intuitive to set up for the player, but at the same time feels like the units move stuff around in a non-stupid way. Something like the perfect balance between fully automated, micro management and not being stupid like using a transport unit with 4-6 cargo slots to transport 5 tools.
What is a bit awckward in the current system is that trade routes need to be managed at 2 levels:
1- At city level: you need to enable imports and exports, that you can also fix with a minimum of a certain good to keep in the city and a maximum that you can unload in it.
2- At transport unit level: you need to give orders to what you import and what you export.

Instinctively, I would think it would probably be easier to manage if everything was operated at a single level. I'm not sure that's possible, I'm only brainstorming here. In most Transport Tycoon games, routes are strictly determined at the transport unit level. They could follow a list of orders such as:
  1. Load tools (keep 100) and stones (keep 50) in city C
  2. Unload tools (max 100) and stones (max 50) in city A
  3. Load furs (all) and tobacco (all) in city A
  4. Unload furs (max 200), tobacco (max 200), tools (max 100) and stones (max 50) in city B
  5. Load coats (all) and cigars (all) in city B
  6. Unload furs (all), tobacco (all), coats (all), cigars (all), tools (all), stones (all) in city C
Then the cart will follow every order one after the other in a loop. In unloading all yields in the order #6, you would guarantee that no yield would get stuck in the transport unit without ever getting unloaded (even though that means unloading tools that you will load again right after, it doesn't matter).

From scratch, I see four components to determine an "order":
  • Nature of the order: load / unload. We could also imagine that could be "go to" if you want your cart to take a specific route without loading or unloading anything.
  • Concerned yield: could be any yield (obviously only for loading and unloading)
  • Command specificities: that's maybe the most technical but probably the most useful. You could for instance:
    - specifies the max number of yield you want to load
    - wait for a certain amount of yield to be loaded before leaving
    - specifies the minimum number of yield you want to keep in the city warehouse
    - specifies the max number of yield you want to unload
    - wait for the warehouse to be empty enough before unloading (to avoid waste)
    - specifies the max number of yield you allow in the city warehouse
    - if nothing is specified, it loads or unloads everything it can (all)
  • Location: where the order is executed. The could be a city, Europe, Africa, Port Royal and why not any tile which would be specified if that really serves a purpose.
If we are enable to easily specify such orders at the transport unit level, we may totally remove all the management aspect at the city level, which I believe would make things more simple.

What you guys think about it?
 
First of all - thank you very much for your continuous great work on this mod, it is absolutely glorious. I, regretfully for many years only used TAC, and only a couple of month ago found out about RaR and WTP. Thank you, thank you)

Preamble regarding some of the suggestions for transport automation - I have little experience with RAR / WTP (around 100h so far), still have some issues with how "export" settings work, and generally use a lot of micro-management (this will reflect in my suggestions):
  • I have none to very limited programming understanding, so these solutions may actually make the routing system worse / slower (i.e. I do not know how the actual calculations work and how these will impact, this is more of a theoretical concept of improvement)
  • I will also use current system (where export is set only on City level, and not on Transport level)
  • all the suggestions are tied to the example of my current biggest campaign (80+ hours), huge (biggest) map, 2-plot cites, I control around 1/3 of the continent, 40+ cities, 60+ carriages and 1500+ total of my units - load times (freezes) on turn start (when transports are calculated) are around 30-40 sec (with a strong PC).
  • 3-speed "connetoga" takes up to 10 turns to travel from South to North on a fully "paved roads" network (i.e. around 120 - 150 plots distance or more)

The things I think might help reduce the amount of route calculations (lessen route planning times) and make the system more controllable (i.e. automated system doing what you want and not carrying around 30 wood for 50 plots):
  1. Create "Max distance (X plots)" for both (ideally) "Import" and "Export" City level settings.
    • The idea is that you yourself understand how the regional / global country economy should function, and by this setting create "local economic regions".
    • Example 1 - 4 cities exist in the distance of 5 plots around the City you are currently setting "Max distance" for. By setting the "Import" to "Max distance - 5", the automated transport will ONLY consider Import resources (set to Export in those cities) as valid possibilities for transportation (this, in theory, should allow a transport to consider 4 cities as possible options, and not all 40+ cities you have, therefore lowering calculation time)
    • Example 2 - you've created a specialized city (say extracting stone in mountains) that is on the border of your territory (far away from production centers, say 50+ plots). You want its stone to be exported to productions centers at all times, but you want to receive basic resources (food + wood) only from the near-located cities (5 plots). You set "Max export distance = 50" (to ensure stone is being send far away) and "Max import distance = 5" to only get the needed goods from closest cities.
  2. Create "Priority level" (ideally, per import / export setting) - 1 (lowest / default) - 2 (medium) - 3 (highest).
    • The idea is that you yourself understand what is the priority of each specific resource produced, and want to set what things the transport system should transfer first, and what can wait.
    • Example 1 - you want to develop your new cities on the frontier (50 - 100 plots away from your main production cities) and for that you need a lot of tools for buildings, while you also want to move Livestock there, with buildings being more important. You set "Import" for Tools and assign it the highest "Priority level = 3", and "Import" for Livestock with "Priority level - 1". The transport system will prioritize moving Tools in such cities where possible, and only moving Livestock if there is extra space / Tools Import threshold is reached.
    • Example 2 - you've created several new "plantation cities" which produce Cotton, Tobacco and Hemp. You currently have several functioning Weaver shops (using Cotton), but is still building the Tobacco shops and Port infrastructure, so you want to start receiving Cotton ASAP, and Tobacco & Hemp "later, when possible". You set "Priority level = 2" for Cotton, and "Priority level = 1" for Tobacco and Hemp (you are not using Level 3 highest priority, since that is reserved for crucial things like fast movement of Tools, Weapons or Produced Goods).
  3. Creating a setting "Override storage capacity limit" to not include full warehouse (12k+ goods) in "Export" limits.
    • The idea is that if City is set to 'override = yes" and has active "Export" requests (with Max Export amount not fulfilled in city), transports would still export resources & goods to this city as usual and will not lower its priority VS other cities that still have space).
    • Example 1 - you have (I currently do) a "main import city" located closest to Europe ocean transfer zone (and with a good / big warehouse that can hold 12k & sell goods over the storage limit for 90% of their price). You want, for speed of export to Europe + lowest "waste" (over the storage limit losses) percentage (90%) to direct all Europe-bound exports to this city, but it normally has 10-12k of goods stored and when it does - automated transports start to de-prioritize it due to being over the limit (in red). You set the "Override storage capacity limit = YES" tickbox, and now, if there is Export order, its Maxim threshold is still not reached in the City - automated transport system will send all resources and goods as normal (not devalue it in favor of cities with warehouses that still have space).
  4. Transports should, instead of calculating the "global" demand & supply in your whole country, concentrate on the nearest city they are to (if they are empty and not already carrying resources to destination).
    • The idea is that each individual transport assesses the "Max distance" and "Priority level" of its closest city (maybe with "scan area" depending on transport speed, i.e. 5 plots for 1-speed oxcart, 20 plots for Wagon, 100 plots for 4-speed "stagecoach", PLUS the state of the road infrastructure as modifier - none, dirt road, paved road), and thus having the smaller set of Import / Export routes to pick from (ranging by their Priority). Upon fulfilling that order / destination from City A to City B, transport will then consider City B as its source of transport orders for the next turn.
    • Example 1 (based on the system above) - Wagon (speed 2, "scan area - 20 plots") is starting its turn near City A, where City A has "max import distance = 10" and Export order for Tools (Priority 3). Wagon will then find, within 10 plots, a city B that has Tools and Export order for them (and with amount > minimum "keep" threshold), and pick it as it destination ("order - pick 100 Tools in City B"). Should there be space in Wagon (1 slot), Wagon will include the lower priority (2 or 1) resources from City B that are needed in City A (if Import / Export orders and thresholds in Cities A and B are set and if such resources are available), and add them to its standing "order" ("order - pick 100 Tools in City B for City A" AND "order - pick 100 Livestock in City B for City A")
Upon writing the Point 4 I've seen that I do not clearly see for now how to make the system agile enough to change "order" cities (if transport goes A > B > A it will then again start with A), so this is surely not a finished idea (maybe Priorities system + Max distance should be more complex and allow inclusion of additional Cities, like Wagon does "City A Priority 3 order from City B" + "City A Priority 2 order from City C").
I personally would most be glad to see something like "Priority" system, so I can set that my Tools & Weapons should in all cases be transported before common wood or tobacco (which can wait).

Anyways, I hope that I was able to describe clearly at least some of these ideas, and something from this might be helpful (at least for theory / planning).
Thanks and apologies for the long post.
 
Instinctively, I would think it would probably be easier to manage if everything was operated at a single level. I'm not sure that's possible, I'm only brainstorming here. In most Transport Tycoon games, routes are strictly determined at the transport unit level.
I like this idea, though I wouldn't copy the approach from Transport Tycoon specifically. Routes should be stored in the player rather than a unit because then the player can make one and assign multiple units. Updating what to pick up somewhere will then apply to all assigned units. Also units shouldn't be assigned to routes. Selection groups should be assigned to routes. A selection group can consist of one or more units. This means it will be possible to make one group of say 3 wagon trains and 3 dragoons and then they will travel together. It could be useful if you have two groups of colonies and need to link them with a route through "wild animal land". It could also be useful for protecting transport ships from pirates. We might even consider allowing non-transport units to be assigned to routes and that way have automated guards patrolling certain areas.

Location: where the order is executed. The could be a city, Europe, Africa, Port Royal and why not any tile which would be specified if that really serves a purpose.
We can control the code, which draws colored circles on the map. Not only is it not restricted to just circles, we can control the RGB values of the color. In other words we have the ability to place markers on the map. We also have the ability to store/save whatever data we want, meaning we can add the ability to save waypoint location, name and whatever else we want to. I see no technical reason why we shouldn't add the ability to use whatever plot we want as route waypoints. This could be very interesting if armed guards can travel automatically between a number of waypoints rather than just colonies.

Note: routes mentioned here has nothing to do with the existing CvRouteInfo, which already exist in the game.

I have none to very limited programming understanding, so these solutions may actually make the routing system worse / slower (i.e. I do not know how the actual calculations work and how these will impact, this is more of a theoretical concept of improvement)
Nobody considers performance yet. First step is to figure out which user experience we want. Implementation comes later where the concept might be modified for performance reasons, but it's not something we should consider yet. Premature optimization is usually not a great idea.

all the suggestions are tied to the example of my current biggest campaign (80+ hours), huge (biggest) map, 2-plot cites, I control around 1/3 of the continent, 40+ cities, 60+ carriages and 1500+ total of my units - load times (freezes) on turn start (when transports are calculated) are around 30-40 sec (with a strong PC).
Can I get that savegame. It sounds like it could be interesting for testing performance code changes.

Create "Max distance (X plots)" for both (ideally) "Import" and "Export" City level settings.
Interesting idea. We can expand on that and add a "needs road connection" checkbox and perhaps a "same island" checkbox. All those checks already have implementations for fast execution (the road check implementation is in Medieval Conquest, but can be copied).

I also thought about something similar as in making groups of colonies. You can then assign a transport to only transport when both pickup and dropoff colonies are in a certain group.

Create "Priority level" (ideally, per import / export setting) - 1 (lowest / default) - 2 (medium) - 3 (highest).
Currently all considered transportation tasks have a score assigned and automated transports will pick the one with the highest score. Adding something like this to the score assignment sounds very realistic.

Thanks and apologies for the long post.
Sometimes posts needs to be long. Otherwise it can be hard to share ideas.
 
Please see the savegame attached. Looking at it it seems that I've over-stated a bit the amounts (esp connetoga speed S - N), but I still hope it may be helpful.
 

Attachments

  • 116.ColonizationSave
    2.5 MB · Views: 110
When finding groups of colonies, it might be useful to use statistics. If you find the distance between all cities (yes this creates a combinatorial explosion problem, but cities are added infrequently) you could get the average and standard deviation for distances. Then, for each city, you can get the distance to other cities and compare those distances against the statistics to group them into near, medium, and far away. This could be used internally for choosing destinations, etc., or you could give the user a slider when making trade routes as a way to declutter the screen.
 
The current trade route system is a tweaked version of the vanilla trade routes. Any ideas on how this could work? Any ideas on what it should or shouldn't do?

I have been enjoying We the People so much I want to help. Is there specific feedback you would find helpful? Asking for “any ideas” seems like a good catch-all, but it might inhibit discussion that sparks ideas in the implementers. I could pseudocode my implementation, but it would be based on the faintest idea of how Civ4 runs, so probably worse than nothing. As well, a player’s ideas of how trade routes could best aid their economic organization are premised on how they currently manage the scale of the logistics problem with the tools we currently have. So, you might be soliciting several solutions to several problems, rather than a framework to understand different options. I encourage whichever contributor sits down to tackle this issue not to hew too closely to the suggestions here, because any implementation is a sure improvement over the non-solution of the current system.

What follows is feedback from my perspective. You won’t find an explicit attempt to describe how a transport decides what to load or unload for the reasons above.
Spoiler :

TL;DR: I can think of lots of nice-to-have functions that help me tie multiple colonies together through Europe, but I cannot finish a game if I have to manually control carts to ensure feedstocks don't run short. This becomes a headache rather than achievement when my manufacturing yields hit ~20/turn and I have more than 5 colonies. It's not always the case I have one or two backwaters supplying raw goods to a nearby manufacturing hub, but I can work with a trade route system that allows me to specify two endpoints and the priority loads and unloads at each location.


I suppose there are two broad classes of players. Some favor trade route functionality that allows the player to integrate an entire continent into an Amazon.com logistics scheme. That is, one which has high cart or ship (transport) utilization (100/100 slots > 15/100 slots), economically efficient choice of goods ($), and transports that run from distant hinterlands across multiple stops, leveling stocks along the way, filling every valley and making every mountain low. Something intelligent enough to find goods where the player hasn’t manually specified. These players are utilizing the full breadth of the building tree, play on large and huge maps, for tens of hours. I imagine them as model train builders who like systematizing. It's a good hobby, but accommodating this playstyle doesn't strike me as what is necessary for historical economic simulation.

Of course, no one fits this description or has absurd goals for the trade route system. But I want to contrast my vision with the suggestions earlier in the thread.

The other class of player favors the minimum functionality that will let them finish a game. They neither want to manually control every transport to ensure production centers maintain minimum feedstocks, nor learn to use a new trade route system that might still drive a cart across the continent to pick up 50 “raw materials” needed no where. They infrequently use buildings past the mid-tier because winning does not require it, they play on maps smaller than large and huge, and their goals are victory conditions, not personal ones. They would be happy to have carts badly utilized, as long as they did what they were stupidly told and traveled where they be found so problems can be rectified with the ol’ manual load’n’unload. Conversely, understanding where to troubleshoot a complex route system that uses look-a-heads or upstream cities is one more hurdle in front of finishing.

My idea on what a trade route system should do is allow me to level the production yields between two colonies. In a dream world, I would produce as many transports as necessary to obviate the difference between one colony and the next. The two colonies would sit on different tiles, have different liberty ratings, and the interface would purport they have access to different tiles and buildings, but I would consider them one colony. Food grown in Colony A would be available by the end of the next turn in Colony B, the black yield (increasing stock) in one buttressing the red yield (decreasing stock) in the other. In the real world, I will program a cart to take food from the black to the red and I will not build enough carts to eliminate all distinction between their inventories, But a trade route system should let me achieve this theoretical state.

I usually want to bring raw resources from one frontier colony to one huge city without anyone missing a deadline for tools or running out of lumber and ore. I can accept if shortfalls occur because I'm not producing enough at one of the locations for how fast the other location burns through them. I'll get the alert in the event log "Colony A is running out of Y and won't be able to produce Z." I can add more transports, or substitute transports with greater capacity, with goods from other colonies as needed. I do not need a trade route system that intelligently detects shortfalls and goes to colonies outside these 1 to 1 relationships.

I expect to set up a bunch of 1 to 1 routes that overlap. Colony C would never get tools produced by Colony A, but Colony A is dumping it’s tools at Colony B, so I, the super manager that I am, can smartly make a transport bring tools from B to C. So while I’m really beating the drum for 1 to 1 functionality, I want to acknowledge that in practice what I’m proposing is probably similar to what others have earlier in the thread, just with the emphasis that I want to finish a damn game, rather than create something beautiful.

I expect to manually ship colonists and also to run a few special errands, but I want to automate the bulk shipping, the shipping that is never wrong. Even if “automation” is a highly manual process to initiate, I favor that predictability over the non-solution that is the current vanilla system. I prefer something I can set up to prevent work stoppages to intelligent transports that care how the distance to a location affects ROI.

I would like to see the net production and net consumption of each good within the colonies I select. The Domestic Advisor shows me all my colonies at once and one bottom line. I would like to uncheck colonies to see the bottom line for the two colonies I’m setting up to be a cluster. If I see a big deficit, I’ll automate a transport to rope in goods from some colony outside the initial cluster (of two). I will be happy to manually program load/unload what that new transport will carry. If I see a small deficit, I will bring one of the 2-5 transports I’m manually controlling into the cluster every 10 or so turns for the exogenous infusion of tools or whatever is needed.

This improved Domestic Advisor interface would help me see when I’m going to run into overstock problems unless I manually haul products out of the system. Here also the native event warnings will help, “Colony A is running out of capacity and will start hemorrhaging guns, gold, and tools tomorrow!” The timeliness of these warnings decreases as your production and imports grow, unfortunately. The dream version of this interface would let me click a right arrow to project stocks—including automated imports and exports—at each selected colony, one turn into the future. Another click would let me see two turns into the future. Another click would let me see three, and a colony will turn red when it his overcapacity, so I can send manual transports in time.

But it has to show all the goods on one page. I don’t want to click right right right, to see stone, and left left left, to see tools. The Domestic Advisor requires this. If there is no interface solution for this, I would prefer you remove good types, buildings, and terrains than make me click left and right.

I'd love if you warn the player when the net consumption of a good exceeds the net production on a route, but I'm fine watching my stocks run low and just making the necessary tweak to have net stocks grow. I expect to tweak the target levels of important goods at each colony in a cluster every few turns. Afterall, each turn terrain improvements are finished, buildings complete, and colonist productivity increases.

If the system does have the relatively dumb A-to-B functionality I propose, please let me click once to "assign all goods to export down to 0." I will manually change the food, tool, horse, etc. target to 50. But I don't want random gifts from Natives, errant pirate plunder, and other goods to accumulate because I didn't go out of my way to export furs from a colony with no forests.


Priorities:
  1. Colony-based minimums. I need to reserve quantiles of goods for that colony. I am happy to define this numerically, for each colony, without a template.
  2. Cart or ship remains where you might expect it: end point, start point, or in transit between them
  3. An interface that helps me see the net production of all goods between as few, or as many colonies at once. Or rarely all colonies at once, as the Domestic Advisor does.
Not Priorities:
  • Transports that know about future trips
  • Transports that know about other transports
  • Transports that know about colonies other than mom and dad
  • Good cart or ship capacity utilization ()
  • Economically efficient choice of goods ($)
  • Guard units
  • Waypoints


If it makes implementation easier or quicker, assume:
  • Player wants to automate the shipping that's never wrong, not all shipping
  • Player won't set up routes that supply insufficient production of any good for the consumption of that good within the trade route. If he does, the event log will alert him about it.
  • Player cares most about avoiding overcapacity losses
  • Player cares less about economic efficiency than a colonist he placed in a building running out of what that colonist consumes
  • Player wants trade routes to hoover up all the errant goods across his empire.
  • Player manually controls export from 2-3 major port colonies to Europe, Africa, and Port Royale.
  • Cantankerous-to-use > difficult-to-understand > current system
 
Last edited:
  • Player cares less about economic efficiency than a colonist he placed in a building running out of what that colonist consumes
I'm not a UX designer, but if I was, this is when I'd ask the user to designate the colony where they expect the tobacco for that tobaccoist to come from. <---Suggestion for a system with smart transports, needs lists, or delivery queues, not the barebones thing I'd accept.
 
One thing that would help immensely is to have profiles for Governor Import and Export.

Another thing that should help is to assign "sister cities" to get around the problem where an automated Wagon travels for 25 turns on some city you just took from the enemy. This coincidently also increases the speed of the Wagon "order".

Lastly I have to say that this aspect of the game is the single most important of all others, and the one I enjoy most, so any change is welcome!

Thanks!
 
I have to admit that I did not yet read all the content in this thread. I am presently working to implement governor import export save/load functionality. While doing that and exploring the trade route related source code, I have come up with a few rather simple ideas to implement.
  1. I implemented a default xml based governor import profile that fills storage min/max values for set yields (and coincidentally switches them to import feeder but that is easy to change). This works on my personal development version of WtP. Good thing about the xml import is that it persists across games. No need to start from scratch editing the transport limits and checkboxes.
  2. I will next try to implement (save game bound I guess) load/save for the governor import/export variables. Even better would be to load/save to external files the governor settings.
  3. Thinking about the feeder logic and longer transport chains, I came up with the idea to make the "feeder threshold" a customizable parameter. Now, it is always fixed to 75% of min limit (as far as I understand). I will explain this in more detail below.
Let's take an example. Settings for "Colony B"
  • Min storage limit 20 (keep 20 - for domestic market for example)
  • feeder threshold 50 (start import only when below 50)
  • max limit 100 (never exceed 100 when importing stuff)
Trade route graph for a simple situation (that can be extended indefinitely)
-> A -> B -> C ->
This would be a "passthrough" route where A brings stuff to B which in turn serves C etc. Assume explicit trade connections A-B and B-C. What we ideally want is that transport from A-B / B-C is not moving around little to no stuff but at the same time not too much stuff is idling in storage doing nothing. And we want to keep the stock steady and responsive to demand spikes in all positions in the chain.

The usual 75% feeder does not work well since it has so small neutral zone between importing and not, so minimum limit would have to be set high to have effect - binding a lot of resources to storage. (100 limit feeder for example would transport ~25 max goods at a time and would keep stores above ~75 at most times - and you would still need to set max limit higher. Nothing prevents small deliveries from 74->76 for example.)

Not using a feeder works ok apart from giving opportunity for very low transport quantities. E.g. 80 -> 84 import would be possible in this setup.

Custom threshold feeder as shown above would be better: less idle filler goods needed, yet decent minimum import of 30+ at a time. Even better, minimum export available goods is also 30+ when feeder is not active.

A demonstration for tools:
Starting situation: A(60 tools), B(60), C(60).
A static situation in the feeder case. No need for transportation.
Then, B starts to construct a building demanding 20 tools. B is set to (40) in feeder calculation. This triggers A->B transport of 40 tools, pushing B to (100-20 = 80). This in turn triggers import to A from its supposed connections (or makes it wait for production if it is the source city).

Another case analysis. Same static starting situation.
C starts a building demanding 120 tools. B->C will move 40. Then, A->B moves 40. And again, B->C moves 40.
And situation A(20), B(20), C(140-120 = 20). All feeders active.
Next, 80 tools are delivered to A (max single transport quantity in this setup). A->B->C moves it to C in two steps. This is not perfect since the resource spread is uneven but will get fixed with more deliveries to the chain.

I will try to implement this and try how it feels and behaves in practice.

EDIT: looking more at the feeder code I think I might have misunderstood a bit how building resource demand works with feeder int the example above. Not entirely sure though. It makes no essential difference to the overall point of my suggestion regardless.
 
Last edited:
Here is a simple change I would like to see introduced:

Currently when you bring up the automation routes for a transport unit you are presented with every possible trade destination. This can be sorted in various ways, but it can quickly become unwieldy.

If we could have an option to hide/show cities (+ or - icons) from this list it would help with micromanagement.
 
tell me why the import and export in my colonies do not work, as it should?
I need coffee in Kodiak. I set the threshold to 10, but as you can see, there are already 52 coffee in stock.
from Sitka I want to export hemp, stone and horses. but since stone is needed for construction, and horses for their further breeding, I set a threshold of 50 for stone and 100 for horses. however, hemp from the colony is periodically taken away, but horses and stones are not.
Tell me, please, what am I doing wrong?
 

Attachments

  • col_import01.jpg
    col_import01.jpg
    756.7 KB · Views: 285
  • col_export01.jpg
    col_export01.jpg
    755.6 KB · Views: 269
Top Bottom