Discussion: how should trade routes work?

[FlaviusBelisarius]

tell me why the import and export in my colonies do not work, as it should?
I need coffee in Kodiak. I set the threshold to 10, but as you can see, there are already 52 coffee in stock.
from Sitka I want to export hemp, stone and horses. but since stone is needed for construction, and horses for their further breeding, I set a threshold of 50 for stone and 100 for horses. however, hemp from the colony is periodically taken away, but horses and stones are not.
Tell me, please, what am I doing wrong?

This is likely not the right thread for the discussion but nevertheless.

What do you expect to happen with coffee in Kodiak? You did not set a maximum limit (the other field to the right with 0) so once you dip below 75% (7 in this case I believe), import triggers and can bring as much coffee to Kodiak as is available for export in a single transaction. And now you are over the "keep minimum" of 10 so import is turned off. The standard feeder function. What do you want to do otherwise?

For the other case, have you surely set some other colony to import horses / stone? And have you assigned transports to the respective routes assuming you handle the automation by assigning transport units to routes manually?

 

[Mr. ZorG]

it seems that the old topic had a link to this topic, so here I wrote)

I figured out the coffee myself by trial and error) thanks

I export stone and horses, not import. I created routes for an automated trading network. but this, for some reason, did not work. but when I manually created a route for exporting stone and horses (for some reason these routes were not there, when I started to find out what the problem was), everything started to work. so I would like to find out, maybe I did something wrong ?

 

[Ameranth]

Currently when you bring up the automation routes for a transport unit you are presented with every possible trade destination. This can be sorted in various ways, but it can quickly become unwieldy.

Seconded. However, instead of simply hiding cities, I'd prefer to revamp the import/export screen entirely.

I find the most efficient way to set up transport networks is to have cities feed into a nearby transport hub, which gathers similar goods and relays them to the next town in a line heading east to the ports for export. Keeping transports on short linear routes also prevents them making very suboptimal choices about where to go next if they have multiple input/output cities.

---

Under the current system of selecting a city to import and/or export, and then choosing from a list of all possible from/to cities, the length of that list grows and rapidly becomes unwieldy.
For example, a network with 3 cities feeding into a transport route to a final destination would look like:
A exports Hemp
B exports Hemp
C imports/exports Hemp
D imports/exports Hemp
E imports/exports Hemp
F imports Hemp

This will result in transports having to choose from the following routes:
A-C, A-D, A-E, A-F
B-C, B-D, B-E, B-F
C-D, C-E, C-F
D-C, D-E, D-F
E-C, E-D, E-F

Only one good in 6 cities resulted in 17 entries. Considering all the available goods being imported/exported through dozens of cities on larger maps, and you have a problem.

I'd like to see this changed so that we define transport routes with a specific from/to when created.
For the cities from the above example, the route selection screen becomes:
A to C Hemp
B to C Hemp
C to D Hemp
D to E Hemp
E to F Hemp

Only 5 entries instead of 17. While the current method will grow exponentially with each new city added, this method only adds 1 line per city.

The current functionality could still be maintained by allowing 'ALL' as a start/end point. This would work exactly like the current method does, creating entries in the transport screen for any cities with a matching from/to ALL for a good.

Doing it this way doesn't require trying to figure out all the possible edge cases to fully automate transport logic, it just requires the player to set up a series of simple routes.

---

Currently the city export screen only shows any good which has a surplus produced at that city. This is great for not having to scroll through non-applicable goods. However, this hides goods which are only temporarily in negative values, as well as ones that actually have a surplus because of imports. To make setting up trade routes easier, it would be nice if it showed all goods produced in that city, no matter what the current gain/loss is.

Additionally, the export screen should also show all goods being imported into the city. That would make it easy to set up exports for a relay city without having to check existing routes to see exactly what is going into it.

---

As a further shortcut, the route selection screen could default to sorting by city name, with all the routes from that city indented under it in an expanable list, and expand/collapse all buttons at the top. Clicking a checkbox next to each city name would select all the routes from that city.

Having a button to sort by good type would present the list as good names instead, with an expandable list under each one showing all the routes involving that good. Clicking the good name would select all those routes.

 

[Vaeringjar]

quoting myself from here: https://forums.civfanatics.com/threads/release-candidate-2-8-2-0-testers-needed.665065/

So I'm on a borderline potato laptop at the moment, and the method I'm using for internal trade routes, with my current 20 settlements, takes over 20 seconds to load the "Assign transport routes" interface, and I assume it grows exponentially with more import/export orders. Would it be possible to, when entering the interface from a transport which is in an owned settlement, to default both "export" and "import" filter to the current settlement? (then the player just has to change at least one of the 2 filters) I imagine something of this sort could reduce that load time a whole lot?

I read of the shuttle approach to trade automation... https://forums.civfanatics.com/threads/wagon-train-routing-shuttle-based-approach.382590/
...but couldn't get it to work in both directions as OP seems to have.
Still, it's working quite well. I basically got every single settlement set to import and export everything (thus my terrible 20-30s trade UI load times). Things needed for production (larger buffer if net negative) and domestic market have a minimum value set. Then there's a "shuttle" transport connecting each 2 settlements, set to import everything from one and export it to the other. Doing this thus chains settlements together, then converging at a "Hub" (like the capital). This means instead of having to micro every single settlement's trade, I only need to micro a chain itself, with occasional adjustments to minimum values, the rare manually emergency adding or removing mid-chain, sending things back from the "hub" to the first settlement in a chain (like tools or luxuries for the domestic market, which btw could also be automated by assigning a route and setting max value on the target), or upgrading from "wagons" to "treks".
I have tripled all storage values from buildings though, vanilla is too claustrophobic for me lol.

 

[raystuttgart]

Hi guys,

maybe I can give you some ideas from a "problem oriented perspective" (sometimes useful in System Design).
(Also see this post.)

Sometimes I feel that the "Automated Trade Routes" (a really difficult feature) is considered too simplified.
There are several very difficult problems to be solved and thus would also need very complicated logic.

----

Those are some of the problems that are quite obvious and maybe even the most important.
(The logical order of importance may however change during gameplay.)

• "shortest path" problems (considering movement paths / distances)
  
• "current offer and demand" problems (considering current Yields in storage)
  
• "future offer and demand" problems (considering supply by other transports and production)
  
• "highest value / need problems" (e.g. which Yield is most needed for construction and production)
• storage capacity problems" (e.g. How efficient can Storage of Transport be used? / How much Storage is in City?)
  
• "ideal distribution problems" (how can I distribute my supply most efficient for e.g. Domestic Market)
• ...

Or to write it easier to understand:

• Distance between Cities (Start and Destination)
• Yields in Storage of Cities (Start and Destination)
• Production of Cities (at least Destination, maybe even Start)
• Supply from other Trade Routes (of Destination)
• Need of Yield Supply (e.g. for Production or Construction - Strategic Yields need higher value)
• Available Storage in Transport (maximizing Storage usage)
• Available Storage in City (to avoid Overflow)
• Optimal Supply Distribution (maybe 50% can be taken elsewhere where it is also needed)
• ...

----

The only kind of logic (which I know) that can handle these things at least a little bit is "weighted value calculation" logic.
Simply because it can make decisions a little more dynamic.

Do not even try to create something like a simple "if / else" logic" because it is too static.
And it will also be a nightmare to program and maintain.

----

Important to understand:

• Some weighted variables will impact the decisions (e.g. which speficic Transport / Destination City / Starting City / good / amount of goods) positively
• Some weighted variables will impact the decisions (e.g. which speficic Transport / Destination City / Starting City / amount of goods) negatively

Basically those 5 aspects stated above need to be packed together in a decision set determined by the weighting logic (Transport + Destionation + Start + Good + Amount).

The logic has to run a while loop, until all decisions sets have been computed (no more available Transport, no more open Trade Route configured by the user, no more goods to pick up ...) or if it just has no more valid options.

Basically the "weighting" should tell you that there is no good decison set anymore.

See next about "Queues":

Also important to understand:

• One Transport will not simply have one Start and End City, with one specifcic Good and Amount.
• It will need to have a work queue of these that more or less needs to be updated (and maybe even resorted( every turn.

Once "orders" of the queue have been served, it can be filled again.

----

What might be considered:

Current Automated Trade Route System thinks from perspective "Start / End Destination".
(It is simple but it is inefficient to manage complex and long Trade Routes.)

Future Automated Trade Route System may additionally think from perspective "Route of Transports".
(It may allow to plan real sequences of actions for the Transport along its route.)

Summary:

If the Automated Trade Route System gets 2 perspectives it will be much more powerful.

• Start / Destination perspective
• Route of Transports

----

The big questions:

How much weight do you give each problem factor?
(Balancing, balancing, balancing ... --> Lots of testing.)

How complicated do you want to build the queue logic?
(The longer you make the queue the more heavy performance impact gets.)

-------

I am fully aware that my suggestions above may be one of the biggest challenges and efforts ever discussed in Civ4Col modding.

Maybe however this will give you an idea how such a "Automated Trade Route System" might work.
(It is still a very very tricky problem to solve.)

 

[Nightinggale]

Sometimes I feel that the "Automated Trade Routes" is considered top simplified.
There are several very difficult problems to be solved and thus would also need very complicated logic.

I agree. Creating fully automated trade routes, which are fully optimized will be an epic task. It will also likely be extremely slow. There are multiple reasons why it's not a good idea to set this as a goal. AI needs fully automated meaning it will likely have to rely on the current automation code forever. Not a huge problem. We just have to teach the AI to favor speed over cargo slots when it comes to choice of transports as well as spamming transports seems to be the answer. This also means my ideas for optimization for the current system will be useful as it reduces the waiting time for the AI. This is true even if human players stop using the existing system.

Human players should ideally drop using fully automated transports. I never liked the system, but it's the best we have right now unless the player wants to micromanage everything. A good automation system is one where the player can tell what the transports should do without spending too long on setting it up.

"shortest path" problems (considering movement paths / distances)

We have an advanced version of the travelling salesman problem as ideally it's coordinating a fleet of transports to get the job done. I'm not even going to try to solve that one. Step one would be to reduce the complexity of the problem.

And now we are back to my earlier post. Human transport automation should use the idea from classic colonization where the player would set up a trade route, which lists colonies in an order and then assign transports to that trade route. The transports will then visit each colony in order. This completely ignores the complex math problems of shortest path and traveling salesman as this takes place in the head of the player. This way rather than dealing with math issues, it becomes a game challenge. If the player is unhappy with how it works, the player can change it. The game won't be at fault.

When the problem of "where to go" being solved, the coding issue of what to carry becomes way less complex. Even a primitive if-else setup would work, though it's possible to add more complex logic like "skip colonies with nothing to pick up".

If trade routes can overrule limits in colonies, then a colony can export when over say 100, but a trade route can fill up to maintain 300. The colony then becomes a hub where other trade routes can pick up cargo. This can distribute cargo like tools or for the domestic market.

We can likely think of a number of decent addons, but that's the key here. They are addons. The core is to ask the player to make a list of colonies and then the transports will know where to go. That's the core of the idea.

 

[raystuttgart]

Creating fully automated trade routes, which are fully optimized will be an epic task.

I fully agree.

That is why I have and always will consider "Automated Trade Routes" an AI feature.
AI needs it, but it will never satisfy the expectations of Human Players that really expect "optimization".

In small empires (few Cities, few Goods, few Transports) it will most likely not even work that bad.
But in giant empires (many Cities, many Goods, many Transports) things will simply get too complicated to really have satisfy human expections.

It will also likely be extremely slow.

Also correct.

There are multiple reasons why it's not a good idea to set this as a goal.

I absolutely agree, but what is it that we are talking about then?
So it is about havig a system that is optimized for performance and for small empires and AIs?

Actually perfectly fine for me and my preferred solutions, because everythig els is not realistic.
But then we should make that goal clear to everybody and not set false expecations.

We just have to teach the AI to favor speed over cargo slots when it comes to choice of transports as well as spamming transports seems to be the answer ...

Perfect. I fully agree with that goal.
I was thinking you wanted this "large Empire Automated Trade Route System for Human Players".

This is true even if human players stop using the existing system.

No, I do not think they will, but we should simply set realistic expectations and goals.
The Automated Trade Route Sytem will simply never ever work even closely as optimized as if a Human Player manages manually.

Human players should ideally drop using fully automated transports. I never liked the system, but it's the best we have right now unless the player wants to micromanage everything.

Wow, that is the first time I hear you say that.
(Because it is 100% my opinion as well.)

----

Ok, will answer the rest later.

 

[Nightinggale]

So it is about havig a system that is optimized for performance and for small empires and AIs?

I was referring to CPU performance and that will work regardless of size. The issue is that when the selection group loops through all available routes, it checks if it can move to both ends of the route. This is done by generating a path to both colonies.

Say we have fur from Plymouth to Jamestown. The wagon train then generates a path to Plymouth and then one to Jamestown, both from the current plot. Next route is cotton from Plymouth to Jamestown. Again to check if it's valid, it generates paths to Plymouth and Jamestown.

However because the game is essentially frozen (nothing happens while looping through all possible trade routes) we know that the result will be the same as the first time. My idea is to store the result meaning it will only check for access to each colony once regardless of how many times it needs to check if that colony is accessible. Since generating a path is expensive in terms of CPU time, not keeping the results between loop iterations is an AI slowdown.

This will not have an impact on gameplay, only how long people will have to wait for the AI. I'm not touching this now though as I don't want to risk breaking traderoutes and hold back the next release for that reason. It's a future performance boost.

I was thinking you wanted this "large Empire Automated Trade Route System for Human Players".

I want a partially automated system for human players. Or perhaps you could say I want a way easier to use interface for manual setup.

I do use the fully automated system now because it's either that, a million options to configure for manually configured routes or completely normal operation.

 

[raystuttgart]

The core is to ask the player to make a list of colonies and then the transports will know where to go. That's the core of the idea.

I want a partially automated system for human players. Or perhaps you could say I want a way easier to use interface for manual setup.

Currently can not really imagine how your solution is going to work.
But I will let myself get surprised.

As I said, for me the most important thing with "Automated Trade Routes" is AI.
Anything that comes on top for Human Player is a nice addition of course.

 

[bogwart]

A good automation system is one where the player can tell what the transports should do without spending too long on setting it up.

Human transport automation should use the idea from classic colonization where the player would set up a trade route, which lists colonies in an order and then assign transports to that trade route. The transports will then visit each colony in order. This completely ignores the complex math problems of shortest path and traveling salesman as this takes place in the head of the player. This way rather than dealing with math issues, it becomes a game challenge. If the player is unhappy with how it works, the player can change it. The game won't be at fault.

This is a great idea.
Additionally I think an important part of a new Trade System should be a good interface where you can easily edit trade routes and select the respective wagons.
Currently it's suboptimal (Military Adsvisor > Sort By Unit > scroll down to find the relevant Wagons).