If I may chime in since Joe asked those of us on C2C to take a look at the thread here and comment...
Ok, so ultimately 4 core points have been made so far as I can see:
1) The automated selection of trade route cities is a wretchedly time-expensive processing waste.
2) Player choice is next to non-existent in the trade route structure. There's some few variables players manipulate but to keep from there being far too much micro-management, there's a horrendous streamlining that makes the whole structure akin to a set of 'random' bonuses.
3)The concept of trade producing valuable yields/commerces from nowhere is rather flawed (though maybe not quite so bad for the Commerce Yield itself as it may be the only thing truly justified to be 'generated' this way.)
4) It would be faster and potentially more meaningful still if we were to establish new tags to adjust trade route manipulations on buildings, civis, etc... to manipulate Commerce per connected cities instead.
Ok.
So I'll respond directly to those main points and then offer some additional points for consideration and then offer a solution I've not seen suggested here that I feel would ADD to gameplay rather than simply streamline and shed what I feel are SOME very valuable game mechanic features attached to the concept of trade routes to begin with.
So responses:
1) I can't argue here. Surely this is a horrendous processing effort. Koshling may have cached the results significantly but yes, it goes a long ways towards making the later game take a lot longer turns and overall the design is indeed terrible.
2 & 3) I've always wished that aside from Commerce generation, it would be nice if Yields and Commerces could actually BE traded between cities based on player choices. Would be far preferable if percentages of the resources (not to be misconstrued as BONUSES) of a given city could actually be given to another city via thereafter automated 'routes'. If cities could make agreements directly with one another to trade say, some food from one city FOR a certain amount of gold FROM the other. ALLOW the establishments of trades (and not just bonuses between civs at a player level) to be a PART of the game.
For example: You know you want more food for your arctic city because it has a tough time doing so for itself so you establish a trade between that city and another where it will give some of its gold income for some of the other city's food income.
Another Example: You want more production in this particular city because you're training up an army or you're trying to get it built up from an infancy status so you make a trade with another city for some of its production.
Added Commerce Yield 'from thin air' would be a positive side effect for both cities because it's the learning from each other and witnessing of the other's tricks and clever concepts in action (research), small time wheeling and dealing and thus added taxation from commerce taking place between the populations of both cities(gold), the sharing of attitudes, artwork and cultural ideas(culture) and the opportunities to learn secrets and valuable strategic information from the other locations(espionage) that is what the Commerce Yield represents and just establishing inter-city trades would generate this - in fact would be one of the most powerful ways TO generate Commerce Yield!
Sounds like something that would get more complex still and thus be a processing nightmare of an even worse magnitude right?
I think it can be done and make it done much cheaper in terms of processing.
4) I object somewhat. I liked the idea the first time I read it here but I've been thinking on the concept and it really does leave a LOT of game features out of the picture. Cultural sharing, espionage (as has been pointed out), tech diffusion issues, (for C2C we HAD trade route interactions for Properties and I still think we just need to restructure it so it can be re-established - and the advanced disease etc... structure I was working on would heavily use trade routes in a more controlled manner), Religion Sharing - You'd be wiping out automatic religious spreads here, Corporation interactions I think use trade routes. Then you've got Internal Only trade vs International Trade, trade agreements in diplomacy, events, resolutions... a TON of building and other tags that are devoted to interacting with the trade route mechanism that would need to be entirely rethought... There's a LOT of considerations that would make this seemingly simple adjustment highly complex and would leave the game lacking some useful interactions.
You'd be adding more tags that would need to be implemented through a tremendously laborious effort that includes the removal of all the previous implementations for trade route tags. I realize that AND probably doesn't use as many as C2C since we have bridges and road 'buildings' I don't think AND uses but still... that's a lot to consider. You have Civics and Traits and Techs to reconsider how to rework... As noted, this is a horrendously HUGE restructuring to go through for either mod.
However...
I have a solution to propose and it's probably what I would do for C2C at some point soon. (I've got a little work to do there yet to pave the way for this further in the way I'd implement it but it could be done in AND differently.)
So the solution I would propose:
Don't know about AND but in C2C we have Merchant units. I've got a few issues with them too. They ALSO generate production and food in a much greater volume than it takes to train them (in the case of the Food Merchants to an almost ridiculously imbalancing extent that makes it so that if you want to grow a city you'll build these until you're happy with your population no matter how high that population may become.) They seem to be a good idea in theory but are incredibly imbalancing.
If you want to use them to generate gold or commerce which is a paltry use for them (unless they're naval trader units and have thus no other use) they're laborious to utilize and problematic for the AI who currently sees them as primarily great scouting units since they can explore enemy territory as they can't be kept out by closed borders etc...
I would suggest
1) Disable all automatic determinations for trade route establishment. This immediately fixes the processing hell and 'lack of choices' problems. Give them back to the player!
2) Do so through trade units. Give the units an ability to Broker a trade route. Track the unit's 'home city' and make it necessary to send them, successfully, to the city you wish to establish a trade route with.
3) The route itself that's brokered could be of a number of different route yield types. Commerce generation would be a natural side effect for both cities but one city would give a yield in a volume determined by the upgrade phase of the trade unit for gold from the other city in a volume determined by the upgrade phase of the trade unit.
4) Otherwise disable yield/commerces generation by default.
5) Keep the max # of trade routes limits and trade route commerce yield modifiers all current buildings, civic, etc already give. Wherever possible allow trade routes to continue to work as they currently do and maintain the majority of the UI it already utilizes in the city screen, except that we are no longer talking about an automatic system of assignment. For maximum routes, your city's trade units would not be able to broker more deals than the city is limited to.
6) Obviously the somewhat heavier processing would swap over to unit generation for this purpose and the ai for those units. But I'm thinking this could be done with much less processing weight.
7) Deals must be accepted or rejected and must remain able to be cancelled. So there would be some new popups and AI considerations there but may not need to be toooo complex, particularly if the AI is programmed to be fair and generally open to trading without good strategic reason not to be.
8) Deals must be registered as an act of international trading for diplomacy's sake.
9) Some deals could be setup to be humanitarian, like food aid sent to another city where nothing is asked in return.
10) Each deal still counts as one trade route. And perhaps more than one deal could be struck with the same city even.
This way you'd make trade a lot more strategic and even adventurous as you'd need to get your merchants to their targets without those merchants being destroyed before brokering the deals you're wanting to setup. That would certainly serve C2C as it could lead to a lot more Criminal interactions and such as well.
Anyhow, take this as you will - just a suggestion from another modding mind.