The problem with the current Trade system is that it isn't a Trade System, it is TWO Trade Systems. One requires Caravans and Cargo Ships and only carries generic Gold-producing cargo, the other magically covers the world no matter how big, is established diplomatically and carries only strategic or luxury goods.
Both systems have serious logical flaws and having both in the same game makes sense only if you realize that the entire 'trade route' system with the cargo ships and caravans was added wholesale onto the existing system with no attempt to merge the two.
So first, let's agree that the game needs ONE Trade System only.
Second, let's try to give the gamer the same options in the trade system that historical civs had in trading.
That means, first of all, that there are limits on what can be traded, which limits will change as technology changes. The most obvious limit is that from the start of the game (4000 BC) any trade route moving over land is limited to what can be carried on a pack animal or bearer. Wheeled vehicles aren't going where there are no roads, at least not for long and not with any amount of cargo. That means that any overland trade until railroads will be small quantities.
Now, this is not as much of a limitation as you might think. It doesn't take a lot of copper (bronze) or iron to equip a soldier/warrior all the way up to the renaissance 'era', and horses move themselves: 'Strategic' goods can still be moved. By definition, Luxury Goods do not require huge amounts to be valuable: they can also be moved and traded.
BUT
When you get into the Industrial Age, the quantities increase dramatically. It takes 1000s of tons of iron to build a railroad, or an ironclad. A city the size of 19th century London, Paris or New York is not going to be made much happier or content with a few hundred pounds of Incense or Wine (in fact, they may not give a Rat's Patootie for Incense at all, but that's another problem in the game)
AND
To make a difference, Food always has to be traded in really Mass Quantities: bread or wheat for Rome in Classical Times came in on 1500 ton cargo ships - a couple hundred pounds on the back of a camel would make no difference to the city at all.
So the real distinction in Trade is not between Strategic or Luxury Goods or between diplomatic or caravan-started trade routes, it is between Mass Quantities of goods required in the later stages of the game for everything and food all the time, and the 'piece' quantities required for most goods earlier in the game. The former can only be moved by Water or Railroad/modern Highway. The later can be moved by just about anything, including human porters (no draft animals or roads required)
Now, about setting up the trade routes.
It makes no sense to 'start' a trade route by Building a Caravan. The caravans will build themselves, from your merchants, traders, entrepreneurs and such once you (the ruler) give them a reason to. That reason is a market in another city where they want goods that you have, and a way to get there without getting waylaid by barbarians, bandits, or the other cities' army.
SO Trade routes are established diplomatically. All of them.
The mechanism for getting trade routes long enough to reach all the markets should have some limitations, but not the artificial ones in the game now. If I have enough merchants, traders, etc in my kingdom, the number of such is the only limitation on the number of places I can go to trade. The distance over which I can trade depends on establishing 'depots' or transfer points - places where caravans can rest as part of their journey, or cargo can be transferred from boat to camel or river boat to sea-going vessel. The number of those Depots will govern the amount and distance of my trade routes.
Here's my idea for a mechanism.
Every Market, Seaport, Harbor or Caravansary (to use the buildings in the game now) has a slot for a Merchant or Trader. Later Technologies will increase these slots, and later buildings will add more (Airport, Container Terminal, possibly Stock Market).
Every Trader allows you to establish a Depot. When you establish diplomatic talks with another civ, you offer Goods, which can be both specific (gold, iron, wine, etc) or General (food, Manufactured Goods). You agree on what you want to trade and how much Gold or goods will change hands to make it happen. Then you agree on who provides the Depots and how many - because both nations should have a vested interest in making the Trade Route happen, and each Depot will add to the amount of profit (or percentage of the total value of the trade route) you take out of it.
Example:
You and Slobbovia decide to Trade. You have Silver and Manufactured Goods, which since your cities are mostly in grassland and forest means woodwork, pottery, and possibly exotic art, since Slobbovia is all desert this is worth something. They have Spices and more Manufactured Goods. Since the route will include some overland segment, you cannot trade Food, Timber, or Building Stone - it's too heavy and bulky to move with your current technology.
From your nearest city to Slobbovia you trace a route across country 7 tiles to the coast, which is less than 10 tiles away, so it only requires 1 depot, on the coast where the goods will be moved onto a ship. 15 tiles away is a City State, where the second depot will go (default mode for City States will be to welcome Trade and depots, unless they are actively at war with one of the trading civs). Another 18 tiles beyond the city state is a Slobbovian coastal city. The trading partner usually provides the depot in his own city, but you might offer generously to provide it from your own merchants - which also increases your profit and gives you potential spies in his city!
So, you have a trade route 40 tiles long, requiring 4 depots including the ones in your city and Slobbovia's, which initially will be carrying Silver, Spices, and general Goods back and forth between you. Graphically, a caravan or cargo ship will move along the route, each graphic representing one turn's worth of profit from the route. If barbarians or other raiders 'take' a caravan, they DO NOT DESTROY the Trade Route: they remove one-turn worth of income from it, and the advantages you got from any goods (strategic or luxury value) for one turn. Once a trade route is up and functioning, you can negotiate to carry other goods along the same route.
Great Merchants have a new ability: they can single-handedly establish a trade route of any length, with any number of depots - very handy if you want an early route stretching across a Pangea continent.
Note that with this system, the limitation on length and amount of trade routes is simply the amount of effort your civ puts into the infrastructure: build lots of markets and caravansaries and harbors, establish lots of depots, rake in the Golds (and probably diplomatic clout with your trading partners).
Note also that any depot away from a city is vulnerable to Barbarians and raiders, and if they destroy it, it has to be reestablished (using up another Merchant) or the trade route starts to fade away (the graphical caravans stop generating, and as the ones on the route already reach the ends all profit and goods from the route cease). That means depots should probably be garrisoned. Such garrisons and depots if not in a city already, will start to generate settlement, and may eventually become a city or city state. It also means that where possible, putting depots for several trade routes on the same tile makes sense: easier to garrison, and a city will grow up around the depots that much faster.
