The formula starts with resources city A has that city B doesn't.
Gold buildings are a multiplier to it for both. (In addition, there are flat additional bonuses to city owner from market & bank)
Distance is also a multiplier.
Harbors increase gold from Cargo ships for route owner.
East India Tea company provides more income to city owner.
Cargo ships provide 2X as much gold for both.
Exploration has a policy that boosts all cargo ships income to the route owner an additional amount.
Freedom contains a tenet that boosts route owners income when the destination city is to another civ following Freedom.
Morocco's UA includes a boost when other players make trade routes to Morocco.
Portugal's UA includes a boost for their own trade routes.
Patronage has a policy that boosts income to route owner to city states, but it's usually not enough of a boost to be worth it (compared to having the route with a major civ)
Commerce has a policy that boosts income to route owner when its a caravan, but cargo ship income would still be higher even with that policy.
There is also a building that does the same thing to Caravans as Harbor does to Cargo ships, but again you'll get more income if its a cargo ship.