Yes and no.Most times-yes.
There is a one time bonus of science beakers equal to the amount of gold when delivering a caravan.Check your science advisor before and after delivery.Early returns are low but they can really climb.Over 1000 gold coins/science beakers per caravan/freight is quite possible.
Caravans can also be used to build wonders.Each one contributes 50 sheilds so having a bunch "stored up" is a good plan.You can build the wonder fast without tying up a city for a bunch of turns building it.
Your build order is another of the "depends" things.What level is being played,is there an early threat,is it a land gobble before its gone, etc.
A good rule is when you can't think of anything to build,build caravans.If you think you have enough,then build some more

Another thing that can be good about building caravans is if some upstart trys to put your palace to the torch,you can change production to military units with no penalty.
I like/try to have "specialist" cities in that cities have specific tasks.One or two may get barracks and build my defenders.Others may build only caravans or settlers while one may only build wonders.