the 3gp maintenance for a harbor is almost always less than the maintenance on the roads it would take to connect your city to the road network
PROs
cheaper maintenance by usually 1-4gp (2-8gp vs railroads)
+% to ship production
CONs
costs city Hammers to build
can be blockaded by nearby ships
don't get the benefit of faster unit movement
Mandatory for island cities, of course, situationally useful in others. Forinstance, I'm currently playing a huge earth map, capital is on the coast of southern brazil on the east side of the continent. The Andes mountain range runs almost the entire length of the continent. If I wanted to connect my west coast cities to the trade network, I'd have to run a ridiculously long road almost to the southern tip of the land mass and back up again.... I haven't counted, but it would probaly be more than 20 or even 30 hexes. giving that city a harbor is far more cost effective, even if I paid for it in gold, I'd make my money back in 25 turns or so.
One strategy that could frequently be effective is to put roads everywhere, harbors on all the coast cities, and railroads to all the inland cities. Strikes a good balance between mobility, maintenance, and uninterruptable trade routes