For trade, no. But it depends on what kind of victory you're going for. If you're going for Domination, you need Open Borders to scout, but it's also an invitation to be mapped out yourself. For all other victories, you don't need open borders at all since nearly all city states are accessible from the sea; you just may have to go an extra tile or two around the other civs to find them, that's all.
There's no diplomatic hit for constantly refusing two-way open borders agreements. Being friendly with other civs is a mug's game, but if you insist on trying you get better results with pacts of cooperation with strong civs and pacts of secrecy with them against weaker civs.
In short, you may as well refuse all two way open borders agreements UNLESS you are determined to go for a domination victory and don't mind the risk.