Puppeteer's posts addresses the crux of the issue, but I think I'll add some more specifics that have worked for me.
First and foremost, I'd download the
CivAssist program. It's enormously useful, helping you determine your excess research (make sure to turn down your slider when you're in the last turns of researching a tech; it saves a bundle and every penny counts at higher difficulties!), while it will also let you check all AI civilizations' trading abilities without manually checking it. If you see one or two Civs has an advance and the others do not, be sure to buy it with accumulated gold; offload it to any wealthy backwards civs. This will often grant you a good profit. If nothing else, offload a tech to whoever has the Great Library; they're going to get the tech for free when you end your turn anyway, so may as well profit from it.
If you want to even the odds, play Mass Regicide. Use your Kings as scout units; it's a tad risky, but their high movement rate is invaluable in the Ancient Age, and will allow you to rapidly cross a continent if you send out one in every direction. You can also use them to pop goody huts; while the chance of barbarians is higher with higher difficulties, the fact you only need one King to survive mitigates this, plus you can stage your exploration of villages in such a manner that will let your King run away (have one movement point). Since barbarians fight with Warriors anyway that are equal to the King in power, you have a good chance provided you're up against only one.
The Regicide Scout strategy is definitely one of the most useful in advanced difficulties, allowing exploration of new city sites, meeting with new civilizations, and possibly allowing you to pop all the goody huts before the AI even knows what hits them. Plus, if you're not against save scumming, you can just reload until the backwards folk in the huts give you tech.
Provided you get an early enough tech lead, you have much more of a shot of maintaining it to the end of the game, as you can regularly buy technologies and sell them off to other parties. You cannot really hope to outdo the AI short of invading them, but you can most certainly keep up with them if you spot the niche each civilization has versus the others.