First off know which techs are required and which are optional, to progress an age. This allows you to concentrate on getting ahead in ages a tech from an age ahead can usually be traded for all the outstanding techs of a previous age. The AI tends to research every tech, be it optional or required to advance the age. The Middle Ages is the best example there is at least 5 optional techs, although most optionals contain a Great Wonder, you can come back for them after advancing the age. If you have a Scientific Civ the free tech from each age advancement can be traded for those still outstanding in a previous age. Especially Nationalism which is usually your Industrial Age freebie, although be careful with it since your handing over riflemen.
Secondly know what each tech is worth to the AI, and although you may call it exploiting (I say its using experience) you get to know after a few games which techs the AI tends to research first and the ones they leave till last. Polythesism is good example, most times AI civs will research the Writing tree and Mathematics tree before going onto Polythesism. One of the reasons is probably because Polythesism doesn't give anything except fulfilling the prereq. for Monarchy. There are several other examples and there is a good article in the Strategy and Tactics explaining the AI and tech research. Know how to trade effectively with Civs.
Thirdly trade peace for techs, in the Ancient Age to have a strong army you need 5 techs (Warrior Code, Bronze Working, Iron Working, The Wheel and Horseback Riding) depending on your starting civ you may already get two of them and usually you will get at least another two in trades or goody huts. Horseback Riding is an optional tech but usually worth getting, especially for the upgrade path to Knights, the retreat ability and covering distances faster. Alternatively you can go the Bronze Working then Iron Working only path and then rule your continent with swordsmen. I usually always research Iron Working and The Wheel at 100% Science, just so I can see where the resources are or aren't. After you have these maximum of five techs, you should have located your neighbours and mapped out most of the continent. Put your research to 10%, balance the rest between gold and entertainment and beat the snot out of your enemies. Sue for peace whenever they have one or two techs you want, then hopefully you'll have two or more enemies that you can alternate between, before long you'll be neck and neck in tech. But just make sure you have been continually building up your civ, so that once the benefit of beating up your neighbours runs out you can undertake your own study.
The quickest way to drag behind in the tech race is if your play large or huge maps or on a pangea map. The AI civs trade to each other at a discount rate, and they will usually trade each turn if possible. I can't be bothered running through 10+ civs in dimplomacy each turn to see what is for trade etc, although if you wanted to be thorough you could do this. Often you can broker tech deals and profit well in the end. Especially if two major civs are warring with each other, but your at peace with both. You can become the middle man trading each others techs to your advantage. I don't know if AI civs colaborate on researching techs but sometimes it can sure feel like it, especially when you know your just ahead then all of a sudden after 10 turns, a few major civs have 3 or so techs from different branches and you are left with nothing to trade to any of them.
