Inspired by this thread I used horse archers extensively in a game I started last night. Here's how it went: (monarch/hemispheres/normal speed)
I was playing as Freddy and Joao was my closest neighbour. He was REXing like crazy and starting to box me in. I had marble in my capital and horses nearby. I was able to build 4 cities incl. capital before feeling the need to war vs. him. I did something a bit unorthodox and took HBR off the oracle. I cranked out 20 horses archers (about 5/city with stables) and declared war. I took his capital first and one other city. I got a GG and took a ceasefire because my stack was very injured (but many lived!). Once they healed up quickly thanks to my MASH unit and I had built some reinforcements I attacked again. Joao had built an amazing 10 cities in the BC era so I had to take a ceasefire 2 more times over the course of the centuries-long battle, but I took him out ca. 1000AD giving me a total of 14 cities.
I was a bit worried about feudalism and engineering coming on scene soon (they hadn't yet, but it wouldn't be long) so I decided I needed to tech a bit. I was close to liberalism and had lucked out getting the Mausoleum late. With a great artist and the taj mahal I was able to get 20+ turns of golden age after cottage-spamming my empire (11 of the cities became commerce cities, 1 production, 1 gpfarm, and my capital was mix comm/prod).
I was teching like a madman and was first in score by a large margin. I teched to cavalry got the GM from economics, sent him overseas, upgraded about 10-15 horse archers to cav, had built some cav, declared on Rameses and took him out except for a couple cities. Took peace to heal. He vassaled to Hamurabi.
That was ok because I had been beelining to combustion-industrialism. Now I am in the process of finishing out a domination win with panzers. The factories-coal plants are helping a lot and to deal with unhealthy after industrialism I beelined medicine-refrigeration-superconductors (labs)-genetics.
I feel like I am going to shave some decades off my usual finish time (space or domination) of around 1930AD (I'm pretty consistent at that mark).
Anyways, here are my conclusions:
1) Horse archers work very nice for taking cities prior to longbows/pikes. Joao had quite a few swords and spears, but a mix of flanking 2/combat 2 took them out even with high cultural defenses.
2) Some ceasefires are indeed necessary to heal up as obsolete mentioned. I don't know how this compares against lugging around 1-move units and siege and sacrificing a bunch of siege to wear down the defenders. It's hard to make this comparison I think.
3) I felt like I had my target of ~15 cities a couple of centuries earlier than normal. I normally shoot for 1500AD and usually get it lately around 1300AD. In this case I had it around 1000AD.
4) Having veteran horse archers makes for nice upgrading to cuirassiers/cavalry. But generating a GM is important because it's not cheap.
5) I probably could've pushed again with knights. That was my plan: to spam knights against pikes/longbows is on par with h.archers against spears/archers. But since I lucked out and got the Mausoleum I couldn't resist the 20+ turns of golden age when I had very nice cottageable land empire-wide. If I hadn't got the Mausoleum I would've gone for knights which would've meant I took out Rameses sooner. Then I probably would've used either knights or cuirassiers against Hammurabi.
I think that mobile units are ideal for domination wins and starting with horse archers is something I would highly recommend if the situation permits it.
EDIT: Something to keep in mind that I didn't prepare for enough in this game is that when you are sacking cities quickly with mobile units it becomes important to have an "army" of workers prepared to come in behind when it is safe and develop the land.