Interesting question. I've had either situation in my games, often dependant on where midgame trades put me on the techtree, so not a wholly free choice.
If you have a war at hand shortly before discovering either tech, I would beeline for grenadiers and get rolling. Fighting over smaller areas, grenadiers are cheaper and more versatile (offense/defense role). Also, chemistry is en route for cannons, which rock when taking cities. If necessary you can always follow up with cavalry later.
In this months' gotm however, I prioritised cavalry, as I had much land to conquer and a big enough tech-edge that I could send in the horses supported just by bombardment and not worry about having to defend the cities I took. If you can deploy a good size cavalry force before your opponent gets grenadiers and rifles, you are pretty much unstoppable on the offensive.
My main concern about cavalry is that they don't upgrade well. Once rifles, mgs and infantry come online, they just sit around and cost you money. And by the time you get to upgrade them to gunships, many of their promotions will be useless.