One could really go crazy with this concept.
- if you unmount in one of your cities you get the money back (your horses don't run away or get eaten by monsters)
- you could have horse classes like "old nag" for 10 gold, +1 movement, "something fancy" +2, 20 gold, "war / armored horses", +1...