Excellent!
Here's a rifle wielding cavalry-man. Basic idea.

Here's a better view. His costume is good, but knowing how you hate headdresses, I think you should make the unit without one.

Maybe something like this for his head: Just a few feathers. A huge headress is cumbersome and probably wouldn't be worn in battle anyway. This picture is my favorite, by the way.

And here's one last picture for good measure.
I'm just throwing this out there, but for a possible fortify animation you could have the horse rear up on its hind legs, turn sideways 90 degrees, and come back down on all fours. For the Attack, have the brave (turned sideways at the waist so he is facing his enemy) fire his rifle. The victory could just be the horse turning back around.
That would be the complicated way. You could just have him and his horse face his opponent, lean over a bit, and just fire like the regular cavalry. Whichever you want; its just a suggestion.
BTW, I'm pretty picky when it comes to graphics, but I really liked your Ethiopian horseman. The size and proportions are perfect, and the animation looks as good as Utah's cavalry units.