Hmm... when I wind up in cities of under 100,000 people on my own continent, it's not usually because of the city, but because I wanted to go there for something in the city or area.
Somewhere near the Atlas Mountains in Morocco could qualify. And I have a Moroccan friend who could make some recommendations.
One of my mates is engaged to a girl who lived in South Africa for two years, and while IIRC it was one of the larger cities (Pretoria?), she loved it there and may be able to recommend a smaller town, and almost certainly things to do.
I have a good friend from Tanzania, who could likely recommend a good location.
And I suppose that would be my approach: Talk with people I know who are from or have lived in Africa, collect their ideas, and choose one. Not a whole lot different than if you asked the same question about France, except that I know more people who have lived in Africa for an extended period of time than in France.
I'm curious about the exclusion of non-mainland Africa. Too good of weather in Madagascar? Or the islands in the Indian Ocean?