"Baba Yetu" is Swahili for "Our Father", the widely-accepted title of the prayer, so it makes sense as a chorus. Though the composer, Christopher Tin, confessed that the "yetu, yetu" repetition is only there for rhythm. It would have made more sense to insert "Jesu" (Jesus) instead, but Tin isn't especially religious and didn't want that element of the music to be in anyone's face. Given Zombie's (over)reaction, which is by no means unique, I'd say his instinct was correct.
I'm not religious myself but I quite enjoy this piece of music. It entirely suits the game (especially since this is the first Civ to include religion as a game element), appropriately polycultural, and dare I say uplifting. I enjoy Handel's Messiah and Beethoven's Ode to Joy as well, not in spite of their religious elements, but because of them.
What with all the "my god can beat your god any day of the week" perspective that far too many people have of their religion, it's easy to see why so many people get turned off of the whole idea. It's far too easy--for both believers and non-believers--to forget that at the heart of all religions is a spiritual element that is all about being part of something greater than ourselves, and that connects us to everyone.
That being said, music is all about personal taste. If you don't like it, fine, turn it off or substitute something else, which is quite easily done. Jimbob, I'm not at my Civ computer right now, or I'd tell you the path and file name. I'll try to remember to post it here later.
To (finally) answer the OP, I haven't come across too many other types of music like this. Tin has, on his web site, made both the menu and opening movie music available for download (just Google Christopher Tin). I have a small collection of somewhat-similar world music; check out the Quango label for some interesting titles.