primordial stew
Emperor
but his religion isn't mine, and thus I've no use for him (he'd already converted some city so no holy place can be built). Is it ok to "delete"? That wouldn't lead to problems down the road would it?
NO!
Do not delete him!
I was playing Rome and I capture some random Great Prophet. Seeing he was some religion I totally never heard of before and completely different from my empire, I deleted him. 10 turns later, a 12 more come out of nowhere, walked around and disappears. All of a sudden, all my happiness starts going down. Like half my cities started to go to the religion of the guy I deleted. Then barbarian rebels start spawning everywhere. My economy crashed and burn. Finally after all of that, I fell behind and the empires around me like Persia, Vikings, and Germans starts attacking me. Then out of nowhere, the Huns attacks me. Seriously, the Huns come out of nowhere. I gave up after the Germans took my capital.
Maybe it's just me, but I always use captured GP's as scouts to recon the territory of other Civs. They can enter foreign lands and they won't slowly die like missionaries do, so I just send them in and walk them along roads to gain vision on all the AI's cities.
Since spreading religion is all a missionary can do, should a captured missionary "switch sides" to spread its captor's religion? Might seem simplistic to you, but it would seem unrealistic to me for a GPr or missionary to get so easily "unstuck" on its original religion -- these are supposed to be true believers, after all.
If the GPr still has 4 spreads left, you can plant it to form a Holy Site, which does provide benefits to the capturing civ (think of it as the location where the heretic GPr was executed, and followers of your religion go there to gloat, generating faith as they do so).
I suppose, by extension, a captured GPr or missionary that was simply deleted in your territory could provide a small dollop of faith (similar to how a deleted unit will provide a small bit of gold).
It seems to me that the GP or missionary scout is kind of a loophole that the creators couldn't find a way out of. The sight is very limited and there's no auto-explore option. So you have to waste a lot of time micromanaging him to get any use out of it.