BUG and the Unofficial Patch work great together; that's what I use. I wrote much of BUG, so take this with a grain of salt.
I have a pretty fast computer, but it's no monster. I haven't noticed any slowdown using BUG, and the only thing in BUG that should slow you down would be the Autolog feature. It logs everything happening to a file as it happens, and as that file grows, I could see it slowing down some. I'll give it a try later.
You can easily disable the Autolog, as you can any other BUG feature, using the ALT + CTRL + O options screen. Switch to the Logging tab and uncheck Enabled. You may need to restart Civ once to be sure it's off as I don't remember exactly how that code works (not mine).
Everything else in BUG happens only when you open advisor screens, so it shouldn't slow down in-between turns at all.
As for the Unofficial Patch, I don't see how that could slow down turns either. I've looked over the code and it is mostly small changes to existing checks. It doesn't alter the normal flow of the game, so it shouldn't add any more time.