I guess you're right that it's just the city state + barb processing time. Which still strikes me as odd, because the way that it works for me is:
1. all humans press end turn
2. the main civs process their turn - this takes 1-3 seconds
3. the game's 'unit focus' moves to one of my units that needs orders (indicating that it's my turn). Except that unlike in single player, I can't move it, give it any promotions, etc. During this time, the barb counter is flashing.
4. 3-5 seconds later, the barb counter stops flashing and I can move as normal.
Would've thought the logical way to do it was to move step 3 behind step 4, and remove the fake delay!