The vassal system is definitely the top one for me. I always turn it off.
It's a way to accelerate the tendency of a few civs to dominate. Fewer (free) civs doesn't make the game more fun. And the "enemy you are fighting vassalizes itself to someone else" mechanic is the worst part of it, for reasons already covered. In Civ III, they could sign another civ into an alliance to help them against you, or sign a mutual protection pact with someone - usually tough to do if a war was already going on - and that could spoil your plans. But not to the same extent that them becoming a vassal of someone else does. If they brought an alliance in III, you could peace them out so you could focus on the new enemy. If it was a mutual protection pact, wait 20 turns for it to expire, make peace with both. But vassalizing is essentially forever.
That there's no warning of it is also part of what makes it so frustrating. More fun just to turn it off.
How could it be fixed? In EU4, you can't diplo-vassalize yourself if you are at war. There also must be a much larger differential in size than there is in Civ4. As a result, I have no complaints with vassalage in EU4.
Spy Specialists being auto-assigned is also annoying, and I sometimes turn espionage off - it's probably the second-weakest system in the game, IMO. Courthouses are too good to not build due to their maintenance reduction, but I'd rather nor have a chance of my great person being a Great Spy, all the other ones are much preferable. This could be fixed by having a button in the city UI to prevent certain types of great people from being auto-assigned.
The force civics is also one I'm not a big fan of. In part because it's always the last-tier civics. What if you had a bunch of Communist powers who wanted State Property to be a global civic? Or capitalists who wanted Free Market? It's just kind of boring having all the civics wind up the same at the end, and the last-tier ones aren't always even the ones that make the most sense for the map. For that matter, instead of "global civic", it might also be more fun to simply have "forbidden civic". Forbid Slavery, and any nation that uses it is an outlaw and has the outlaw penalty. But you could still choose the other four options without a penalty.