For me it's not just about trades. On contrary, being a stubborn and single-minded person I like when they remember me i have excess luxury. (the problem is that their deals are often insulting)
The main thing, and that's where it becomes funny, that annoys me is their value judgments and their insults. The best one : "I always wondered why you talk so few, I think that it's because you have in fact nothing to say". It touches me PERSONNALLY, because in real life I have nothing to say, sadly. However I wish I am just like my 10 years old, full of life and speach. But that's not the case anymore, i'm a sad old uncle who have nothing to do, nothing to say and who cannot sleep well.
That and the denouncing early "because they don't like you, all simply", the "it's for the best" when asking to stop spreading religion without the possibility to declare a religious war straight up (although it's off topic), and all the kinds of remarks they can do which are useless and interupt your game.
Oh, and I also press Alt+F4 when they conquer my capital on turn 10. Happens too often. Youtubers are lucky.
