The main reason of sudden unpredictable drop is that happiness is a "per yield per city integer", which mean that if you have 10 cities, you can have +/- 50 happiness in one turn without real reasons. (though it would be quite unlikely to have a drop THAT big)
The solution "stop using integers for happiness" would smooth up everything and really improve the experience, but basically ask to recode everything linked to happiness and track for every use of it everywhere (which would need to a lot of works, and a lot of bugs, so even more work). Not to say it could break any mod relying on the Community Patch or Vox Populi.
The solution "average happiness trough turns" is an alternative way to force a smooth up, but would lead to the unintuitive "your global happiness is not the sum of happiness of all your cities, since it is averaged on multiple turn". On the plus side, it would make unhappiness far more predictable.
I’ve tinkered with top value smoothing but it causes exactly the problem you mentioned.
To head off complaints: There is no bug in the code. The smoothing of global averages helps but ultimately as you noted no amount of smoothing changes the fact that there is a hard conversion to single digit integers in civ’s deep happiness code.
Two possibilities I haven’t explored:
Use top level smoothing _plus_ an ‘overflow’ pool of ‘becoming unhappy’ citizens. So happiness can only change by 1 per turn, but the overflow pool of ‘pressure’ holds the other changing values. Make sense?
I could also have the unhappiness penalties only start to trigger once you are out of golden age points (much like running out of gold affects science). We’d probably need to reduce golden age points or something to compensate.
G