Yeah, the bonus for being above zero-level happiness doesn't need to be too big since it will already result in a golden age. Those bonuses should be capped though. Make them linear increases up until you hit +10.
The negative bonuses, on the other hand, need to start out gentle, then increase severely as you get to 10-20 below your requirement. I think those should be percentage based... small maps you only need a total of about +40 happiness to keep your 3 cities running. But on larger maps where you might have 15 cities, you need more like 150-200 happiness. So 10-20 difference is a drop in the bucket (only 5-10%) compared to 25-50% on a small map.
So the big penalties should not occur before -5 (absolute) and any bonuses should stop at +10 (absolute). Anything at or below -5 should be calculated based on the percent deficit, with exponential impact as you decrease below that point (and it should be pretty harsh by the time you hit -20%).
Some clever ideas there. Thing is, if you made the penalties for below -20 unhappynes exponential you would be hammered if you conquered a rival civ with even 3 cities and would have no time to adjust quickly enough. Not too low to make conquering one empire an insurmountable problem, but high enough to ensure you can't go on a roll and simply ignore the unhappyness.
I think the real trick would be to make the penalty also proportional to the number of cities annexed.
So expanding from 5 of your own cities by conquering 5 cities wouldn't harm you too much if you went into unhappyness, it would only be multipleid by 5 and could be offset by courthouses. You'd have the opportunity to deal with the problem and reasonably quickly if you managed the situation well.
But it would really kick in if you then took it from those 10 to 20 too soon. The effect would then be multiplied by 15 (and it would thus appear that the penalties increase exponentially) if you didn't bother dealing with that initial unhappyness by building courthouses or by failing to puppet state them.
assuming each new city would add 15 unhappy (probably a conservative estimate) and you're neutral happyness.
1 city annexed would result in 15 unhappy and a penalty factor of 15. i.e. 1.5%
2 city annexed would result in 30 unahappy but a penalty factor of 60. i.e. 6%
3 city annexed would result in 45 unhappy but a penalty factor of 135. i.e. 13.5%
Once you get to 5 cities you'd end up with 75 unhappy but a penalty factor of 375. 37.5% The 6th a penalty of 54%. the 7th 73.5%. Annexing cities beyond this would quite simple crash your economy.
At which point, you probably wouldn't want to continue expanding until you've sorted the mess out or were going to raise or puppet the cities you took anyway.
This would discourage people form going on a roll by forcing the player to deal with the unhappyness before further aggressive expansion, or limiting the amount that could be gained by annexing (because you can't control what is built) but not from expansion or aggressive expansion in general.
The idea is to punish a lack of management and poor strategic choices, not military and tactical success.