A long time ago (at a time happiness was not really a problem for tall), you could grow as much as you want, as long as you have enough production to build every building as soon as you research them, and enough gold to invest in it systematically (and enough science/culture to not fall behind). Not sure if it is still the case. Agree with that idea. Though I would rather have it affect growth, since affecting food production could go into a cycle "not enough food for specialist -> more unhappiness -> even less food". In fact, we could focus the unhappiness malus to growth and science (and reduce it to gold and culture), so that if you have unhappiness problem, the game prevent you from deeping even farther in those problems.