I consistently settle 6-8, and can go 10 if I want. I've had games where I settle 12-14 cities on my own before ~125. Happiness is just a number for most of the game, and I'd rather be at -15 with 6-10 cities than +10 with 4 most games. At least until later when the yields are a huge factor for happiness. Early on it's mostly the combat strength I care about, and that's solved by having 2x more cities than my enemy.
If you are focusing on combat I could see this.
I can say that 4 cities will generate significantly more culture and science than the 6-10 cities you mentioned with that happiness penalty, so it definitely matters