I'm also finding the unhappiness UI misleading, because it suggests that "if you have boredom and you build this building that removes boredom, you will be happier". But most of the time once I build it, the unhappiness just get replaced with something else.
I've bolded my main points below after playing 3 wide progress games on the current patch, but generally I've found happiness manageable to an extent over the past several months. This is what I'm not a fan of: other than tall, most times it feels as if there's barely any breathing room, even when playing 'good'.
I usually play below supply for the meat of my games due to getting away with it on a lower difficulty and funneling 90% of my (land) units through Heroic Epic (usually my capital) -- definitely not efficient, I know -- but the pro is that it allows my secondary cities to focus infrastructure alongside constant building investments + hammer ITR's. The sad reality is that it usually doesn't matter; despite doing nothing but focus infrastructure while keeping the populations of those secondary cities more than reasonable (10-25 pop between turns 150-300), those cities still fight unhappiness the entire game, and also rarely end up contributing to units because of the tangible production malus associated with local unhappiness. Unless it's the capital/holy city, it's probably in the red despite any efforts, and it bottle necks my unit production even more (yes, I realize this is the intention, but it's mainly intended for unit spamming warmongers, not peaceful wide expansionists that want to actually build a few units in secondary cities without it taking 10 turns per unit). And then to avoid growth and increase city yields, you work specialists, but then those specialists create more unhappy from urbanization. It's a vicious cycle that seems like chasing your tail most times.
I'm going to lay out a basic happiness template dealing with ranges that I think most would agree are realistic expectations of ideal scenarios (standard map/settings) for the three main styles of gameplay. This won't be some elaborate number crunch, just a simple outline to display some thoughts.
Group A - Tall (5-10 cities, usually tradition)
pro = unhappiness almost a non-factor (80%+)
con = small empire/supply
Group B - Wide peaceful (10-20 cities, ideally the goal when selecting progress)
pro = more land, bigger supply
con = unhappiness can be an issue (anywhere between 35-80%), only average military presence due to focusing expansion and infrastructure
Group C - Wide aggro (20+ cities, almost always authority/imperialism)
pro = biggest empire/supply/military
con = unhappiness is a constant fight (struggle to hover between 35-50%)
The civs in group A aren't necessarily looking to expand further, otherwise they just transition themselves into group B while hurting their desired VC with wasted hammers and increased tech/policy/tourism costs. Meanwhile, civs in group B aren't necessarily pumping out units and filling out their supply cap, otherwise they just transition themselves into group C and might as well take over the world. The weaknesses for A and C are fairly obvious, but for group B I think there needs to be a bit more wiggle room in the current happiness, while allowing the primary disadvantage to continue being threat from opportunistic AI -- Recursive & ilteroi continuing to work their magic and have the AI assess and attack in correlation with the game's events is still the end-goal for any instance, and situational violence should always be the ultimate trump card or counter for any type of runaway. Peaceful over-expansion, or group B's main weakness, should come more primarily from vulnerability to the outside, than from internal strife (with modest city pops and decent infrastructure, of course). Personally I'd like civs in group B to have some more leeway --
in the mid-game,
initial settling phase is more than fine after the change to 'Equality' -- to expand peacefully, without all the harsh repercussions applied that are designed for warmongers.
My issue is that I can play style B 'flawlessly' on a difficulty like King (I can't even imaging the unhappiness frustration on something like Deity...), yet still manage to have limited to no happiness breathing room despite keeping populations modest and neglecting army for infrastructure (yes, and Public Works, which have already had their issues highlighted earlier in the thread). I.e,
I think group B needs to be able to consistently find more middle ground between group A and C, instead of gravitating so easily towards group C without 'bad' play, simply 'because you went over 10 cities'. If I'm a progress civ that's managed to establish solid infrastructure across my empire while simultaneously settling 15 cities mid-game (standard settings) by beating my opponents to the punch while they used their hammers on other needs, then there should only be minor happiness accountability or resistance on my side, IMO -- especially considering the scaling Pioneer/Colonist costs and loss of pop. It's usually grueling enough, so let my opponents realize the thin defenses and punish me the proper way, the way a human would by taking my vulnerably cities.
I realize this has nothing to do with puppet discussion, though I agree they'd be better off more simplified in terms of the specialists, but I feel the need to tie this in from events in my recent games, and generally speak on behalf of peaceful wide play. I will reserve any judgements on group C happiness once I'm able to hopefully play a domination game in the near future. Gazebo has addressed my happiness qualms in the past regarding domination, so that should hopefully still be in a good spot aside from the current puppet/specialists talk.
As a send off from my ramblings, here's a small idea/solution I'm just going to throw out here, but
what if Pioneers/Colonists granted +3 happiness on settle (similar to the code for constructing landmarks) as a little initial buffer? Pioneers/Colonists (now) costing 1 pop + a poop ton of (scaling) hammers, on top of all the diminishing returns being currently discussed regarding benefits of settling new cities, and it only points to one thing for me: sucking all the fun and excitement out of the exploration / colonization phase, and nullifying wide expansionist's primary advantage / purpose once the map opens up. I've said this before and I'll say it again, the best bet while playing VP often seems like a 6-city turtle, regardless of any other conditions (unless gunning for world domination, obviously), and I'm just trying to open up some other avenues to help improve dynamics while still keeping the integrity of this amazing project.
Based on my in-game feel, wide expansionist needs just a tiny lil' extra 'somethin' somethin'. I'm not asking for something drastic to be achievable, like peaceful 20 cities by turn 200, and I'll reiterate that the current system is mostly fine, but I don't think it's unreasonable that based on my my presented example
it's hard to estimate the future.
This is why I'm arguing for more flexibility when it comes to peaceful expansion throughout the mid-game. My proposal combined with a slight PW adjustment will probably do the trick of providing some breathing room for ultra-wide 'good' gameplay.