I just had a thought... what if instead (or in addition to) city flipping, there was a migration mechanic of some sort? Related to Tourism and happiness, to distance between cities, and possibly to religion (e.g. people of a certain religion would move to a city with a majority population in that religion; or conversely, people migrating and mixing between cities, leading to religions spreading)?
With science not being the end-all, be-all of progress (and culture having its own tree), maybe such a mechanic would make food cease to be the sole determiner of empire population.
I doubt actual pop migration.
But I could see trade routes to ugly cities being weaker. That town you founded just to 6 Iron with no appeal might get a weaker rate of food, gold, and religion from traders. While that city with 2 beaches, a natural wonder, and a temple might grow fast to work good tiles via a trader.