@TruePurple Everything you say and suggest has indeed in some shape or form crossed my mind a decade ago, not because I was being a smartass or angry at Civ 4 developers, but because my mind was racing about on how I could improve on this awesome game. When discussing a game mechanic like this, bear three things in mind:
1) Civ evolved to food and slavery to what it is in Civ 4. Apparently this was lost on Creators of Civ 5 and 6.
2) Food distribution and overall impact of it vary greatly between world sizes.
3) It's pretty clear devs had several ideas how to do this, and agreed on what it is in Civ 4 is the best.
The evoultion of food, slavery and pop migration in Civ games
Civ already had some variations on theme on how food (re)distribution worked. Many concepts were pioneered in SMAC.
For example, in one civ (I think it was civ 1 or 2) a settler costed 2 food per turn to maintain while active and could've been "merged" into a city for pop growth.
In SMAC, you could terraform the land so well you could have a perfect food city, you could build crawlers that could ship energy, minerals and nutrients (food equivalent in SMAC) to other cities.
If we look closely at how slavery works in real life, it's pretty much like this: you capture a settlement, kill all the fighters, enslave the rest. Depending on how well you can do this, a portion of the population is now portable, perishable goods that can fetch a price and work to death someplace else. In civ terms, you'd sack a city and bring back settlers or workers to "merge" with your other cities "to work the fields". I think you can already imagine how this would feed even more into a snowball in Civ4 mechanics. Civ:Col fine tuned this because you got "intendured servants" (Basically white slaves) and "native converts" (native slaves) that have different yields from "specialist citizens" like master carpenters. This is a fine idea, but it would be too complex for a game with such large maps as civ4, especially for multiplayer. Simply too much micro, even for a civ game.
So what Soren did instead was to keep the same mechanic of "true slavery" that I described before, but restricted it to player's own cities, as well as contain it within the city (no pop migration). Might not be as realistic (you're doing to yourself what you'd be in reality doing to others), but the mechanics are the same. It balances out well the snowball effect that slavery had IRL with the game mechanics of a game that's supposed to be fun and played in multiplayer.
Food
If you play on extra large maps (especially on lower difficulties), you'll notice how food can play a very important role in various city sizes across your empire. Some cities will truly be very small cities on the borders of your empire, maybe pop 1 or 2, perhaps only connecting a fur, for example.
The reason why you see so much food around is two-fold, first, smaller maps have resources condensed more, second, players tend to regenerate maps for starts they can be bothered to play. If you start with a plains hill sheep bordering the tundra, you won't be doing all those "double wet corn whips" we are so fond of. You probably won't be doing any whipping and your cities will barely grow.
The truth of the matter is, we don't feel it now in 2019, but food yield of the land does matter. If not for Egyptian wheat, there would be no Rome. Perhaps population wasn't as high 2000 years ago, but the differences in population per square mile were just as drastic, if not more.
This is represented in-game with wet corn (the highest yield food apart from fish, which has its own restrictions), which has a yield of 6. But Biology (or floodplains) puts every farm at a yield of 4. If you don't start on a continent with pigs and corn, but rather with wheat, sheep and rice, the differences between a farmed rice and a farmed floodplain are almost negligible.
What I'm trying to say is, in Civ4 there will always be areas of the map that will be richer in food and poorer in food. It took us (players) time to realize we can't just dotmap cities into a perfect grid, we needed to actively seek out food sources if we wanted our cities to be effective.
Would randomizing food distribution fix the issue of whipping and city growth?
In one word, no. For the same reason you wouldn't want to have traffic signs that change every hour. If you want to have a complex system that works, on its lowest level you need to have consistent pathways to consistent results. Tile yields are the lowest level language the whole game is built upon. By the same logic you apply, why every tank costs 180 hammers? Wouldn't it be more realistic to have it cost between 150 and 220 hammers? Would it be better if if would cost 5% less with barracks, 10% more with camo paint, 30% less in a city with steelworks, extra 50 hammers if overengineering it like Germans? You'd say "wow, wouldn't it just be easier to have it dependably cost 180 hammers?" They tried it with SMAC unit workshop - didn't work really well.
Civ6 kinda does this with districts (removes the lowest level language and replaces it with pink elephants), but it works only because the map is actually really small. If civ4 is a city simulator, Civ6 is a neighborhood simulator.