Sorry, I thought I was clear. It just doesn't make sense to have anything less. If there's a reason why there cannot be massive stacks, the answer is supply. There's not enough supplies for troops, so you cannot move over more units. If you did, there would be plague, famine, and general bad stuff.
No, the reason is overcrowding. That's both the game play issue and the excuse/appeal to realism. Game play issues is the more important. Resources are already handled under unit maintenance cost.
For supplies, you need a dynamically growing and changing world so that the amount of supplies in a region makes sense, and so a reasonable amount of units can be produced.
No you don't. Those are already handled through the Resources (for strategic bonuses) and maintenance mechanics, which doesn't need to be changed.
Production, commerce, and food will be capped so that there are no unreasonably-sized cities, no unreasonably productive cities, and no unreasonably fertile lands.
Also already handled by the terrain gains of a city fat cross. Cities are reasonably sized and productive based on the terrain they work. Putting a cap on them is unnecessary because it's already limited by food available. (Food-transfer is also a idea that bears looking into, but that's neither here nor now.)
A proper supply dynamism will ensure that there are never areas with an unreasonable amount of resources in a land.
That comes from the map script. It's irrelevant to a unit-per-tile limitation.
To have the game progress logically, there must be a timeline established. If we don't know how long a turn is, how can the proper starvation effects for troops be determined? How would we know how much a city can reasonably grow, produce, or trade, in a turn? How would we know how many turns a spell would take to cast? Et cetera.
How does this matter to point in question? Hint: it doesn't. You're demanding an entire system change for a much more minor change.
The AI must then be made to understand all of these new features, and it must behave like a player because anything less just doesn't make sense.
No it wouldn't, because you wouldn't have to introduce all those new features. The only new feature you'd have to make is to teach the AI to build smarter stacks.
That's the reason it won't happen, because our resident modder doesn't think he can do it, but that doesn't mean it's not possible and it certainly doesn't prove your other complaints.
Why have AI at all if they can't perform at our level? They're just filler.
Because not everyone plays multiplayer. I never do. That's why games have AI.
You're right. The entire concept of levels is unreasonable and should be replaced with a combat engine similar to Dominions 3, so

can occur.
Stop with the strawmen fallacies. I never said the concept of levels is unreasonable. I said horrific stacks of doom (and I'm talking about 40+ unit stacks of tier 2+ units) are unreasonable.
There will always be stacks of doom. There has to be, with how the Civilization combat engine is set up.
There doesn't 'have' to be anything. We aren't even proposing abolishing stacks, just preventing the 50-100 unit stacks of doom that are unreasonable.
Stacks will still exist. They'll still be powerful. But the answer to 'how do I counter a stack of doom' will be 'move two stacks', not 'make a stack twice as big, and nothing after that will get in your way.'
Do you know what would happen if there was a limit to the amount of units on a tile, or penalties for massing units? Firstly: The AI would be screwed so badly. They just wouldn't stand a chance, especially at higher difficulty levels where they have so many more units to amass.
If the AI* is programmed properly, the AI has the same limitations as the player. You don't have infinite stacks yourself. The AI might do even better: being more likely to use forts, and with the extra-troops-in-city increase, they'd be better defensively. There are already modules where the AI is a better defensive player.
Secondly: The defender's advantage would be so ridiculous you would almost never be able to conquer or win at anything when fighting offensively unless you have a stack of doom. Yes, you can attack from different angles, though this is not always possible or feasible.
Not true at all. If you're at a point that you can't conquer a city , than by all rights you aren't prepared to do it properly without abusing broken stacks of doom. Which is the problem that needs to be fixed.
Unit production isn't being hurt. You can still make the same number of units, and you can still invade. You can still attack a city with 2 stacks of 20. If you can't (it's on a peninsula, mountains, etc.), then it's already sacrificed a great deal of production, and will have fewer units to oppose you with.
Neither of those is an actual argument against: you can have the same number of units, more or less, but they're simply dispersed across a front.
Conquest is still perfectly feasible. You attack, kill an enemy stack and move in on the tile. Rinse and repeat. All it means is you have to be careful not to exhaust your stack to a counter attack. Which you should already do, so it's not much of a change.
A complete design of the combat engine would be required to ensure that, in the early game, a player can actually conquer cities and at the late game, that cities can actually be defended from the highest-tier units.
The way to defend a city from highest-tier units is with your own highest-tier units. No redesign of the combat engine is necessary; in the early game it's unlikely for you to be able to reach the tile cap regardless, and later in the game you can build your own units. If you're so far behind a foe that he has higher tier units than you, you're already screwed.
But even so, cities (and fortifications) are still made to be able to host a higher number of units than normal tiles, so a city still has the potential for higher defenses on with weight of numbers on their side (not only those in the city, but defenders outside the city as well).
Magic would have to be redesigned, too, because AoE spells would be even more crippling when you need to position stacks around a city before you attack,
No they wouldn't. You're splitting the uberstack of doom into smaller stacks, not replicating it. A AoE spell would harm just as many people in adjacent tiles as it would if they were all stacked in one.
AoE spells would actually
decrease in effectiveness as the Stack of Doom is dispersed across tiles: more units are likely to be out of the AoE range. Similarly, collateral damage is lessened in scope. But AoE spells won't harm more units than they already do in a stack of doom.
and summoning spells would become absurdly useful in defense, or else useless, because of the combat unit limit or supply concerns.
Summons are already absurdly useful if you have enough casters. This isn't a game-breaking change, though it certainly does make them useful. A summon remains very useful for offense as well, more so for the same reasons even, because it's a renewable soldier who doesn't need to be brought up from the back when he dies. Just send in a cannon fodder, open up the space, and you have effectively an extra soldier per turn per tile to work with.
Supply concerns, of course, remain your own insistent invention. Why should elementals or skeletons require supplies?
Unit specialization would also become even more important than it already is, and promotions would have to be changed to accommodate this.
Specialization is already important, but there's no reason to change the promotions. It's player choice on how to specialize, and all this does is change priorities of how you want to do so.
I think that's everything. I hope I'm not overlooking something.
Yeah, you are: why limiting stacks of doom to reasonable sizes fundamental rework of the entire civ 4 game. Yes, it changes the value of certain promotions and units, but it doesn't break the game in doing so. You're so insistent on 'no, this could never work and you'd have to change EVERYTHING' that you're missing that (A) no you don't, and (B) it can be retooled to fit the idea and still work better. No one is suggesting stacks of ten per tile, 20* would probably be fine, but there is no justification for 50 unit stacks. Those rightly demand your supply mechanics than my suggestions do.
*And that's neither a final number or universal for all: some units could be reduced in presence, while others 'cost' more space.