Who told you that Civ4 tried to "mitigate" SoDs? If the Civ4 design team had wanted to do that, then there'd be lots of possibilities (stronger stack counters like collateral damage, give units in a stack negative modifiers, implement higher logistics costs for large stacks, implement a chance that a non-optimal defender is chosen which increases with stack size, etc.). Mitigating stacks is easy. The problem is whether the AI can grasp the rules.
Civ4 chose the route to make large stacks the best option most of the time, and Soren wrote an AI that coped with them pretty well (assemble stacks, move and attack with them in a coordinated fashion, transport stacks across oceans, etc.). It's a design decision that paid off with an AI that actually poses a challenge for new players. Calling it a "failed experiment" by supposing that the devs wanted to mitigate a feature that you don't like is a bit of a stretch, imho.
Actually, I remember reading someplace when Civ 4 came out that things were incorporated in Civ4 to punish the tendency to have SoD's. Specifically, artillery/bombardment affecting every unit in the stack was designed with this in mind.
Human SoD vs AI SoD was a joke. Send in several artillery units to weaken everything (they would die in the process, but oh well or you could use bombers many times for the same effect if they were available) and then clean it out with little further loss to you and that is the point. It gets to the point that the human player can inflict massive casualties against the AI so that no matter how much the AI has, it doesn't really matter to a point because a human will win the battle of attrition by a huge amount.
AI attacks on a stack are tough from the standpoint that typically, a full attacker will lose to a full defender in many cases, especially in the case of Civ4 where the stack automatically switches to the best defender based on who you attack with. The very wounded defender is then covered by a fresh unit against a fresh attacker. Repeat. Only if the attacker has a massive superiority in numbers will the defender actually take losses from their damaged units being attacked repeatedly. More often than not, the result is the attacker loses a ton of units while the defender takes few losses and winds up with a lot of damaged units that simply retreat back and heal up.
1UPT changes all that from the standpoint that the human can't hide their units in stacks and force a huge disadvantage when it comes to attrition. Your lead unit stands a really good chance of dying if attacked by two enemy units or a unit that just took a hex stands a good chance of getting pushed back by the AI since it probably took damage and if the AI can counter attack, it will likely die. Granted, the AI will still take a unfavorable attrition rate, but nothing like it did in Civ 3 or 4. At least this is what I think was going on when the decision to adopt 1UPT was made. Note that while I think this was a theory they had in mind, the implementation of an AI to make use of this is another matter.
One of the trends they have followed since Civ3 is lower unit density overall in game. Civ3 could grow to ridiculous unit amounts in game. Civ4 lowered that and Civ5 continues that trend, especially with 1UPT.
I noticed there were a lot of mentions about stacking in other games. Most of the classic strategic games (Third Reich, World in Flames, etc) feature 2 normal unit stacking. Panzer Blitz/Leader was 3 per stack in most cases. Typically, the more tactical a game becomes, the more stacking there is (Squad Leader for example to continue the mostly A/H line).
To me, I like the 1UPT, but I would not be upset if they bumped it to two and that would fit the scale of the game better I think while keeping mostly intact some of the ideas of the advantages of 1UPT in AI vs Human combat when it comes to attrition ratios. This would also allow an escort for troops at sea and troops to cover either cav/armor or artillery units.