You've got a point, even with 100% coverage and extensive balance testing you can't find everything. I agree with you entirely, can't always make assumptions about what the project lifecycle was like. I was admittedly being too cynical in my post.
I think the main problem is the current methodology of software development. Object-oriented design is thankfully, finally replacing function-oriented design (and very popular in the game industry), but it's still lacking in many areas. Requirements are often poorly done or overlooked, a major source of project failure in a lot of situations I've been involved in. It also can often overlook critical domain issues, great example being the questionable Patriot missile system deployment two decades back - got all the functionality right, but didn't properly account for domain issues regarding human operators. The problem with OO or function-oriented development is it can get bogged down in the details and overlook the big picture. I feel the emerging methods of goal- or agent-oriented design could help with this somewhat.
One example of this in CiV is how gold maintenance costs seem logical from a low-level gameplay perspective, yet in a broader sense limit player choice (passive drain on gold, which can be used for much more decision-making now in V) and maintenance feels like it's penalizing the player after spending all that time building something.
There's always something to be said for outside perspectives, too. Even some testers directly involved in the dev and testing process, like Valkrionn, still feel the game could use some adjustment.
Another issue is how the majority of fault localization techniques rely on a single-fault assumption. Encounter a bug, balance issue or something else you might think is broken, go look for the likely cause. It assumes a one-to-one relationship. I've been involved in research using clustering techniques for multiple-fault localization, many-to-many situations in programs to hopefully alleviate this some day and provide more automated tools. I'll admit that even I'm using a single fault ad hoc approach for Civ V right now, whack-a-mole style, yet broader or semi-automated approaches typically have to be built in through each iteration of the whole development cycle though.
This is basically what I think the problem is - yes, they had unit testing and balance testing done, but those sorts of things can get too focused on small issues and narrow scopes, and overlook integration and how things work together on a larger scale. I know they likely did plenty of integration testing too in this regard. Still, there's always room for improvement, and I've just been trying to improve the 1% of the game I feel is still "not quite there" yet, little things here and there.
I'm not sure any of this make sense... somewhat caught me rambling there! Well, off my break so back to work.
, adds +2
etc). If you go from an Archer to a Crossbowman (or worse a Cho-ko-nu) while still in the early ancient era, that's a dramatic jump in power for a game that focuses on fewer military units. That single ancient ruin could let you kill 1 civ before tech levels can equalize, since most will have only one city, or at the very least choke the civ's development. It's arguably even more powerful than the free tech reward.
