Jesus is Lord33 said:
Come back to me in a month with all their "true answers" catergorized into meaningful statistics. Until then, you cannot call this evidence. It's called confirmation bias.
I had to stop up when I read this, as the usage you gave for confirmation bias is so innacuarate it had my jaw drop.
Confirmation bias is a phenomenon common to all humans (and probably a lot of animals too) whereby unusual events are assumed to happen a lot more regularly than they actually do. To use an example from another game originally brought up in this thread by JLoZepelli (sp?) Football Manager (FM for short). Over at the FM official forums, where I'm also active, there are 3 main types of thread (which constantly pop up) which showcase this 1) "Why do I always concede an equaliser/winning goal in injury time", "why do I always get an injury to players after 3 subs" and "why does my star striker never score 1v1s". Now all of these are cases of a few incidences of the above being inflated to "happening every game," thus confirmation bias.
Confirmation bias is not now nor ever will be a statistical metric which will give you information about the numbers, it is a psychological metric which gives you information about people and how they tick.
Now onto my own musings, I was thinking about John Shafer running through the thread and it struck me that he is a bit of an "ideas man" who seems to have a bit of a problem with implementation and finishing projects. Examples from his Civ work:
Final Frontier: beautiful concept, never anywhere near to completion until far later with work from voluntary modders.
Espionage: Good idea, poorly implemented, with some very overpowered tactics (to my mind no thought to how it would mesh)
Civ V: Some very good ideas, lots of initiative, but poorly implemented, no thought to how they would work in a Civ environment or whether it would make the game enjoyable. And then some badly broken mechanics (city states) and some very well done mechanics (Social Policies).
So I am actually optimistic for future projects from John (maybe not this year or next but down the line) as I feel what he needs is the discipline to work thoroughly and the patience to see things through. It is a case of better applying his ideas, keeping what works, fleshing them out, and jettisoning the ones that don't fit, and I personally think a lot of this will come with experience.
So best of luck to you John, and I hope you are now and always will learn from any mistakes you make.