1. We have added a Gift Upgrades feature that allows you to gift an account upgrade to another member, just in time for the holiday season. You can see the gift option when going to the Account Upgrades screen, or on any user profile screen.
    Dismiss Notice

Do you believe they actually play tested Civ 6?

Discussion in 'Civ6 - General Discussions' started by tanktop4158, Dec 8, 2016.

  1. TheMeInTeam

    TheMeInTeam Top Logic

    Joined:
    Jan 26, 2008
    Messages:
    25,876
    It's disingenuous to assume that playtesting wasn't done, correct. Quite likely, a heavy proportion of the issues with the game were identified. Usually only off-kilter things that require atypical things in sequence or rarely occurring/hardware specific issues get past testers.

    It is also disingenuous to look at civ 6's state and say "well, dev companies don't fix every bug they find". No. They knew the controls were broken and that the tuning in the game was nowhere near balanced and were perfectly happy to release the game while it was still in beta, just like they did with Civ 5. I hope they don't leave the controls broken forever, like they did in civ 5.

    Finally, I find it very doubtful that anybody at Firaxis or 2k can make a reasonable, evidence based estimate of the hit or benefit to the bottom line of bugfix cost vs hit to sales on a bug by bug basis, or even close. How many people won't buy a DLC in a month or two because unit cycing doesn't work and the controls are bad? It's non-zero, but putting a precise number on that for estimation purposes would cost more than either fixing the controls or leaving the game broken, so they're not going to do it.

    They do.

    But they might as well not, for all the good it does. It's not like you can listen to nuanced balancing comments from high level players when it's a time struggle just to get the game out of the door in a state where sub-average players don't realize it's broken.
     
    Akbarthegreat and Lewi11 like this.
  2. Siesta Guru

    Siesta Guru Prince

    Joined:
    Dec 2, 2007
    Messages:
    498
    Location:
    The Netherlands
    I'm sure they did plenty of testing. One problem tends to be though that most testers don't actually have either the skill or the responsibility to go into the code and fix small things themselves. This means that they will have to accurately communicate their problems to the devs. The devs are probably swamped, and need to prioritize core features or gamebreaking bugs, so they aren't playing the game much. Then there'll be some organisational overhead, when say a tester and the dev may have agreed on a change, but the designer is not cool with it. So everything needs to go through the designer first before it even reaches the dev. In the end, a lot of what the testers may experience gets dilluted and never ends up in the game.
    It's one of the reasons why the big studios so often struggle with the finer details and the finetuning, while small indie companies started by 3 programmers and an art guy do better.
     
    George Abitbol likes this.
  3. Tuvok694

    Tuvok694 Civ6 addict

    Joined:
    Nov 11, 2002
    Messages:
    329
    Gender:
    Male
    :thanx: :agree:
    Thanks for this statement!!! Those were my exact thoughts during my first two or three games of Civ 6.
    And I still cannot believe how they could release this wonderful game with such obvious flaws. :wallbash:
    IF they tested it before release, then WHAT exactly did they test, if two of their test-results were not "ooops we forgot to implement sentry" and "having no build queue gets quite annoying"?
     
    Lewi11 likes this.
  4. Victoria

    Victoria Regina Supporter

    Joined:
    Apr 11, 2011
    Messages:
    11,397
    sounds like a Microsoft product, or an Oracle product... or a VMWare product. In fact any jolly product.

    When a project goes live at our work the customer has to test and accept it. I guess if we were paying like 2K each we would get that ability.
     
  5. TheMeInTeam

    TheMeInTeam Top Logic

    Joined:
    Jan 26, 2008
    Messages:
    25,876
    I'd be willing to bet that beta testers mentioned these things and the issue was ignored. Firaxis either doesn't know how to make proper controls, or they just weight it so lowly that they don't bother. The best game controls in the series was civ 4 and that had a lot of issues with basics too, just less than 5 or 6.

    Fun trivia: Every patch of every version of civ for the past > 10 years has involved the game forcibly selecting units different from the one you selected, such that you can accidentally move the wrong unit. Three mainline entries to the series, dozens of patches and DLC, multiple expansions have not been enough to make unit selection work consistently for even a single patch.

    Broken unit cycling is not new, but you'd think maybe one of the line of developers would have considered that the controls not working could be an issue at some point. Alas, that is not a requirement in TBS/4x genre apparently. Can anybody REALLY blame this kind of longstanding, obviously-confirmed issue on testing? It would be inane to NOT know it if working on the game, even before development of civ 6 began.
     
  6. rschissler

    rschissler King

    Joined:
    Dec 18, 2003
    Messages:
    789
    Location:
    So. Cal, USA
    That's what I could never figure out. It seems like all major reviews are 5/5 or 9/10. It's like the reviewers were in such a hurry to get their review out ASAP, to garner more hits, that they hardly played the game.
     
    Zuizgond, Lewi11 and Fromage10x like this.
  7. tanktop4158

    tanktop4158 Chieftain

    Joined:
    Jul 20, 2012
    Messages:
    66
    ok let me explain further...yea the game seems free enough of bugs...I mean play test for play-ability enjoyment,its seems to be missing all the components that made previous Civs great
     
  8. TheMeInTeam

    TheMeInTeam Top Logic

    Joined:
    Jan 26, 2008
    Messages:
    25,876
    There are actually a few, but the opaque nature of the rules makes it hard to identify more.

    What made previous civs great? That's a non-trivial question to answer.
     
  9. Stringer1313

    Stringer1313 Emperor

    Joined:
    Sep 10, 2014
    Messages:
    1,103
    It has to do with publisher deadlines. There is never enough time to finish a game. Ever. So they do the best they can with the limited time that they have. We're noticing all the problems now, but there are problems that were fixed that we didn't know were ever problems but were of a higher priority.
     
    George Abitbol likes this.
  10. Gort

    Gort Emperor

    Joined:
    Nov 7, 2010
    Messages:
    1,518
    Oh, of course. It might never be addressed, like many of the Civ 5 bugs. Is that better? Or worse?
     
  11. TheMeInTeam

    TheMeInTeam Top Logic

    Joined:
    Jan 26, 2008
    Messages:
    25,876
    It's not good to give developers a pass for longstanding, systemic project management failures though. The release state of games in more competitive genres is far superior, in everything from performance to MP to balance to UI. You can't blame all of these struggles on the fact that the game is TBS, not when many of these problems were solved by TBS games before quite a few posters here were born.
     
  12. Stringer1313

    Stringer1313 Emperor

    Joined:
    Sep 10, 2014
    Messages:
    1,103
    Fair point. The excuse i suppose is that they built everything from ground up and just didn't have time or didn't prioritize duplicating code from past iterations that worked.
     
  13. Gorbles

    Gorbles Load Balanced

    Joined:
    Nov 24, 2014
    Messages:
    5,477
    Location:
    UK
    Nobody is suggesting that the developers be given a pass. I think it is important that blame is placed at the right feet, however (i.e. the publisher, for the case of overarching deadlines and so forth). UI is important, but there will be other things that are considered more important. A game is playable (to whatever state) with a bad UI (or AI, or whatever), but a game isn't playable if it crashes, or if it has obviously-missing assets, etc. The priority of "playable" is defined against market expectations and how much of your target demographic is perceived to prioritise each area.

    As such, UI is obviously (for better or worse) further down on the list of consumer priorities. Because if it wasn't, and it actually had an observable impact on sales, business management would force it up the ladder.

    I didn't say it was better or worse. I'm commenting on the existence of playtesting, and the fact that reporting will have occurred, that's all.

    There seems to be too much of an emphasis on "Firaxis are bad / incompetent / lazy", rather than an analysis of the industry and how it leads to an acceptance of trends that result in a specific end product.
     
  14. drubell

    drubell Prince

    Joined:
    Sep 20, 2010
    Messages:
    513
    Okay so what you're complaining about here isn't on the onus of playtesters or QA. You're now delving into the realm of the design team's responsibilities.

    And I know I was curt in my responses on my posts on the first page, but I think it's important to know whether or not your complaints toward a certain part of a video game have merit, and whether or not you know who your complaints should be directed to.

    So if you're trying to complain about the design:

    "play-ability enjoyment": What is this? Whether or not you're enjoying the game? If so, what aspects are you enjoying or not enjoying (which is also by the way a question almost any playtester will be surveyed for)? Otherwise, "play-ability enjoyment" is not a measured value.

    "missing the components that made previous Civs great": What is this? What components are missing? What components were in previous Civ games that were great that Civ 6 does not share? Are these measurable components for you?
     
  15. drubell

    drubell Prince

    Joined:
    Sep 20, 2010
    Messages:
    513
    You were referencing, specifically, "bugs and balance issues"

    QA searches for bugs, yes. What bugs are you referencing?

    Playtesters, on the other hand, are not responsible for release bugs.

    Balance issues are not in the realm of either the QA team or the playtesters.

    Or in your post "game worked properly": What do you mean by working properly? The launch was stable and the multiplayer is stable. Is the game crashing for you? Or do you mean that "properly" is a subjective view for you? If so, what is the game not properly doing?
     
  16. Uncle_Joe

    Uncle_Joe Prince

    Joined:
    Sep 9, 2005
    Messages:
    476
    As a former QA Manager in the industry, the most frustrating part of our job was coming to the end of a development cycle and seeing so many bugs/issues being flagged as ‘Known, shippable’. That basically means exactly what it looks like…they acknowledge that the bug/issue persists, but they are not going to dedicate resources to correcting it before the game ships. Some would make it onto the post-release list for fixing as resources allowed but the vast majority were just considered as livable.

    The other frustration QA faces is delivering gameplay feedback and being told that ‘x’ or ‘y’ is ‘AD’ or ‘As Designed’. Yes, we accept that it is working as designed, but the design itself is not good lol. We back-handedly referred to those issues as ‘APD’ or ‘As Poorly Designed’ ;)

    In any case, rest assured that any half-way decent QA/playtest team most likely uncovered many of the same issues that you’ve found for Civ6 but for whatever reason, those reports are probably sitting in a bucket somewhere with either a KS or AD tag. As patches and DLCs roll out and certain coders find they have some bandwidth, some will be addressed. I’m sure they have a Community Team that keeps them apprised of the hot-button issues on the forums and other media and they use that feedback to prioritize changes and fix lists.

    But yes, I’m quite sure the game was extensively play-tested throughout a lot of quality feedback was likely delivered.
     
    4N4C0ND4, c4c6, REDfeller and 4 others like this.
  17. TheMeInTeam

    TheMeInTeam Top Logic

    Joined:
    Jan 26, 2008
    Messages:
    25,876
    It may, or maybe not be, fair to blame the publisher for overarching deadlines. You do need a release date eventually. Since the developer and publisher are not the same people in the case where placing blame between the two is relevant, the assumption is that the developer is providing an estimation of completion time based on the anticipated features of the title, and negotiating that based on what the publisher is willing to do. In the case of a game where developers are still fixing crash bugs close to release date, this estimation was likely *grossly* misaligned with a realistic anticipated timeframe.

    That puts the publisher in a tough spot, because money/time is finite. I can easily envision scenarios where the publisher's influence on the release date causes undue pressure and influences the quality of the vanilla game also, but without insider information it's not so clear to conclude that vs developer trying to be too ambitious within an agreed upon timeframe. The planning fallacy isn't an easy thing to beat.

    I call (potential) BS that it's further down on consumer priorities. From a market research perspective it's a noisy measure with a variable minimum threshold between consumers (IE no interface = no players, terrible UI = some players, decent/great UI = more players respectively, with it very difficult to measure how many more). Rocket league and sports games are pretty successful. Would they be materially less so if they require triple the current inputs to do anything in game? Very likely, same with RTS.

    In TBS, the ability to input quickly is not crucial to being able to play at all...and yet you do see generic complaints about the pacing of the game, that war is too tedious, or that it's too much of a slog to play through the late game. These consumers are complaining about the UI. They don't state it as a UI issue though. They might not even recognize it's a UI issue. Yet what's the most obvious, lowest hanging issue with regards to how long it takes to get through the game? It isn't AI quality :p. Considering the % of players winning on deity right now, it is very likely that the game's tuning between choices and UI are the largest contributors to criticism of its pacing and depth.

    My primary gripe here is that UI is, in principle and in practice, clearly easier to do well than amazing AI, graphics everyone agrees are amazing, breakneck performance, game balance, historically accurate civ representations with good voice acting, etc. The easiest way to find evidence for my belief is to look at the existence of these things in other games, both within and beyond the TBS genre. There *are* decent to excellent UIs, and they don't take nearly the budget of some of the things civ 6 actually did pretty well.

    It's not competent.

    Advertised features do what they advertisement claims (serious problem in 4 and 5 MP for example when released). Controls in the game doing what the game says they will do fits this category also.

    Meaningful decisions (to the outcome of the game) per unit time spent playing the game. By increasing time the player spends doing nothing while playing, you will necessarily lower engagement. To how much extent is variable between people. Anything from long turn times to being required to input 15 extra commands that are rote memory/canned actions to simply being forced to wait on animations that a given user doesn't care about can influence this.

    On the other side of the ledger is the number of potentially meaningful decisions. You can improve decisions/time both by making the game play more seamlessly, or introducing more choices that matter. The latter is more difficult to do, based on any example I've seen, unless the game is already a very seamless experience with excellent controls (like Rocket League, where meaningfully improving its UI or controls would be very challenging).

    Those are the measurable that were clearly present and better in previous titles. Civ 6 still has leaders, unique civ theming, animations, important decisions to make, etc etc. So a complaint of difference between the two games is not likely to be that Aztec are different in civ 6 or something. What is missing is that the strategy game has less strategy while you play it, and more of what equates to shuffling cards.

    I don't doubt it, despite that my experience is more limited than yours I do have some. It's frustrating to see people blame blatant misprioritzation and bad project management on the QA team, as if QA actually has a say in what happens.

    Basically, I'm well aware that the developers considered the UI lying to the player as "shippable", in this game and others. And it directly damages their credibility for me that they do stuff like that or leave the player no means to determine rules of the game other than through trial and error. You expect that out of indie devs who make games like "I wanna be the guy" type games where fake difficulty is a selling point, not a AAA title selling in the strategy genre.
     
  18. drubell

    drubell Prince

    Joined:
    Sep 20, 2010
    Messages:
    513
    In most cases, at least if you have a good design and programming team, it's not disingenuous. Though yes I know exactly what you're talking about having also having had a leading QA position.

    Any (ANY) change has the potential to absolutely break a game's build. Depending on how close the project is to the launch build, there truly is the reality that a change can do more harm than good, even if the change would make a boneheaded idea into an idea the entire team would agree with. If the game is in the playtesting phase though, it's typically too late for major systems to be changed for the release build. There is always potential for minor change to the code to have serious, unintended consequences that are much worse than the original, proposed "problem."

    It is important to have someone be able to acknowledge that a project has a fixable problem, but be able to determine whether or not it's reasonable to fix because the release build or not.
     
    4N4C0ND4 likes this.
  19. oov

    oov Chieftain

    Joined:
    Jul 25, 2012
    Messages:
    20
    TheMeInTeam is dead right. UI needs major help. I think it's going to come down to modders to clean it up unfortunately.
     
  20. megabearsfan

    megabearsfan Prince

    Joined:
    Jan 17, 2006
    Messages:
    523
    Location:
    Las Vegas, NV
    There's probably reasons for why some of the things you listed might not be in the game. For example, the district-placement mechanics may have introduced some technical complexities to a production queue.

    As far as reviewers giving the game very high scores, I think that most reviewers approach a game knowing that it's going to be patched and updated. So the review is focused more around the framework that the game creates. As long as the bugs that are present aren't game-breaking, reviewers are probably going to assume that most of them will be fixed post-release. Unfortunately, the increased cost of making games nowadays means that almost all released games are effectively paid-for-betas.

    I personally love Civ VI (especially compared to the launch of Civ V) (http://www.megabearsfan.net/post/2016/11/02/Civilization-VI-game-review.aspx), and my experience so far has been that the game is very stable and works. There are issues, obviously, but I see a lot of the issues listed as being more nags & nitpicks rather than game-breaking flaws (http://www.megabearsfan.net/post/2016/11/10/Civ-VI-nags-and-nitpicks.aspx). They bother me, but not enough to make me stop playing the game. I do agree that A.I. issues need to be ironed out, as those are the most likely to make me stop playing as I go up in difficulty (currently playing king until I get a better feel for the game).

    With a game this complicated, if you wait for every bug, imbalance, etc. to be fixed, then the game would never release.
     
    Last edited: Dec 9, 2016

Share This Page