Letter from the Editor: Bethesda and the Changing Face of Game Reviews

More and more companies are withholding review code until on or near release day.

13

Bethesda is the latest publisher to change its media policy. Today the company announced that going forward, it will provide review copies to outlets such as Shacknews only one day before release. As members of the media, we wanted to take this opportunity to explain how we plan to address this shifting media landscape.

To address the decision itself, it's difficult to fault Bethesda for what is essentially a sensible business decision. More than most publishers, its name carries a certain cache with gamers, and its success launching Doom under similar circumstances has effectively decoupled late-breaking reviews from poor buzz. In hindsight we can look at that game as a proof of concept. Its well-deserved glowing praise must have proven to Bethesda that sales don't necessarily suffer without early reviews, and that escaping the stigma of withholding reviews is possible.

Bethesda is also correct in its advice to wary buyers: if someone feels uneasy about buying a game without seeing a critical response, they should absolutely wait to purchase it. This has been the position of many game critics for some time, which may have seemed self-serving, so we appreciate having a publisher's support for the idea. While there is some dissonance to seeing a publisher dole out that advice while simultaneously pushing pre-order bonuses, it's still a solid nugget of wisdom.

It would be inauthentic to claim this move by Bethesda, and similar ones by other major publishers, has not and will not impact Shacknews. It's certainly true that pre-launch coverage of any kind, including reviews, gameplay videos, and interviews, are more successful than similar types of content after a game has already been released. 

However, the difference is probably more negligible than readers may expect, and post-launch coverage tends to be more considerate, thoughtful, and responsive. At Shacknews we pride ourselves in our ability to separate our mindsets from the "launch hype," and our lower-than-average scores reflect that. We take a long view regarding how a game will stand in six months or a year. Ideally, this move will encourage and reinforce such a mindset both among our writers and industry-wide.

That isn't to say that we don't have valid concerns. As members of the media we take our jobs as consumer advocates seriously, and we are less effective in that task if we cannot act on it until after some unwitting customers have already bought and opened their copy. No matter how much we and even Bethesda say so, there will eventually be consumers left disappointed by their purchases because we could not warn them in advance.

This shift also sets games apart from other forms of popular media like novels and films, which means it runs the risk of sacrificing some of games' artistic legitimacy. We can hope that moves like this one lead to more thoughtful critique on the whole, but it's just as likely to end in rushed results. For all the grief embargoes receive as a sign of the cozy relationship between publishers and press, they serve an important function by putting writers on an even playing field. Rather than racing to the finish line, a writer can determine his or her own pace of play, with knowledge of how much time to allow to digest the game and then put a review through the editorial process. Bethesda's decision will most likely incentivize speed over thoughtfulness.

It is also likely to contribute to a general sense that video games do not have a community of critics differentiated from the wider Internet reaction, even as the space for analytical and sober analysis in video games is still finding its footing. If video games are to elevate their own Roger Ebert or Peter Travers, it raises the question of whether such notable names could have ever gained prominence in an environment without having been given access.

Shacknews will continue to cover games from Bethesda, and any other publisher, in a timely manner. We will publish Reviews-in-Progress for late-breaking games like Bethesda's, to give the reader insights of our first impressions, when a full review is unavailable. We will encourage readers to await our full review before reaching a final buying decision. For transparency, we will also update our disclosure footnotes to give notice of our estimated playtime throughout the review-in-progress process, as well as the final review.

We look forward to talking with our readers about what they expect from video game reviews moving forward, and how we can meet the needs of our readers. Please leave us feedback in the Chatty.

Editor-In-Chief
From The Chatty
  • reply
    October 25, 2016 2:25 PM

    Steve Watts posted a new article, Letter from the Editor: Bethesda and the Changing Face of Game Reviews

    • reply
      October 25, 2016 2:56 PM

      Why not stream or record your in progress playthroughs, even without commentary. Seems like with the twitch generation, people may rely more on their favorite streamers than on text, especially in light of the delay of text?

      • reply
        October 25, 2016 3:06 PM

        I'd like to start doing this. I wanted to stream some of my Dark Souls 3 DLC playthrough but the video embargo was too tight.

        • reply
          October 26, 2016 12:30 AM

          Good to hear man, I think that's one way forward.

        • reply
          October 26, 2016 6:18 AM

          Yeah, that's definitely a solid idea for game reviews that aren't sent in advance. Usually for advance copies the embargo includes explicit streaming dates, but for things like Bethesda's copies, I doubt the restrictions will really exist anymore.

    • reply
      October 25, 2016 3:15 PM

      I thinks its because sites leak info early, as well as Twitch streamers. Now some may have permission to put it out early but from what I've heard from PR peeps, is that IGN, Polygon and Kotaku are notorious for breaking embargoes., so are Youtubers & Streamers. It may just be because of that

    • reply
      October 25, 2016 4:21 PM

      I definitely see the difficulty spot this puts reviewers in. Rush something out to get clicks? Take your time, do it right, and lose page views? It sucks.

      That said, I give kudos to Bethesda for 1) stating a policy explicitly and publicly instead of playing the sometimes-we-will, sometimes-we-won't game, and 2) for stating straight up "if you're not sure about a game, wait".

      • reply
        October 26, 2016 6:19 AM

        Agreed on both points. I'm probably a lot less fired up about this than a lot of editorial out there, and part of it is that at least Bethesda was straight-forward about it. They approached it well.

    • reply
      October 25, 2016 5:12 PM

      Do you think you will stop providing preview coverage for Bethesda games after this? It seems like this statement is them just saying "we don't care about review sites anymore, we're just going to rely on youtubers"

      • reply
        October 26, 2016 3:33 AM

        Luckily we have two YouTube channels with nearly 20 million views and a twitch channel, but yeah we will clearly have to change how we approach covering them. That being said, Bethesda and some of their studios have a long history of working with Shacknews and I think we can figure out new ways to work together on preview content.

        • reply
          October 26, 2016 3:57 AM

          I was thinking about this yesterday when catching up on Cassidee's TITANFALL2 review-in-progress. There are all of 3 screenshots in the review and zero video. It's early and I didn't sleep for shit last night so please pardon any unintended bluntness, but that seems wrong. I love that it's all on one page, so I can just read it, and I'm definitely not saying I want a video-review (I actually dislike those), but a 5-minute accompanying video of gameplay seems like such a natural inclusion that I'm assuming there's an embargo on media pre-release. When the "review-in-progress" gets updated to "review" is the plan to add on a video?

          Also, do you guys have internal deadlines for changing a r-i-p (lol that's unfortunate) to a r? If the r-i-p ended with "check back one week from the date of publication for an updated r with video" I would be much more likely to seek it back out and see if additional time (and fully-populated multiplayer) changed her/Shacknews' perception.

          Also also, for this review in particular I didn't see an indication of what platform it was played on. That's not vital or anything, but I always appreciated the "this review was based on a XYZ copy provided by the publisher" tag.

        • reply
          October 26, 2016 5:05 AM

          I would like to see more action on the twitch channel

      • reply
        October 26, 2016 6:23 AM

        I doubt we stop previewing their games altogether. Our first priority is still to the readers, and information about an upcoming game serves the public interest.

        That said, Jeff Gerstmann raised that as a point as well, and I think he's right. This policy means that Bethesda is relying on previews for coverage that glosses over the bad spots with a "maybe they'll fix it by launch" approach, but doesn't ultimately lend itself easily to that final evaluative step. With that in mind, we may be more inclined to give less benefit of the doubt during the preview cycle, since our preview coverage might be all readers have to go on by the time the game releases.

        (We don't shy away from criticism during previews, but the tone is more wait-and-see.)

        But to be honest, previews for Bethesda games are pretty few and far between anyway. It doesn't really make many press appointments during E3 and it hasn't traditionally done much in the way of the usual media junkets. The last one I remember was a pre-release junket for Elder Scrolls Online.

    • reply
      October 25, 2016 5:15 PM

      Unfortunately, this just makes preorder exclusives even worse than they already are.

      • reply
        October 25, 2016 6:29 PM

        The Truth

      • reply
        October 26, 2016 12:31 AM

        God I despise them. They're just hand jobs for physical retail partners who are increasingly irrelevant anyway. It's even more ridiculous that different retailers get different bonuses, so they all get different shorty handjobs and gamers never get the full game.

    • reply
      October 25, 2016 11:34 PM

      I normally buy nothing right away. I read, listen to friends (a little), and a week or two after release I may take the plunge and buy the game. Take No Man's Sky, or rather don't, but I almost got that day one, I've been wanting to play it since the trailer at E3. I held off instead, I figured what would waiting a week or two hurt? I did a good thing. I will use it going forward. On the other hand if Naughty Dog makes another Uncharted or The Last of Us, I don't care what anyone says, it's mine.

      • reply
        October 26, 2016 12:33 AM

        I pre-ordered BF1 after the beta and don't regret it. Also Dishonored 2 because as far as I'm concerned that studio shits gold.

        Otherwise I'm pretty careful - I pre-ordered Arkhak Knight on PC and we all know how that turned out.

    • reply
      October 26, 2016 3:47 AM

      I don't have anything substantial to add to the conversation, but I wanted to tell Steve that was a measured, well-constructed response.

      • reply
        October 26, 2016 6:25 AM

        Thanks, I really appreciate that.

        • reply
          October 26, 2016 8:19 AM

          It was "fun" to read. It wasn't doom and gloom, it wasn't bile and vitriol, it was simply, "here's a thing that I understand, even if I don't really like it, and what I expect us to do in the face of it's unchanging reality."

    • reply
      October 26, 2016 3:53 AM

      So they will send the code month earlier to "influencers", who will speak about their game only in superlatives because if they don't they will not get the code again, and only a day earlier to proper, objective critics. That's shitty, but also to be expected, sadly. And will go on as long as there are enough people who listen to these "influencers".

    • reply
      October 26, 2016 6:10 AM

      I wonder: Would it be worthwhile to reach out to Bethesda for an interview or Q&A to discuss the whys, the impact on review quality, and to address concerns like what Kub666 mentions?

Hello, Meet Lola