Metacritic has a review bombing problem. Here are 6 ways to fix it.

Critics and fans have come to regard a Metacritic score as an essential measure of a game’s quality. But Metacritic has an increasingly glaring flaw in its system.

The Last of Us Part II is the latest example of “review bombing” on the review aggregation site, which is owned by CBS Interactive. The phenomenon refers to a deluge of negative reviews — often from politically right-wing users — who intend to hurt sales of a product they decide offends them or is otherwise culturally or politically objectionable. These negative ratings are almost always applied by users when the product -- a video game or a movie, typically -- is very new or even unreleased. Other widely reported cases in recent years include Rian Johnson’s divisive 2017 Star Wars movie, The Last Jedi and 2019’s Captain Marvel.

Like those movies, The Last of Us Part II is drawing ire from gamers for what they see as its “SJW political agenda.” In this instance, Metacritic user reviews object to developer Naughty Dog’s decision to show queer relationships and gender non-conforming characters. Others straight-up admit they haven’t actually played the game.

As of June 24, more than 75,000 user reviews of The Last of Us Part II had been posted on Metacritic. Almost 44,000 were negative. The game only came out June 19, and takes around 25 hours to complete, but the first user reviews appeared just after the option to post user reviews unlocked, which appears to happen around midnight on release day. This suggests many early reviews came from people who have not actually played the game but want to discourage others from doing so, thereby “punishing” Naughty Dog for its creative choices.

Metacritic has been cagey about how its aggregation process works. In its FAQ section, an entry asking for details about critical weighting is answered with a snarky “Absolutely not.” Elsewhere on its website, Metacritic describes the process in the following vague terms:

Metascore is a weighted average in that we assign more importance, or weight, to some critics and publications than others, based on their quality and overall stature. In addition, for music and movies, we also normalize the resulting scores (akin to "grading on a curve" in college), which prevents scores from clumping together.

It’s not clear what “quality” and “overall stature” refer to here. Is it a particular number of readers each month? Flowery prose? There are several quantifiable metrics one could use to determine it, but Metacritic doesn’t disclose what those are, even for news outlets interested in getting its reviews included in the score. Most outlets need to apply to be included on Metacritic, and must score game reviews to be considered.

Since 2015, Kotaku and Eurogamer have opted out of scoring reviews in response to what they see as Metacritic’s undue influence on the industry, but that’s a luxury smaller outlets often cannot afford. Getting on Metacritic may mean more readers, greater visibility to publishers and developers, and thus better access to early copies of games and embargoed information.

In May, Metacritic’s audience size was 2.3 million, according to Comscore, which is the industry’s go-to source for site traffic. Kotaku was 3.9 million and Eurogamer was 5.3 million on Comscore in May. (For transparency's sake, Inverse’s Comscore count was 15.7 million in May.)

While the process of becoming an official Metacritic-approved outlet is obtuse, there are hardly any barriers to entry for armchair user reviews. If you can bother to spend about one minute setting up a user account, you’re good to go — and review-bomb, if you so wish.

User-generated content is likely a lynchpin of how Metacritic draws traffic and therefore makes money, so it’s probably not going anywhere. Bringing more people to the platform — even if it’s to write a hateful shitpost about Abby from The Last of Us Part II or Brie Larson’s Captain Marvel — gives Metacritic leverage to set higher ad rates, both to advertise directly on the Metacritic homepage or to use aggregated scores in promotional materials for games or movies. There’s nothing particularly nefarious about this business model; attracting users to relevant ads is how many websites work. But it is also probably a reason why the site has been slow to address the problem.

Metacritic says it takes score manipulation seriously and enforces numerous policies to maintain score integrity. A moderation team regularly reviews the site and removes offensive entries. Users can also report abusive posts and flag unusual behavior for the moderation team for review and possible removal. Even so, a lot is slipping through the cracks. While most review bombers have enough common sense not to say the quiet part loud, unambiguously offensive content still gets through. 70,000+ reviews is a lot for any team to read over in just a few days.

CBS Interactive declined to respond to specific questions relating to TLOU 2 or future plans to limit review bombing, but we have a few ideas of our own. Here are six suggestions for addressing the problem, some of which have proven successful on other platforms with a lot of user-generated content.

1. More human moderators

All that’s required to post a Metacritic user review is an account and an email address. User review posts go live on the site immediately. A team of human beings should evaluate user reviews before they go live, weeding out baseless claims and one-off comments to make review bombing less effective.

It’s not feasible to make this happen for every single game, but you can usually spot the ones that are likely to attract this kind of attention a few weeks out. In the case of The Last of Us Part II, this icky conversation has been going on since major story details leaked in April. Metacritic could reach out to editorial staff at its approved media outlets to flag up potential upcoming targets for review bombing and plan ahead to allocate mods for reviews of a specific game.

2. Incentivize reviewers to engage long-term

Twitch gamifies progression on its platform, allowing newbie streamers to unlock achievements to access new functions, most notably the ability to solicit paid subscribers. A simplified version of this for Metacritic could involve writing a small number of mod-approved reviews before getting instantaneous publication access, to discourage one-off flame posts. User badges indicating high number of reviews and positive community feedback could be helpful, too.

3. Prohibit super-early user reviews

Some games are relatively short experiences or lend themselves to being played in quick bursts, like fighting games and platformers. The Last of Us Part II takes 25-30 hours to beat, so there’s little chance thousands of people can fully evaluate the game just a few hours after launch. (Rotten Tomatoes did something similar after Captain Marvel, removing the ability to leave a user review before a movie’s release.) Pushing back the timeline for posting user reviews wouldn’t eliminate the problem, but may discourage it to a worthwhile degree.

Update (July 17): Metacritic has quietly extended the period of time before users can review a game until 36 hours after launch. The page for the new PS4 game Ghost of Tsushima, out today, contains a note that says "Please spend some time playing the game. Come back to review it starting at 12:00pm PST on July 18." Original story follows.

4. Confirm ownership of the game

This could be an opt-in functionality for avid armchair reviewers to expedite a more rigorous moderation process, allowing them to share their digital storefront credentials to confirm they actually own a copy of the game before reviewing. Making it mandatory probably wouldn’t work due to privacy concerns, but adding it as an option can let readers know when those sharing their opinions have actually experienced the game for themselves.

Amazon gives “Verified Purchase” reviews higher placement on an item’s page, and a similar verification feature on Metacritic could enhance the visibility of high-quality user feedback.

5. Make media outlet selection more transparent

Suspicion of media outlets is a key motivator behind review bombing, given the implicit incentives from publishers to score a game favorably in order to ensure continued early access to future games for coverage purposes. This is an industry-wide problem that’s far bigger than TLOU 2 or Metacritic.

That said, if Metacritic were more upfront about its selection criteria for media outlets and how those scores are weighted, it may help to inspire greater trust in aggregated critics scores. This won’t solve the many underlying issues surrounding access in games journalism, but would provide Metacritic users worthwhile context.

6. Remove user reviews altogether

There’s already a site that does this — OpenCritic. OpenCritic's cofounder and CEO Matthew Enthoven tells Inverse that one of the site's aims was to be a games-only alternative to Metacritic.

"When we launched, we saw numerous gaps with what Metacritic was offering," he explains. "That's why we launched with features such as crediting authors, including non-scoring reviews, and displaying both the percent recommended and overall percentile ranking."

Though OpenCritic plans to add user reviews at some point in the future, Enthoven says "review bombing is absolutely the reason" the site has not yet done so. "We don't think we can have an effective user review system without some level of moderation," he adds. The site has considered several ideas for how to do so, without opening the door to review bombing, such as moderating all user reviews, requiring a minimum account age, and blocking user reviews until two weeks post-launch.

"We want to make sure that our system supports users who put in serious thought and effort, while minimizing or eliminating low-effort or fake reviews," Enthoven explains.

These are just a few solutions among many possibilities. None of them are perfect fixes or easy to implement overnight, but any one of them would demonstrate that this is an issue Metacritic takes seriously. Right now, it’s not doing enough.

Until review bombing is dealt with, review aggregation platforms like Metacritic will continue to be yet another battleground in an exhausting culture war, rather than a reliable resource for recommendations on a new game like The Last of Us Part II.

Tomas Franzese contributed reporting to this story.