A New Taylor Swift LP? Metacritic Crunches the Reviews, as Fans Watch.

For Metacritic, a website that collects and quantifies reviews of music, movies, TV shows and video games, a Taylor Swift album drop is one of the best days of the year.

“There’s nothing quite like Taylor Swift,” Marc Doyle, 51, one of the site’s founders, said in an interview last week. “We get a great deal of traffic and user participation, a lot of people sharing it on social media.” In 2020, when Swift released “Folklore,” her eighth studio album, traffic swelled by “roughly a half million page views,” including user review pages, he said.

Metacritic, as its name suggests, aggregates entertainment criticism using a principle of meta-analysis, stripping reviews of their qualitative assessments and assigning them a value between 0 and 100. And it has helped turn pop culture into a game of sabermetrics.

Its tallies, known as Metascores, started off simply as a consumer guide. But over the past decade, as music superfans have gone to battle on social media wielding data — sales and streaming figures, Billboard chart positions, tour grosses, number of Grammys won — Metascores have increasingly become ammunition. Passionate fan armies keep careful track of the scoreboard, and one of the most fervent is devoted to Swift, who will release her 10th studio album, “Midnights,” on Friday.

But who is behind Metacritic, and how does it tabulate its figures?

In 1999, Jason Dietz, like Doyle, a graduate from the U.S.C. Gould School of Law, had the idea for a website that applied meta-analysis to a range of media, and asked Doyle to join his effort to build one. (The movie aggregation site Rotten Tomatoes went live that year, but Dietz was unaware of it.) Dietz, the site’s current features editor, had learned how to code HTML, creating websites including one called List of Possible Band Names.

In late 1999, Doyle’s sister and her husband contributed the majority of Metacritic’s start-up fund. (Earlier this month, Metacritic and six other sites were acquired by Fandom, a developer of entertainment platforms dedicated to superfans, in a deal estimated at $50 million; Doyle declined to comment on the sale.) Together, they began poring over thousands of print and online reviews, compiling them into an Excel spreadsheet and organizing them according to their own schematics — what would soon become their trademark Metascores.

Doyle said the group started making daily visits to publications that run reviews. “Every time they publish a review, you throw it in the system,” he said. “Once you get to four reviews, then you generate the Metascore, which is an average score.” For the games section, the site sends outlets a list of questions “so you can really get to know their scoring philosophy,” he added, a process it has only recently started “for potential movies section partners.”

Metacritic went live in January 2001 with a film vertical and a rundown of how its staff calculated Metascores. For letter grades (used by publications like Entertainment Weekly), an A represents 100, while an F corresponds to zero. For reviews that aren’t assigned an alphanumeric value, the site’s staff — Metacritic currently has five full-time employees who work remotely from Los Angeles, San Francisco, Dallas and Portland, Ore. — will assess the tone of the review before assigning a value themselves.

“We get comments all the time like, ‘This review seems so much better than a 3/10,’ so then I’ll take that comment to another editor, ask what they think, and we’ll give it a reread,” Doyle wrote over email. “Over the years, we’ve also been lobbied to either de-publish a review or drop a publication from our system for a variety of reasons. If it’s not a case of plagiarism or fraud (which usually is self-reported from a member publication), such appeals are generally unsuccessful.”

Before they are averaged, the scores are weighted according to the critic’s perceived prestige and volume of reviews. “From the very beginning we’ve believed there are so many critics out there who are so incredible at what they do — why should they be treated exactly the same as a brand-new critic at a regional paper?” Doyle said.

But Metacritic declined to explain more about which publications and critics are given priority status. “That’s really the secret sauce,” Doyle said. So, how do they avoid biases? “You just have to trust us,” he added. “We’re a professional outfit.” The site makes money from advertising, licensing Metascores and affiliate revenue.

Metacritic’s music section began in March 2001 with a scoreboard of recent album releases. Pulling data from 30 publications — today, that number has expanded to 49 — on launch day, Aimee Mann’s “Bachelor No. 2” ranked highest with a Metascore of 90, while Juliana Hatfield’s “Pony: Total System Failure” landed lowest with a Metascore of 25. (The site has tracked reviews from 131 sites in its history.)

For almost a decade, the section didn’t gain much online traction. Attention remained mostly fixed on the site’s games vertical, which has had the “greatest notoriety and impact,” Doyle explained; its metrics have affected game design, marketing strategies, even employee compensation.

In an interview, the game designer Chris Avellone said that in 2010, Bethesda, the publisher of the game Fallout: New Vegas, “chose to include a clause in the contract that said if you deliver a title with a Metacritic score above 84, we’ll give you a bonus.” The game missed by one point.

Metacritic began playing a larger role in music around the same time. In December 2009, after collating 7,000 reviews, the site released its first top artists of the decade list. Its No. 1 came as a surprise: Spoon, the indie-rock band.

Before long, users began posting Metascores on Twitter as empirical proof that an artist had succeeded or failed. “Kanye got 93 on Metacritic, Taylor Swift got 75. Yeezy Forever!” one fan tweeted in 2010.

“People used Metascores as an argument settler, a metric to put in each other’s faces,” Doyle said. “That really was not the intention of the site, and we hate to see it used as a sword or shield to go into battle with different pop fandoms.”

Critics themselves got caught in the crossfire. In 2016, an anonymous Ariana Grande fan started a petition against Christopher R. Weingarten, a writer who had reviewed Grande’s “Dangerous Woman” for Rolling Stone. In June 2020, a Pitchfork editor was doxxed and threatened after writing what fans perceived as an unjust review of Swift’s “Folklore” that would lower its Metascore.

“I’ve heard from critics whose inboxes have been slammed with complaints,” Doyle said, noting that low scores are often equated with bias. “Despite this, we certainly want to encourage critics to tell it like it is.”

Swift, who has a particularly active online fan base, has been the spark for other Metacritic dust-ups. The music site Consequence of Sound announced last month that one of its biggest regrets was giving her 2017 album, “Reputation,” a D+ and “screwing up the Metacritic score.”

The idea of scoring artists may seem unnecessary or make some critics uncomfortable — Rolling Stone recently abolished its star rankings — but there’s a strong appetite among listeners to have numbers at their fingertips. Perkins Miller, the chief executive of Fandom Inc., compared Metacritic to the N.F.L. — where he previously worked — and its Next Gen stats platform, noting, “There is a greater crossover between sports fans and music fans today.”

Among very online pop fans, data capital is tied to social capital. “Metacritic is always brought up on Taylor Swift Twitter,” said PJ Medina, a 21-year-old fan from the Philippines. “If she gets a high score, it means that she’s critically acclaimed. It means that more people will care.”