Payne: Not All “Review Bombing” Is Bad for Business

November 1, 2024

Not All “Review Bombing” Is Bad for Business

Read the original article on Rutgers Today, October 31, 2024

For a business on the receiving end of “review bombs” – the sudden influx of online customer reviews following a political or cultural controversy – an interventionist approach to content moderation might seem like a prudent strategy.

But a new open-access study by a Rutgers researcher finds that when review platforms such as Yelp enact tough moderation policies in a bid to sanitize political speech, it can unnecessarily constrain reasonable opinions and cultural context that consumers depend on to decide where to spend their money.

“Simply put, everything you think you know about review bombing is wrong,” said Will B. Payne, assistant professor of geographic information science at Rutgers’ Edward J. Bloustein School of Planning and Public Policy and author of the study, published in the journal Big Data & Society.

Online reviews can have a significant impact on an independent business’s revenue, particularly those on Yelp, the leading local review platform in the United States. One study found that a one-star increase in the average Yelp rating causes a 5% to 9% increase in revenue for nonchain restaurants.

To understand the geographic reach of review bombing incidents and how platforms define acceptable speech, Payne assessed Yelp’s moderation of comments on U.S. businesses embroiled in political controversies between 2004 and 2021.

First, Payne created a database of businesses affected by national and local politics. Using news sources to identify specific cases and date ranges, he built a dataset of tens of thousands of political-themed reviews. Topics included the 2016 and 2020 U.S. elections, the Black Lives Matter and #MeToo movements and the COVID-19 pandemic.

Next, he analyzed Yelp’s publicly available metadata for reviews of affected businesses, including review date, username, star rating and user location.

Payne then selected two businesses with large numbers of Yelp reviews for in-depth analysis: Washington, D.C.-based pizzeria Comet Ping Pong (subject of the Pizzagate conspiracy theory in 2016and St. Louis-based Pi Pizzeria, whose owner, Chris Sommers, became the target of online and offline harassment by pro-police supporters after he publicly backed the Black Lives Matter movement in 2017.

In Comet Ping Pong’s case, Payne found that review bombing resulted in primarily negative comments by reviewers mostly on the West Coast – thousands of miles away from the restaurant – while Pi Pizzeria experienced a much more local pattern (largely from the St. Louis area), with an even split of supporters and detractors.

Payne found that Yelp’s automated and human review filtering systems largely responded the same way to each incident, but with considerably different effects. For Comet Ping Pong, of the 283 reviews flagged and removed by Yelp, 229 were negative one-star reviews. By contrast, of the 588 Pi Pizzeria reviews that Yelp removed, most were in support of Sommers’ actions, positive reviews that averaged close to the restaurant’s four-star rating of Yelp-approved reviews.

“Local customers were censored for simply thanking Chris Sommers for standing with them as they marched against police violence,” Payne said. “They weren’t fake reviews about a conspiracy theory; they were legitimate statements by people supporting a business, in this case for the support its owner gave to the neighborhood.”

Payne also looked at Google’s approach to content moderation and found that, unlike Yelp, Google rarely removes politically themed reviews. This, too, can be a double-edged sword; Comet Ping Pong still has dozens of public Google reviews referencing the false Pizzagate conspiracy.

The data does have several limitations, Payne said. First is the possibility that the self-reported location of Yelp users was inaccurate, or that some users could have moved between the time they set up their Yelp profile and when they wrote a review.

Additionally, reviews on Google Maps – a popular Yelp competitor – don’t contain user location information and can be removed by Google without leaving the public metadata traces that Yelp provides for transparency.

As review bombing continues to test review platforms’ approaches to political discourse – the most recent example surfaced this month, when Yelp halted reviews of a McDonald’s franchise in Feasterville, Penn., where former President Donald J. Trump had held a campaign event – Payne said it’s worth considering whether content moderation has gone too far.

The question is particularly relevant for Yelp, which has used corporate communications and review search filters to support Black-ownedwomen-owned, and LGBTQ-inclusive businesses – speech that isn’t permitted by reviewers themselves unless accompanying a customer experience review.

“Having a one-size-fits-all, review bombing or political speech policy can lead to the suppression of legitimate expressions of support for the role a small business plays in the community, as in the case of Pi Pizzeria,” Payne said. “Some might disagree that the political positions of a business owner should guide consumer behavior, but on Yelp, it’s a choice that users can’t even make for themselves.”

Article Citation

Payne, W. B. (2024). Review bombing the platformed city: Contested political speech in online local reviews. Big Data & Society11(3). https://doi.org/10.1177/20539517241275879

Recent Posts

NJSPL Report: Higher Education in New Jersey – A Policy Review

By Angie Nga Le, Stuart Shapiro, and Elizabeth Cooner New Jersey’s state higher education plan, “Where Opportunity Meets Innovation – A Student-Centered Vision for New Jersey Higher Education” (the Plan), was introduced in 2019. It envisions an ecosystem where every...

Healthcare, Policy, and the Opioid Crisis: Bridging Gaps in Access

Healthcare, Policy, and the Opioid Crisis: Bridging Gaps in Access with Zoe Lindenfeld This week on EJB Talks, new Bloustein School Assistant Professor Zoe Lindenfeld talks to Dean Stuart Shapiro about her research on substance use disorders, particularly the opioid...

Public Policy and Surveillance Tech

By Marc Pfeiffer ICMA.org, February 1, 2025 I asked Claude.ai to define surveillance technology. In response, it replied, “devices, systems, or software that are used to monitor, track, and collect information on people, places, or things.” Municipalities use a wide...

Lindenfeld & Mauri Find Uptick for MOUD at FQHCs

Introduction To reduce morbidity and mortality among individuals with opioid use disorder (OUD), the evidence is strongest for medications for OUD (MOUD), which include methadone, buprenorphine, and naltrexone. Federally qualified health centers (FQHCs) are essential...

NJSPL: Ensuring Accurate & Equitable Vaccination Info

By Yonaira Rivera and Vivek Singh  In today’s digital world, large language models like ChatGPT are important sources of healthcare information, particularly for those who face barriers like insurance issues or language constraints. As part of the IMPACT-NJ project,...

Upcoming Events

Event Series Student Services

Bloustein Librarian Open Office Hours

Bloustein School, Civic Square Building 33 Livingston Avenue, New Brunswick, NJ, United States

Have a research or library question you need assistance with? Visit Open Office Hours with Bloustein Librarian Julia Maxwell. Every Tuesday from 12:00 - 1:00 pm in the Civic Square […]