• Star rating systems alone cause products to be rated more similarly to each other
• This makes it harder to differentiate between quality of products
• Complementing with written reviews could increase informativeness of ratings

During the worst of the Covid-19 pandemic, full lockdown measures were enforced in many countries around the world. This included being unable to leave your home to go shopping in-person for weeks on end, something most of us had taken for granted pre-pandemic.

Thankfully, many stores and brands already had an online shopping presence, or were able to implemented one swiftly during lockdowns. We couldn’t leave our homes but, if we needed something, we could order it at the click of a button. While ordering items through a screen does come with some challenges, such as being unable to feel the material or gauge the size of an item of clothing, it also has its benefits, such as each item being accompanied with reviews and a star rating.

When shopping in-store, while you can see the item in-person, see how it fits or looks in reality, you cannot find out how it fits long-term or whether the material deteriorates quickly over time, for example. Reviews and ratings available through online shopping provide a wealth of information about previous consumers’ own experiences with the item. Platforms such as Amazon, Google, and Glassdoor, use such rating systems to indicate product quality to customers and offer shoppers reassurance.

However, critics have previously argued that ratings on Amazon were too high and that “paid for” ratings had become a problem on the platform. Therefore, in 2020, Amazon changed their policy: Whereas previously it was only possible to rate a product if the customer also provided a written review, the change to the policy means that it is now possible to submit a rating on Amazon by simply tapping a scale from one to five stars, requiring less effort from customers.

The hope from industry experts was that genuine customers would be more likely to leave an authentic review if it was easier for them to do so, overwhelming the number of “bought” five-star ratings so they had less influence on the overall rating. If this was true, then average ratings should decrease, and standard deviation increase – meaning that ratings for different products are less similar to each other – across products.

So, how does this change to a simpler system affect the product reviews?

According to research by Matthias D. Mahlendorf, Professor of Managerial Accounting from Frankfurt School of Finance and Management, and Oliver Hegers from Amsterdam Business School, it might not have as beneficial an impact as previously hoped.

They found that an easy-to-use online rating system, such as a star rating without a written review, decreases the informativeness of the ratings and blurs the lines between ‘good’ and ‘bad’ products.

The research focused on the rating of books purchased through Amazon. To study the effect of the policy change on the informativeness of ratings, they investigated whether average ratings and standard deviation increased or decreased across products.

They obtained 70 million product-week observations for around 400,000 books, and for a control group not affected by the policy change, used the ratings of around 31,000 books from the website LibraryThing.com. 

Their results showed that products were actually rated higher after the policy change – the average product held a 4.46-star rating versus 4.37 before the policy change – and there was a decrease in standard deviation across products. This means different products were rated more similarly to each other. This makes it increasingly difficult for shoppers to distinguish ‘good’ from ‘bad’ products.

Further analyses also confirmed that higher ratings for a product were associated with more sales. Specifically, there was a significant increase in sales from 3.9- to 4-star rating shifts, suggesting this as the threshold for differentiating between ‘good’ and ‘bad’ products.

In short, the study suggests that making it easier to rate products does not improve the informativeness of ratings, and no evidence was found that the proportion of authentic ratings increased after Amazon’s policy change, contradictory to what industry experts had hoped. In fact, making rating easier can cause the average rating to rise, something industry experts had hoped to reverse.

Discussing the impact of their findings for shoppers, Professor Mahlendorf said, “Customers heavily rely on the five-star rating system in online marketplaces, especially those where choice is extremely high, and there are little differences in prices. When approaching birthdays or during the lead up to Christmas, customers rely on these systems even more, to distinguish between ‘good’ and ‘bad’ products. The current ratings system on sites like Amazon needs to be improved. Simply making rating easier is obviously not the solution.”

Given the results of this study, Professor Mahlendorf suggests that requiring written reviews alongside star ratings could improve the informativeness of rating consensus of other platforms where paid incentives for ratings also occur. Ensuring the reliability of customer satisfaction information is not only important to protect consumers, but also investors who may use ratings to price assets.

After all, we already know that authenticity is the way forward. Tactics such as paying for reviews or enlisting social media influencers to endorse products can often backfire.

Therefore, companies with an online presence should consider whether it is worth implementing both star ratings and written reviews for their products. Incorporating both may just provide their consumers with a better shopping experience.