Data Science of Reviews: ReviewMeta tool Automatically Detects Unnatural Reviews on Amazon
ReviewMeta is a tool that analyzes millions of reviews and helps customers decide which ones to trust. As the dataset grows, so do the insights on unbiased reviews.
By Tommy Noonan, ReviewMeta.com.
As Amazon's total review count continues to climb towards the one billion mark, it leaves an enormous trail of data behind. This data can be used to help identify trends and predict which reviews are unnatural at a much greater scale than just "gut feeling" alone.
ReviewMeta.com has been collecting massive amounts of review data and is using it to sort out the fact from fiction. Consumers can use this tool to run an analysis on the reviews for any product on Amazon by simply copying and pasting the product URL into the search at ReviewMeta.com.
Not only is this tool helping check the reviews on individual products, the aggregate data collected is being used to further understand review trends and discover which types of reviews are more likely to be biased.
One of the relatively new trends many Amazon customers have been suspicious of is the reviews with a disclaimer that they "received this product for free or at a discount in exchange for an honest and unbiased review". Many customers insist that these "incentivized" reviews are exclusively 5-stars, and immediately discredit them.
Incentivized reviews typically originate from third-party "Review Clubs" that aren't affiliated with Amazon. These Review Clubs are websites that connect Amazon sellers who are offering free or discounted products with Amazon reviewers who promise to write an "unbiased" review in exchange for the freebie or discount. They claim to promote honest and unbiased reviews only, but the data suggests otherwise.
A recent study that leveraged ReviewMeta's dataset of over 7 million reviews shows that these "incentivized" reviews are, in fact, much more likely to be positive.
Below is the distribution of ratings for incentivized reviews vs. non-incentivized reviews. As you can see, incentivized reviewers are 12 times less likely to give a 1-star rating than non-incentivized reviews, and almost 4 times less likely to leave a critical review in general.
While this skewed distribution is pretty telling, one could still make the argument that the sellers who offer their product to the review clubs are going to make sure that they are only offering quality products, and this would explain why the incentivized reviews are more positive. However, exploring the data further reveals that reviewers who received a product for free or at a discount are more positive than reviewers who purchased that same product at full price.
The graph below shows the distribution of discrepancies between the incentivized average rating and the non-incentivized average rating for 251 individual products. 86% of these products received a higher average rating from their incentivized reviews than their non-incentivized reviews.
Here are some products that exemplify incentivized reviewers giving a much higher rating than the non-incentivized reviewers:
- Rxvoit Noise Isolating Earbuds (Incentivized avg: 4.5; non-incentivized avg: 2.9)
- SpaWorks Konjac Exfoliating Facial Cleansing Sponge (Incentivized avg: 4.9; non-incentivized avg: 3.8)
- iPhone 6s Screen Protector (Incentivized avg: 4.8; non-incentivized avg: 3.6)
Possibly the most alarming part about the data is exponential growth in popularity of Review Clubs and incentivized reviews. The following graph shows the review volume during the 48 months between May 2012 and April 2016:
2 years ago, incentivized reviews accounted for less than 2% of new reviews. Since February of this year, they make up the majority of all new reviews on Amazon.
This trend is partly being fueled by the rise of super reviewers; Amazon customers with thousands of incentivized reviews under their belt who are reviewing multiple products every single day. The data shows that reviewers who have never written an incentivized review have an average of 31 reviews each, while reviewers who have written at least one incentivized review have an average of 232 reviews each.
While Amazon has recently stepped up their efforts to eliminate fake reviews, incentivized reviews are still explicitly allowed per their Terms of Service, as long as the reviewer includes the disclosure in the text of their review. However, Amazon has been removing the "Verified Purchaser" badge from reviews who receive a big enough discount. Only 4% of incentivized reviews have the "Verified Purchaser" badge while nearly 75% of non-incentivized reviews have the "Verified Purchaser" badge.
As an Amazon customer, it may seem daunting to try and sort through every single review and pick out which were incentivized - fortunately this is one of the tests conducted when you copy & paste a product URL into ReviewMeta.com. As more data is collected, we're able to further leverage our dataset, sort through the reviews and leave you with the most honest customer feedback.
Bio: Tommy Noonan has been in the online review industry for over 10 years. After running SupplementReviews.com, he was able to identify the trends and signals associated with biased reviews and recently launched ReviewMeta.com, a tool that automatically detects unnatural reviews on Amazon.
Related:
