Monday, September 06, 2010

How to use Amazon reviews

I wrote a negative Amazon review of Apple's battery charger (2/6 batteries were defective). As expected "0 of 2 people found the following review helpful".

This is very common with certain items, such as Apple products, Microsoft products, Christian conservative books, and other products that have "fans". It also happens with lawn mowers and dehumidifiers [1], but in those cases the negative feedback comes from manufacturer employees and retailers.

The "helpful" metric on Amazon reviews is not only worthless, it's harmful. It points people away from important reviews. It's also used to create reviewer rankings, so those are also worse than worthless. (By using these metrics Amazon is setting itself up for emergent fraud.)

There's another weakness of Amazon reviews -- name changes. Just as Google's Ballmer Schmidt tells teens they'll need to rename themselves as adults, so to do vendors change model numbers to dodge bad reputations.

There are workarounds for both problems. Here's how to use Amazon reviews:
  • Always read the negative reviews, even on a 4.5 star product. The two star reviews are usually the best, some of the 1 star reviews are nonsensical.
  • Remember statistics, a 50 review product will usually have meaningful negative reviews.
  • Look at other models by the vendor to defeat name change strategies. Amazon keeps older model information around for a while, so you can usually find the previous model number. Vendors don't change their behaviors as quickly as they change their model numbers.
  • When looking across a product category, sort the category by sales, not by average rating. The rating averages are not discriminating and are unreliable.
  • Give more weight to True Name (authenticated) reviewers. If a review seems unusual, look at other reviews by the same person.
- footnotes

[1] Based on my experiences with appliance purchases over the past few years, I think Sears or even Best Buy are better options than Amazon -- because it is practical to use the warranty.

No comments: