Crowdsourcing’s Critical Connundrum

You can read more about the phenomenon here, but the basic gist is that there are people out there who have deliberately given things low-score reviews, even when they actually loved the product, app, etc., that they’re reviewing (and they’d say as much in the body text of their reviews).

Why? They wanted you to read their review. They want their reviews to be rated as “most helpful”, and there’s a higher chance of you looking at a lone “1” when most people award an app/product a “5”.  Their reputation score mattered more to them than the reviews they were writing.

Individual goals trump group goals more often than not.

I remember an attempt at Microsoft Consulting Services several years ago to build a website of best practices, technical advice, etc. The plan at the time was to have consultants rewarded for the number of posts they made. What happened? A few of my then-cohorts tried to game the system and submitted article-after-article of near-useless, obvious, already-covered-in-product-documentation stuff.

Same phenomenon: the individual goal of meeting metrics trumped the group goal of building a body of reusable knowledge.

There are many cases of perverse incentives. The most extreme one I’ve heard of is the Cobra Effect. A less dangerous one involved paying developers for every bug they fixed (after a while, they’d ignore code quality just so they could find/fix bugs more easily).

What can be done?  I can think of three options:

  1. Make the game more elaborate by adding more rules. Think through the possible perverse incentives. More rules can make participation less attractive, and if so, the reward had better be worth it.
  2. Make the payoff so low that the incentive to misbehave just isn’t present. Of course, that may result in lower levels of participation.
  3. Alternatively, don’t crowdsource.  

I’m not a curmudgeon when it comes to social computing, in SharePoint, in Yammer, or anywhere else, but I do believe in caution.  Too many people hype “social” to death.  “Social” is useful, but it’s not magic.  If you treat it as such, you might find that sometimes it’s actually black magic.


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: