Getting reliable insights from crowds

This post is about more insights derived from The Wisdom of Crowds by James Surowiecki.   His opening anecdote was about a country fair where there was a prize for guessing the butchered weight of meat from a live ox.  The average of the guesses from the public was not only just about spot on; it was also more accurate than that of any of the cattle experts.

This is exactly what happens when we collect insights about what are the Power Attributes that influence customers choice of products within a market.  In over 20 different studies where we have done the assessment in workshops with customers and management, the collective view seems much more accurate than that of any one person.
We often look at which attributes are important to people when they are considering the purchase of a product.  Each manager or customer then produces their own ranking.  When we analyse the ranked scores for each attribute, the group result seems to make a lot of sense.  Not only that, but individuals within the group are often happier with the group score than their own individual assessment.
So what is the implication of this for gathering insights and making decisions?
Well structured quantitative assessments of large groups of customers will always provide the most robust insight where this is practical and valid.  But often this is not affordable or just impractical. 
Our experience would suggest that gathering opinions from a smaller group is still helpful.  This can be done in a workshop meeting where you can focus on the questions and generate discussion and consensus, or it can be done via simple online or paper surveys which takes up less time and is often quicker.
The important thing is to remember the criteria for getting reliable insight.  In his book, Surowiecki identified four criteria necessary for a crowd to have a useful insight.  His criteria are in bold and our interpretation is after the hyphen.
  • You must have diversity of opinion – pick your group to represent the market group 
  • People need to be thinking and acting independently – always collect individual votes and responses 
  • There needs to be decentralisation with people relying on their own local knowledge rather than an aggregated average experience – make sure people report on what they think and not what they think someone else thinks. 
  • There needs to be a way to aggregate all the private opinions of the participants into a collective decision – do some good analysis on the results, count the responses, collect scores and analyse them properly.
You can generate insights quickly and economically using a number of self help techniques if you stick to the important principles.
We often advocate an internal survey as a prelude to a workshop.  This always adds considerable value as we are able to work with The Wisdom of the Crowd and not just opinion of the boss.   We have also used customer workshops as a cost effective alternative to high cost and slower quantitative research.  But we apply these principles every time.