The polling business has had some cause for self-examination following their failure to predict that the Tories would get 37% and Labour would get as little as 30% of the votes in the recent general election. And there has been plenty of debate about this. See this article by Ben Page of Ipsos Mori. What I wanted to do in this blog post is to draw some practical observations about what business leaders and marketers can learn from this, where we should use market research and what to be wary of.
But before looking at the implications, what went wrong?
There has been a lot of debate and discussion. I particularly like the assessment made by Dave Penn click here (@davidpenn1) and Keith O’Brien click here (@keithobrien) and discussed by Mark Earls (@herdmeister) and Phil Barden (@philbarden). They argue that pollsters have failed to absorb and react to the implications of Daniel Kahneman’s finding that decision making is governed by two systems in the brain.
- System 2 is the rational conscious analytical, slow, considered part. This is what is measured by traditional market research and polls.
- System 1 is the subconscious, intuitive, emotional, gut instinct part that is fast and essential to our ability to get by in life. This is not easily measured by market research polling.
Since polling concentrates on system 2 rational thinking and behaviour, it can misread what people are actually feeling and fails to predict what they will actually do.
Political commentators seem to miss this point about system 1, when they debate the order of questions, the methodological differences between phone and online polls, the sampling error. Politicians don’t get it with comments like Vince Cable complaining that people misled him on the doorstep and a sense among the losers that voters were confused or misled or afraid and this explains why they were so foolish to not vote for them.
Lord Ascroft has commented that Lynton Crosby’s team seem to be doing something different and they knew they would win. Even Lord Ashcroft is curious about how they did that?
Can we rely on market research? Should we use it?
Inevitably the measured answer is “it depends”. I have long noticed that there are only really three business questions that market research is used to help with. These questions are
- What should we do?
- Are we doing it?
- What will the customer do?
I would argue that market research is helpful and reliable for the first two and quite hazardous and misleading for the third one. The industry promotes products and services that help top answer all three questions. I would suggest that the industry has not come up with a satisfactory approach to answer the third question. the reason they sell a lot of product to answer that question is that business leaders have an insatiable appetite for the answer to the question “What will consumers do?” Executives like to manage risk and market research is used to justify decisions.
What the pollsters error teaches us is that research is not yet a great indicator to understand what people will do. Some like Dave Penn are using neuroscience and behavioural economics to pioneer new techniques to try to overcome this problem.
What can we use market research for?
Lets look at the three business questions that market research is used for.
1. What should we do?
Whilst consumers find it tricky to accurately predict what they will do and they are not good at imagining products they have no experience of, they can tell us a lot about their lives, their problems, their needs, their frustrations. Also we can observe their actual behaviours.
As business leaders we know that we have to offer customers things that help them address needs and solve problems e.g. feed the kids when they come home from school, get breakfast on the way to work, communicate with my friends. We can watch what people do, we can find out about frustrations and problems, we can measure what really matters to people. We can find this out through personal experience and observations or we can use formal market research. So
- Usage and Attitude studies
- Observations and interviews in store
- Panel surveys and interviews
- Consumer workshops and group discussions
This is called exploratory research and if it is well designed it informs our plans. Do not expect consumers and customers to imagine possibilities that we have not shown them. That is our job to create and imagine that.
2. Are we doing it?
Business leaders are suckers for measurement. we want to know if things worked, we want to know about the customers experience, were they satisfied, will they recommend us. This is another area that professional market research is brilliant at. So
- Brand tracking
- Customer satisfaction
- Net promoter scores
These all help us see how we are performing. Provided we did the work properly in the “What should we do?” part and we are measuring things that matter to our customers, then this is really valuable.
3. What will the customer do?
This is where it all goes wrong. It is what business leaders most want to know. The question rings out, “have you researched that?” and executives feel they have to go and that. But much of this work holds out false promise. the worst possible question is
“Will you purchase this product in the future?”
The reason for this is simple. As people we are very bad at predicting what we will do in the future. This is mainly because we do not know what our priorities will be in the future. It is hard to see how market research can help us overcome that. But it is also because of the influence of the unconscious system 1 on our decision making.
My own approach to concentrate all the effort and work on questions 1 and 2. if we get that right and do that well, then the products and services we offer to customers will be attractive to them and enough people will buy them. Asking them if they will buy them does not give us a better indication. Lord Ashcroft understands this and proclaims his polls to be only a snapshot. However even he must wondering about the quality of his snapshots in 2015.
Read this article about Attractive Thinking to see how to answer questions 1 and 2 and avoid the need to use research to answer question 3. This is about coming up with a plan for growth that you are confident will attract more customers.