Digitize or die: Thriving in the future of market research

We know that customer-centricity is essential for the success of any brand operating in a competitive marketplace. Companies that understand their consumers’ needs, desires and preferences are best placed to develop the products and services that people value and to promote them in the most compelling way. The potential value of consumer research has never been greater.  

However, too much of the market research conducted today reflects outdated practices. It is expensive, done too late and often fails to provide actionable findings. If this continues, research will only deliver a fraction of its potential value and risks becoming left behind in the digital age. 

Embracing digital technologies such as automation and artificial intelligence, on the other hand, will transform the industry and allow insights teams to take their rightful place at the heart of business decision-making.

There are four reasons why old-school research is becoming less relevant.

1. It’s too expensive.

2. It’s done too late.

3. It’s too unclear.

4. It’s too disposable.

Traditional market research projects are cumbersome, which makes them expensive. They involve multiple teams, each with its own specialism, and repeated handovers. The process is complex and largely manual. 

This archaic process also makes errors more likely. The industry has introduced numerous quality control procedures to minimize these errors, but this has only pushed costs higher.

The price of research means that only a small proportion of marketing concepts and content ever benefit from consumer feedback. Marketing and insights teams are forced to be highly selective in how they spend the research budget. Inevitably, this can lead to decisions about what should and shouldn’t be researched being motivated more by political needs than business priorities.

Everyone is so resource constrained. Everyone is getting squeezed. You really have to do things differently. You cannot do the same thing that you used to do 5-10 years ago. 

Joseph Chen, former Insights Lead at Mondelēz

When the cost of research projects is reduced, more marketing outputs can be optimized, and more learning can be gained about what works, what doesn’t and why. Insights teams can have a big impact on business success and gain the recognition they deserve.

Thankfully, automated, digital research solutions take away most of the manual steps, reducing costs significantly. You no longer need to spend tens of thousands of dollars for a single project — in fact, for that money you could explore 5-10 different ideas. Artificial intelligence will help to make automated, low-cost solutions even more accessible in future. The platforms are already user-friendly, but language models will make it easier for all users to get the most from the technology, regardless of their technical research expertise.

Market research projects are often slow, taking weeks from start to finish, because of the many manual steps required. Or, they simply come too late in the process — which means that consumer insights can’t have such an impact on decisions related to new products or advertising. As a result, research often performs the role of grading marketers’ work, rather than helping them enhance it. The marketing team, necessarily, has moved on to its next job before the insight is available. This is a waste. It means that the voice of the consumer is not reflected in the majority of marketing output.

Up to 80% of insights and analytics work is wasted in some organizations. And part of the reason could be that it was not connected to a decision to begin with.

The insights person ends up walking with his deck behind a marketing or salesperson trying to get their attention, trying to inspire them, trying to prompt some action. Meanwhile, they are solving for something else and they say, "This was great work, fantastic, thank you." And off it goes into the drawer, never sees the light of day.

Oksana Sobol, Insights Lead at The Clorox Company

Research companies often suggest that the development process should be extended, to create enough time for the research. But is this the right way to solve the problem? Not when digital alternatives exist.

Marketers need to work at pace to keep up with the competition and leverage topical cultural trends. Slowing down the process would be counterproductive. If you decide to delay your decision-making to make time for input from traditional market research, you run the risk of being left behind by competitors that are exploiting newer, faster approaches.

Research can add much more value if it is fast enough to be integrated in the development process, without slowing it down. It can give warning signs if an early-stage idea has a fatal flaw. It can identify what it is about a promising concept that consumers find most compelling. It can identify which ways of bringing a concept to life will be most powerful before significant amounts of time and money have been committed. When it comes to consumer feedback, time really is money.

We used to collaborate with a pen. It was about validation, correcting mistakes and evolving from there. Now we collaborate with a pencil because we're co-creating, shaping things, we're steering, and that is the true power of insights is that we shape. We don't validate.

Stefania Gvillo, SVP, Chief Analytics and Insights Officer at Domino's

Digital technologies drastically reduce the time it takes to obtain consumer feedback. Automated platforms allow research projects to be completed in a matter of days, or even hours. In fact, most of the time can be spent thinking about the business needs and implications, rather than the research details.

The market research industry has been slow to acknowledge the need to provide clear conclusions and advice for marketers. Research reports are often overly detailed, jargon-heavy and difficult to understand. Research dashboards continue to be hard to navigate, use poor data visualizations and provide little in the way of interpretation and actionable guidance.

Marketers deserve and need more. Research findings need to be easy to understand and the implications should be highlighted clearly. Black box approaches with highly technical outputs should be a thing of the past. 

The research just didn't land. The segments did not land at all. I think it was too market researchy. It didn't end up speaking the language that we speak in the right way. It wasn't easy enough for them to act on and it just didn't work.

Lauren Governale, Head of Consumer Insights & UX Research at Hims & Hers

Part of making the analysis clear for marketers is giving them relevant comparisons to put the data in context. Unfortunately, many traditional approaches offer outdated benchmarks that are not relevant to the decision that’s being made today. It doesn’t make sense to compare your ad against a historic category norm if your direct competitors’ current advertising is way stronger. You aren’t competing with what distant competitors did several years ago.

The last thing you want is your recent and relevant norms to become obsolete.

Christine Avallone, Brand and Communication Insights Lead at Verizon

And finally, findings must be valid. Inconclusive results are of no use to marketers because they need to make decisions based on reality. To be of practical value, research methods need to predict in-market outcomes to a high degree of accuracy so that marketers can make decisions with confidence. Otherwise, they might as well ignore the research and trust their gut instinct instead.

The right research platforms allow teams to focus on the business relevance, validity and actionability of their work. Teams can spend more time defining the business question precisely, allowing the technology to configure the most relevant, robust solution. And since AI is highly capable at analyzing results and developing possible implications, insights teams can focus on putting the findings into business context, humanizing them, and ensuring stakeholders take them on board. Influencing skills may soon become more important than technical skills.

Finally, market research projects are usually treated as one-off projects. They are single-use products, quickly disposed of after the debrief. This makes no sense in a world where AI already leverages the entire web to consolidate thinking and provide advice. 

When companies conduct a research project, the data usually belongs to the research agency. Most of the thinking is also done by the research agency. The report is meant to capture as much of the learning as possible, but how often is it referred to again? It sits in an intranet archive somewhere, collecting digital dust. 

Imagine if the insights from every research project could be leveraged to guide future decisions? The technology capable of delivering this already exists. Let’s use it!

Today, every time somebody in PepsiCo does a test, the data feeds the data in the database, the whole of PepsiCo as a result gets smarter. And not just smarter in a vanilla type of way, but smarter in our specific categories and subcategories for our specific brands. And that today is a competitive advantage.

Stephan Gans, SVP, Chief Insights and Analytics Officer at PepsiCo

Putting all your results into the same data storage ecosystem is the first step. You can use AI to analyze the data, add useful tags to it to characterize your ads/products and look for learning from the patterns within the data. Machines can do this better and thousands of times faster than human analysts. Meta-analysis used to be an expensive luxury. Today it is an easily obtained by-product of doing multiple research products – provided you have a thoughtful approach to data management.

Being able to measure the effectiveness of your creative if you do it at a single project level. That gives you one level of insight. The real value in raising the creative bar is when you begin to get to that meta insight, when you connect the dots, when you see a body of work and you can really ladder up and see what is the lightning in the bottle.

Jane Wakely, Global CMO at PepsiCo

The future of insights is digital. Your marketing teams are used to getting actionable digital data from other sources, fast, and they will expect the same from market research. Continued adoption of AI will only make this more true over time.

As a leader of an insights team, investing in the right research solutions and systems is an important step to ensure your success in the future.

To recap, here are the main points to remember:  

• Market research of the past has been too expensive, too late, too unclear and too disposable for modern marketing teams to act on. 

• Unfortunately, many of these traditional market research approaches that have not kept up with the times are still used today. And even new approaches aren’t always being used in a new way to support agile working.

• If your team is still leveraging these outdated approaches, or using new approaches in an old way, it’s time for a change. Success in the future depends on adopting digital today and setting teams and the business up to benefit from the change. 

Dan White is a marketing expert and illustrator with 33 years’ experience in brands, advertising, and market research. He has worked with many of the world’s biggest brands and advised on over 500 advertising campaigns. His summaries and illustrations have influenced hundreds of thousands of marketing and insights professionals through his social media activities and three best-selling books. In this piece he draws on his experience of the challenges faced by market research of the past in delivering what insights teams and marketers need. And he looks at how much more impact the industry could have today if it embraces recent advances in technology.