If data were dating: meta-analysis in market research

You’ve been on a date and it went well. You talked, you connected and you learned a lot about your date. So much so that you feel you are ready to make a big decision. Based on the fact that they have a great job and like the same movies you do, you’ve asked them to marry you.

Of course no one would do that (would they?), but this is like taking a project based approach to market research, rather than a meta-analytical approach.

We get a project brief, conduct research, then make a big decision – run with the ad, go to the next stage of product development – based on the results of that project only. How much more effective would your decision making be if you could draw on a curated body of insight, based on multiple instances of primary research, which connects the dots to other data sources, and puts it in context of everything else you know about consumers?

You’ve been on a date and, again, it’s gone well. You talked, connected, learned etc. But then you ghost them. It turns out that, for you, this was a one-time thing. Not the start of a conversation, not an opportunity to learn more, or find out how they might fit in to your life; instead, you decide to never contact them again.

This is like conducting a great project that answers a research brief, but then never using that insight again. The insight sits in a PowerPoint deck, deep in an online filing system, until such time that the person who ran the project leaves the company, at which point it is forgotten forever.

So you’ve been on a date… And this time it’s going so well that you’ve been on a few dates with the same person. You must be making a great impression. But, no. You’re still asking them the same questions you asked on the first date.

What do you do? What is your favorite movie? What music do you like? Yawn. You already know this stuff, why are you asking again? Didn’t you listen? Didn’t you learn? You might be asking in a slightly different way, but it’s still the same old questions.

Many insights teams end up paying to ask questions they already know the answers to, albeit with a slightly different sample, in a slightly different market or about a slightly different product. How much more value could you get for your research dollar, if you could find a way to stop doing this?

So how do we break out of short-term, one-off dating disaster and find long-term data happiness? The answer is to embrace meta-analysis. We define this as any analysis that combines findings from multiple projects to create an insight of greater value.

So instead of using research to get answers to questions such as “do current users of my product like this ad?” or “which element of this concept is most appealing?” you can build up a longitudinal body of evidence that begins to answer questions such as:

• Are my target audiences finding lavender-scented cleaning products appealing?

• Are we getting better at creating strong TV ads over time?

• Do consumers in the US react differently to testimonial-style advertising than UK consumers?

• What promotional offers appeal to Millennials in my category?

• What creative elements across my TV and Digital video ads are likely to drive disruption?

• Which concept elements are table stakes for my sub-category?

• What package elements tend to resonate across FMCG products among Moms with young children in the household?

Although no one would question the value of an approach that maximizes the return on market research data investment, and enables brands to make better decisions, the reality is that meta-analysis isn’t easy to do. There are often barriers that work against a Consumer Insights professional’s ability to add value by synthesizing the learnings from disparate market research data projects.

• Reactivity: Much of the work of consumer insights teams is in reacting to project briefs, designed to answer one-off questions relating to particular concepts, products or issues. Insights professionals are not always at the decision-making table and aren’t always fully aware of the business questions that meta-analysis could answer.

• Time: The data required to answer the meta-level question is typically not centrally located, and can be hard to find. Once found, the manual process of pulling data, analyzing for insights, and charting the results is time-consuming.

• Budget: Because of the amount of manual labor required, it’s not always feasible to conduct meta-analysis in-house, so Insights teams often have to outsource the work to suppliers or consultants. This gets expensive, fast. Inconsistent data: The wide variety of research methods, audiences, survey measures, norms and data sources that organizations use, combined with black box calculations and databases, makes it hard to make quick, accurate cross-project comparisons. It also creates gaps in knowledge.

• Confidence in the results: When the data is inconsistent, the analysis can only ever be subjective and approximate, rather than scientific. This makes it hard for Insights professionals to sell in the analysis to internal customers, because it’s not rooted in quantitative rigor.

For insights leaders to get to a position where their teams are optimizing their market research data learnings through meta-analysis requires three things: structure, consistency, and tools.

Insights professionals need a way to organize and categorize research findings and insights so that they are quick and easy to find. Sometimes this relates to how the organization is structured, by product, category or region and, within this, the location of the Insights team(s).

Ideally, you should be able to conduct analysis across projects, products and platforms.

• Cross-project analysis: Looking across projects within a single product. Answering questions such as “What’s the impact of celebrity presence on TV ad enjoyment?”

• Cross-product analysis: Looking across products for combined insights. Answering questions such as “How well integrated are all of the ads in my campaign?”

• Cross-platform analysis: Integrating 3rd-party data with research data. Answering questions such as “Is this new flavor variety about to be the next new thing?”

To get to a structure that will support this type of meta-analysis, we recommend conducting an Insights audit, looking at how the Insights team is structured, in two key areas: internally and in relation to other functions.

1. Internal structure: Look at the tools you are using, the range of roles and skills of the people within the team, and the processes you have in place.

2. Relationships: Look at how insights relates to end users such as Marketing, Commercial functions and senior management, and to other collaborators, such as R&D, Innovation, IT, agencies, and consultants.

In each, ask yourself what works, what is broken or missing and where are the opportunities to change so as to enable better insights.

In order to compare data across multiple products, projects, categories and geographies, you need to standardize the way you are doing market research.

For example, if you always use the same solution to test your ads, you can compare results over time, across campaigns and agencies, and throughout your product portfolio. Within this, you will also need to standardize the way you define your categories, audience, question wording, ratings scales, and answer options. Then you can start to build up norms to help you compare performance, and you can create standardized tags to identify and cross-analyze your results.

It takes some work to get the standards in place – but that isn’t the end of the story. Once you know what the standards are, you need to socialize them throughout your organization.

You can do this in part by using the technology in platform-based market research data tools to set guardrails. For example, if you always test ads using product A then you can use the tech to disable product B for ad testing, you can enable specific audiences and set up default questions.

But there is only so much you can do with control. The other side of the coin is that you need to win the heart and minds of your team, your internal customers and your agencies. This requires some effort around change management and selling in the benefits of meta-analysis – and is the critical step without which you will be stuck with the existing project-based, single-use approach.

Many of the features of solutions on Zappi’s platform will enable you to get meta-value from your market research data. For example:

• Theme Analyzer: If you have tagged all of your research results by theme, you can search across projects, products or campaigns in seconds. For example, if you have tagged all of your products that feature a lavender scent, you can pull up answers to questions such as “Do lavender products resonate better than non-lavender scented products?”

• Autocharts: A cross-tab engine that pulls together your curated data to create any type of charts you specify, in seconds. You can answer questions according to any way you have structured your data. So for example, if your data is structured so it can be grouped by year, you can answer questions such as “Are last year’s ads better or worse than this year’s ads?” and have a bar chart to visualize

• Stories: A way of collating results as you conduct analysis using crosstabs or other features, to build up a new report that captures all your insights around a particular theme. Using Stories can take you from the mentality of project-specific results to taking an overarching view.

• User-defined benchmarks: Because you can define a wide variety of norms, for example, by category, market, brand or tag, internal, or external, you can set your own benchmarks based on your business goals. As your database of consistently structured Insights increases, it gives you increasing options for accurate comparison.

Meta-analysis is more than just an approach to market research data, organizational structure and data management. It is a mindset.

Just like dating, if you’re only in it for the short term you’ll be unlikely to succeed. Instead, focus on building a long-term relationship with your data, and getting the very best from every question, tool and data point. If you can do this, you will be able to bring your team, internal customers, C-suite and agency partners with you on the meta-analysis journey. But please, don’t make the mistake of bringing them all with you if you really are going on a date.