How to beat survey fraud in 7 steps

As a fast and efficient way to gather a large amount of consumer data — it’s not a huge surprise that 85% of businesses say that surveys are their most-used research method. 

But no matter how good the approach, it’s all about the quality of your data. 

As we say in our post on data quality in advertising, your data must be accurate, reliable and trustworthy for you to be able to use it in your reporting, analysis, and decision making. 

If your data is incomplete, inconsistent or incorrect then you won’t have the information you need to make the best decisions for your brand. 

Low-quality data can lead to misinterpretations, misdirections, and even drag down your reputation and lead to legal action if your data goes public or is used to purposefully influence consumer behavior.

Using good survey practices, like creating short, engaging surveys and offering compensation, is the first step to ensuring data quality. 

Beyond these best practices, it’s also essential to detect fraudulent survey participants and bots that can seriously undermine the quality of your data. 

The latest figures show that around one in five surveys may contain fraudulent data.

In this post, I’ll explore what survey fraud is, cover the main types of survey fraud, and show you how to combat it.

Survey fraud refers to when respondents or bots purposefully provide incorrect or misleading information in surveys, whether they’re in-person, or completed through online panels. It’s typically done to access a reward or financial incentive — with fraudulent participants often taking a survey multiple times in order to receive several payouts. 

89% of companies say they now use online surveys, — making online panel fraud one of the main types of fraud to look out for. This form of fraud relies on fraud specifically committed through online survey panels, in which respondents (often called panelists) are pre-recruited to take part in online surveys. 

Read on for a breakdown of the main forms of survey fraud.

From bots to inattentive survey respondents looking to rush through questions, let’s take a look at the main types of survey fraud. 

Participant fraud

Here's a look at the most common types of participant fraud:

Fake respondents 

In the case of fake respondents, people may misrepresent themselves to qualify for surveys in order to get the survey reward or financial incentive. For example, respondents may say they belong to a particular demographic, age group, or sexual orientation when this is not the case. 

Duplicate entries 

Another form of participant fraud is duplicate entries. In this case, a respondent will take the same survey multiple times. Just as with participants faking their identities, participants typically engage in the same surveys over and and over to access additional incentives or rewards. 

Inattentive respondents

Inattentive respondents may rush through survey answers — providing inaccurate or incomplete answers. In the case of multiple choice surveys, respondents may randomly select answers without reading the question or engaging in the survey material. 

Research has found that the majority of inattentive responses aren’t malicious, — with repetitive questions, lengthy surveys, and uninteresting topics being the main reasons researchers have uncovered for inattentive responses. In comparison, fraudulent inattentive responses are ones that survey participants go out of their way to provide. 

Bot responses 

According to Researchers Kathryn Irish and Jessica Saba:

“Scientific knowledge is only as good as the data, and at present, it seems highly likely that data collected online is impacted by the presence of bots.”

Fraudsters can also use bots to complete several surveys in a short space of time. This can leave you with a large amount of identical or close-to-identical responses that can seriously undermine the validity of your data if you don’t catch them.

A bot is a type of software program built to mimic human behavior — in this case, providing human-appearing answers to surveys. In some fields, bots have been found to account for up to 60% of survey responses. 

Unlike human fraudsters, survey bots can often be easy to spot because of their ineligible answers. Here’s Sanja Trajcheva at CHEQ:

“Imagine you’re conducting a survey to understand people’s preferences for pizza toppings. You spend hours crafting the perfect questions, publish your survey, and eagerly await the results. But as the responses roll in, something feels wrong. 

You start to notice a bizarre pattern: an inexplicable love for anchovies, an unusual liking for pineapple, and an astonishing passion for… sardines?

If you’ve ever encountered a similar situation, it’s likely due to automated responses generated by survey bots.”

Professional respondents 

Another fraudulent category to watch out for are professional respondents. 

These are people who take multiple surveys in order to access financial rewards. This can bias your results or make your data unreliable due to disingenuous answers. 

According to the research, narrow populations are more prone to fraud than broad populations. That means: if you’re working with a niche audience or market then you’ll need to put more systems and practices in place to help prevent and spot professional fraudsters. 

Data fabrication by researchers

Eve*, Senior expatriate researcher shares:

“I am always surprised that people would take that chance [of fabricating data], and yet they [fieldworkers] keep doing it! Maybe you should talk to some of those people and find out what the motive could have been. (Laughter) Because it's true…we always say…why would you take the chance to lose a job, when you know it's so hard to get a job? But it happens. Why go through this process of being one of…hundreds of people to get this job, knowing what the [employment/economic] situation out there is like, knowing the sort of stand that we take about these things – to then falsify the data?”

Fraud and fabrication don’t only influence the quality of your data on the participants’ side. Researchers can also fabricate survey responses. Almost a quarter of research scientists admit to intentionally fabricating or falsifying data at some point in their career. 

While widely considered unethical in the research industry, research finds that many researchers are up against intense pressures to publish that can lead to them falsifying the data. While other factors may include lack of social control and lack of adequate policies on research misconduct. 

Survey padding

As an offshoot of data fabrication, research can also pad their data. This involves inflating the number of completed surveys by completing additional surveys. 

This is typically done to help make sure they have enough data to meet quotas or deadlines. And like all forms of fraud in research, can dramatically undermine the integrity of the results. 

Incentive manipulation

Paid incentives can offer a number of benefits with the most obvious ones being an increase in response and completion rates. Incentives can also lead to people lying to researchers to get access to rewards. With some studies showing that up to 20% of respondents will lie to get a reward. 

Let’s take a look at the main ways participants go about this. 

Fraudulent claims for rewards

Some people will go out of their way to give false information so they can qualify for a survey with a specific incentive.

In particular, monetary rewards increase how likely it is someone will lie to hit survey criteria. But interestingly, higher incentive amounts didn’t equal higher rates of lying. 

Gaming screeners

In this instance, prospective respondents will lie during pre-survey screening questions in order to hit the eligibility criteria. Researcher June Wang and their team notes: 

“Fraudulent respondents may attempt to outwit preliminary eligibility screeners by repeatedly testing combinations of responses to identify the combination that meets inclusion criteria and thereby grants them access to the survey (known in cybersecurity as a “brute force” approach).” 

Proxy participation

Proxy respondents are often used in research — allowing people who know someone in the population being studied to take a survey on their behalf. However, even in above-board cases, proxy responses can undermine the accuracy of your data. 

Take behavioral observations. The research shows that it’s easier for people to observe covert than overt behaviors. Proxy participants can also lead to survey bias as participants can hold back from accurately reporting on behavior that’s seen as socially unacceptable.

But how do fraudsters manipulate this normally above-board approach to survey responses? 

By getting someone else to do their survey for them without telling the researchers. This form of proxy participation can throw off the accuracy and reliability of your data for many of the same reasons listed above. And without knowing it, you can’t account for their influence.

Survey design manipulation

The way you design a survey can have a huge impact on your data. 

Even when you’re not going out of your way to influence respondents’ questions, the questions you ask and when and how you ask them can dramatically impact the way participants answer. 

Take a look at this study by Dectech — which surveyed people on how likely they were to jump to a new energy supplier in the next 12 months:

Dectech says: 

“The graphic shows the average ratings of the subsequent questions across the surveys. There are two main findings. First, asking people what they liked about the last time they shopped around, rather than what they disliked, has a dramatic impact on reported future ease. Between the two frames there’s a 0.42 swing in the rating, which is both statistically significant and a huge shift for a seven point scale.

Second, the same effect isn’t observed for intention-to-switch. Perceived ease of switching is more malleable. Why? Perceptions are generally more speculative than behaviors and therefore more prone to manipulation.”

Let’s take a look at some of the ways researchers can intentionally manipulate survey design. 

Leading questions

Leading questions are biased questions written to influence participants to answer in a particular way — giving us biased answers. Think: “Why did you like the new packaging?” (leading) vs, “What did you think of the new packaging?”. 

The first question can make a respondent feel pressured to provide information on why they liked the new packaging — even if they didn’t. The second question gives them the space to answer authentically. 

UX writer Amy Schade talks talks about what leading questions can rob us of:

“Leading questions ultimately rob us of the opportunity to hear an insight we weren’t expecting from the user. The more leading our questions are, the less likely the user will comment in a way that surprises or intrigues us, or makes us think about a problem or solution in a different way.” 

Over-sampling certain groups

Oversampling is when researchers deliberately recruit a larger section of people from a particular demographic in order to influence survey results — undermining the integrity of the data. 

While oversampling is a tactic used to help researchers study minority populations by increasing their representation in a sample, oversampling becomes fraudulent when researchers purposefully select a larger sample size of a group of respondents to support a hypothesis or “enhance” the performance of a product or service. 

Organized fraud

While researchers once believed that survey fraud was relatively uncommon — they’ve noted a rise in fraud as a result of the increase in recruitment via social media and the recent rise in survey-taking groups or click farms. Let’s take a look at organized fraud in more detail. 

Panel farming

Panel farming is when individuals or groups of fraudesters collaborate — taking multiple surveys in a short space of time in order to access their financial rewards. Fraudsters may use bots to complete a high number of surveys quickly or they may fill them out manually under different identities. 

Social desirability bias

Online surveys can be a great tool for providing a sense of anonymity that allows participants to share more openly about sensitive or “taboo” topics. But on the down side, this anonymity can also lead participants to misrepresent themselves in order to appear more socially acceptable. 

While not always intentional, participants may give dishonest answers to show themselves in a favorable light, particularly on sensitive or polarizing topics like drug use or politics. This is called social desirability bias. 

On the upside, social desirability bias is typically less of an issue with online surveys than other research methods, such as face-to-face interviews or focus groups where people can keep track of a researcher's reactions.

Let’s jump into some of the main ways we can help fight survey fraud. 

1. Use attention checks

How can you catch a respondent who’s zooming through questions to get their reward? With attention checks. Attention checks are any instruments used to help catch out inattentive respondents. 

An example of a good attention check is asking a question that has an obvious answer, such as: “Which product is the subject of this survey?”. These questions are called “bogus items,” and giving the wrong answer suggests a lack of attention.

Researchers recommend that you: 

• Use attention checks that have clear, obvious answers. 

• Add more than one attention check for each survey. 

• Use covert attention checks over long instructed manipulation checks. 

Note that failing an attention check doesn’t always mean a respondent wasn’t paying attention. They can be aware of how you want them to respond and choose to do otherwise. 

2. Use CAPTCHA tests to block bots

A CAPTCHA test stands for, "Completely Automated Public Turing test to tell Computers and Humans Apart." It's an online test used to determine whether an online user is a human or a bot. You’ve likely had to do dozens of CAPTCHA tests in the past, like when trying to log into a website account. 

Here are the main types of CAPTCHAs:

• Text: these CAPTCHAs use text to catch out bots by sharing distorted words or a combination of characters and asking the user to share the sequence. 

• Image identification: these tests show users several different images and ask them to choose all the images that include a particular object such as a bike or bus.

• Audio: these tests give users an audio clip to listen to which includes words or numbers they must repeat. 

• Puzzles: these tests share a basic puzzle, like comparing the difference between two pictures.

CAPTCHAs are great for catching bots because they struggle to interpret this kind of content.

3. Use KYC verification  

Another way to screen for fraud is to use KYC verification. Participants out to commit fraud are looking for easy access to financial compensation. You can use a KYC (Know Your Customer) verification to run a personal background check on survey participants by asking them to verify their identities with checks such as document verification (like passports or bills) or biometric verification (which identifies people through unique biological traits like their fingerprint). 

On the downside, KYC verification can deter people from signing up to your survey because it’s often time consuming to engage in. You can offer a better user experience and help prevent dropouts by setting up automatic verifications like document verification by integrating your tech stack with government ID databases.

4. Monitor response patterns 

Another way to screen for fraud is to monitor response patterns for fraudsters like speed, consistency,  and quality.

Here’s what to look out for: 

1. Incomprehensive answers: make a note of answers that don't make sense, contain several mistakes or have formatting issues.

2. Speed: make a note of any surveys that are completed noticeably quickly or respondents who jump between responding quickly or slowly.

3. Response inconsistency: look out for patterned responses (such as respondents who select multiple options B) or answers that contradict themselves.

4. Replicated answers: watch for responses that are identical or near-identical answers. 

5. Send a personalized link 

Another way to help protect your surveys from fraud is to send out personalized survey links to your pre-screened applicants. 

Unlike open-access surveys that anyone can easily sign up to online, personalized links provide an added layer of protection — helping to make sure your surveys go out to pre-vetted respondents. 

6. Use digital fingerprints 

A digital fingerprint is a collection of data about a digital device, app or file that acts as their unique signifier. 

Digital fingerprints are an essential type of anti-fraud technology you can use to both protect and deter fraud. 

You can use digital fingerprints to track different online users and monitor whether multiple surveys are completed from a single device to help pinpoint fraud. Beyond detecting fraud, you can also set up digital fingerprints to block fraud attempts.   

7. Add time limits 

Time limits can help put off fraudulent participants by giving them less time to research or think up a believable response to your questions. You can pre-test your survey length to help work out the average time it takes to complete your survey then apply time limits that fit into these averages (with some leeway) — automatically flagging or rejecting respondents that spend too long filling out questions. 

Successfully cutting down on survey fraud is essential for protecting the quality of your data. From the rise in bots to panel farming, fraudsters are increasingly switching up tactics and gaming survey screening criteria and tests with more sophisticated approaches. By combining a number of the above strategies, you can help block fraud attempts and protect the quality of your research data.