21 March, 2024

Fraudulent and Fake Survey Responses: Shedding Light on the Darker Side of Market Research

According to a report by Fast Company, between 15-30% of all market research data collected is fraudulent. Rampant survey fraud perpetrated by bots and survey farms has resulted in a significant drop in the quality and accuracy of survey data, and with the rise of Generative AI able to imitate respondents with uncanny accuracy, the issue is only set to get more problematic. This presents a significant threat not only to the credibility of the market research industry but to the success of businesses that rely on accurate market research data to guide important strategic and investment decisions.

The market research industry has skyrocketed in the last decade, with American companies alone spending over $62 billion a year on qualitative and quantitative research. With such large investments on the line, the market research industry must find effective ways to combat fraudulent responses.

The question is: how?

In this article, we take a closer look at the problem of fraudulent responses in market research surveys, as well as how the market research industry is attempting to address the prevalent issue of survey fraud.

How false data impacts your survey results

Fraudulent responses cause a cascade of critical issues in the market research industry.

At the core of the issue is bad data. Fraudulent results generate inaccurate data, which compromises the validity of survey results. This has a knock-on effect all the way down the value chain.

If a dataset is infiltrated by responses generated by bots, survey farms, AI-based fraud, or other false response methods, results are unlikely to reflect reality, i.e., the true aggregated opinions, preferences, and thinking of the target market. Without a foundation of accurate data upon which to predicate market research insights, it’s impossible to accurately guide strategic decision-making, which directly undermines the core mission of insights teams and their agency partners.

In short, the ROI of a market research investment depends heavily on the validity of the survey’s respondents, and if this cannot be guaranteed, the results and recommendations have no value to decision-making.

Ways to prevent market research fraud

The rise of prevalent market research fraud is going to require a cohesive, industry-wide response. But for businesses looking to conduct market research right now, it’s vital that you choose a research partner committed to addressing the issue of survey fraud, with robust measures in place to mitigate its impact.

Here are some important things to consider when choosing a market research provider:

Choose partners that use strict validation practices

It’s important that your research partner employs strict validation practices to ensure that respondents are genuine and meet the specified criteria for your survey. This includes measures like validating ZIP codes, phone numbers, and email addresses, and real-time survey monitoring.

Choose partners that implement quality control measures

Good data quality is mission-critical for the accuracy of your market research results. Choose a partner that implements robust quality control measures, such as regular audits and checks to ensure that the collected datasets are accurate, complete, and uncompromised by fraudulent respondents.

Vigilant monitoring of open-ended survey comment sections

One of the most effective ways to detect fraudulent respondents is to include and monitor open-ended comment sections. Bots, survey farm workers, and AIs will often include irrelevant or repetitive feedback in these sections or copy and paste text from unrelated sources irrelevant to the survey in question.

Including in-survey “checks” – open-ended questions designed to invoke a particular kind of response from a human respondent – is also an effective way to detect fraudulent respondents in real time.

Predicting the Growth of Plant-Based Proteins

Learn how HUUNU predicted key consumer trends in the plant-based protein market.

Read Case Study

Use a tool that already addresses these issues

By virtue of its unique methodological design, CRG’s prediction market research platform, HUUNU, circumvents the data quality issues plaguing the market research industry.

Our methodology begins with a survey screener, including many of the quality controls noted above, after which qualified participants progress to a prediction market game. Between the survey screener and the prediction market game, we can eliminate the “respondents” most likely to be fraudulent in their intentions. With most survey panels or random samples, this is 20-30% of those that would otherwise qualify for the project. The net effect is that HUUNU is self-policing fraudulent respondents out of our clients’ predictive data better than other survey and online qualitative tools. How does it work? Fraudulent respondents drop out of HUUNU at higher rates as they realize the prediction market game is much harder to cheat because the tasks require more human thinking and engaged behavior than traditional surveys.

As a result of these higher drop rates among respondents that are most likely fraudulent in intent, our clients can be confident that the results of a HUUNU prediction market are generated by genuine participants, responding to the research exercise authentically.

An analysis of HUUNU’s normative database and historical behavioral participant data confirms this hypothesis. The fraudulent respondent attrition rates, the percentage of questions answered, the frequency of changed answers, and the volume of tokens invested have all stayed very consistent over the last five years, as have the length and volume of the verbatim responses. This has been the case even as AI-based fraud, survey farms, and bot respondents have increased in the industry overall.

CRG-PI Fraudulent Response Blog Stats

How a HUUNU prediction market operates

As mentioned earlier, HUUNU inherently combats survey fraud with its proprietary gamified prediction user experience (UX). Participants make predictions and place bets on how they think targeted audiences will respond to a new concept, idea, product, or messaging being tested, and they’re only asked to respond to topics about which they are already knowledgeable. Because they’re being asked to respond based on how a targeted audience will behave, rational response bias is reduced, leading to richer, more contextual, more accurate answers.

What they choose to bet on gives us a view into their implicit reaction to a concept or idea, while how much they bet tells us how sure they are in their answer. As you can see, this would be a tough task for fraudsters looking for an easy payout.

If we have piqued your curiosity, below is a detailed description of how HUUNU helps advance the industry priorities in survey fraud and data quality.

Once a participant completes an initial survey screener, we invite qualified individuals into the prediction market. From there, they are asked to participate in a three-part exercise.

Step 1: Participants read or view the stimuli

If applicable, participants are asked to read or view the relevant stimuli. This could be a concept board, animatic, early-stage idea statement, packaging design, or anything else that needs to be tested.

Step 2: Review the question and place a bet

Next, participants are asked to review the highly contextual and predictively worded questions related to the stimuli, after which they must decide if they want to place a bet. If they choose to place a bet on a particular question, then they need to decide how many tokens they want to wager on their prediction. These are taken from their digital wallet.

Step 3: Defend their bet with rationale

Lastly, participants are asked to defend their bet with a qualitative rationale. Here, they need to explain why they think they’re right about their wager. The qualitative data collected during this step provides rich, critical insights into what the target audience is thinking while presenting a significant challenge to any fraudulent respondents who managed to make it through the initial survey screener.

Be confident in your data with CRG Predictive Intelligence

HUUNU’s three-phase prediction market methodology is more intellectually demanding than traditional market research survey frameworks, which makes it much more difficult for fraudulent respondents to submit fake responses undetected. As a result, our datasets are less affected by the data quality issues caused by fraudulent responses, delivering accurate, deep insights at both a quantitative and qualitative level.

To circumvent market research fraud, choose CRG Predictive Intelligence

Leverage the power of a fraud-resistant prediction market methodology to power high-impact strategic decision-making with CRG’s HUUNU platform today. Click here to find out more about our solution or request a demo.

FAQs

Should I be using CAPTCHAs or other security features to secure my market research surveys against fraudulent responses?

Market research providers may deploy security features like CAPTCHAS, biometric data capture, or other validation technologies to ensure that survey participants are genuine. This added layer of security can help mitigate the infiltration of fraudulent respondents.

What measures can I take to prevent fake or illegitimate responses in my survey?

Confirming that your panel partner is validating email addresses, phone numbers, and ZIP codes is a good place to start. However, the most effective method is monitoring open-ended comment sections, which are difficult to “game”. In-survey checks in the form of open-ended questions are a good way to weed out bots, AIs, or survey farm workers, whose responses to these sections are often irrelevant, copied from unrelated sources, or otherwise nonsensical.

How do I know if my research responses are legitimate?

Regular audits and data checks carried out by your provider are the best way to gauge and ensure the quality of the data generated by your primary research techniques. Ideally, real-time fraud monitoring which checks for speeding, irrelevant, out-of-context responses and duplication should be implemented to catch fraudulent respondents in action, before the dataset is finalized.