Introducing Sentry's Verisoul Integration and What It Means for Data Quality

At CloudResearch, we recently partnered with Verisoul to integrate their device-level fraud detection directly into Sentry. But before we did so, we ran a study to understand what the combination of Sentry and Verisoul produces. This blog presents a summary of what we found and explains what it means for your data.

The Problem

The problem with data quality keeps getting more sophisticated

Anyone running online research knows that data quality is a challenge. Inattentive respondents, bots, people using VPNs to mask their identities, and low-effort responses have been part of the landscape for years. Most sample providers have their own protections in place, but no single provider can cover every angle and the data clearly show it.

In the study I’ll describe below, every respondent we examined had already cleared the sample provider’s pre-survey screening, including an established fraud-prevention system. They looked like legitimate participants. They passed the filters that were supposed to catch bad actors. And yet, when we looked more closely, a substantial portion of them were producing data that didn’t reflect reality.

As an example, when we asked what we call benchmark validation questions—questions designed to see if participants provide answers in the ballpark of expected reality—respondents said they owned Tesla’s at seven times the actual U.S. rate. More than 12% said they were petroleum engineers, which is unlikely given that it’s an occupation that makes up 0.007% of the American workforce. These kinds of distortions are a sign that bad actors have learned how to look legitimate until they get into the study. They are also a sign that the tools panels have designed to catch these bad actors haven’t kept up.

The Approach

What we tested and how

To examine the performance of Sentry plus Verisoul, we gathered 1,044 completed survey responses. Every respondent was vetted with Sentry and Verisoul — both separately and together — to understand what each tool catches and what the combination produces.

The two tools focus on catching bad actors in different ways.

The two tools catch different kinds of problems, which was the rationale for combining them.

The Results

What we found

The results were consistent across every dimension we measured. Both systems caught some bad actors. But they worked best together.

When we looked at attention checks — the simple questions researchers use to verify a respondent is actually reading — we saw 76.5% of participants passed in the unfiltered sample. With Sentry alone, that rose to 94.2%. With Verisoul alone, it was 82.3%. But with both systems together, the number was: 96.3%. That’s a nearly 20-point improvement over the unfiltered baseline.

Open-ended response quality showed an even larger gap. In the unfiltered sample, 54.8% of open-ended responses met basic quality standards. Sentry alone brought that to 79.0%. The combined system reached 82.1% — a 27-point improvement from where we started.

Survey completion time also improved. Extreme speeders, which we defined as respondents who rush through a survey so fast they couldn’t possibly be reading the questions, were effectively eliminated in the combined-pass group. Of the 48 extreme speeders in the full sample, 45 were flagged by at least one system.

Sentry and Verisoul together flagged 84% of low-quality respondents. And, as a reminder, these were respondents who had already cleared the sample provider’s built-in defenses.

Population Benchmarks

The benchmark test: does the clean data reflect reality?

The most meaningful test isn’t how many respondents get flagged but whether the data that remains is valid and trustworthy. We tracked four claims with known U.S. population benchmarks: the percentage of people claiming to own a Tesla, adhere to a vegetarian diet, to have gone scuba diving in the past year, and to work as a petroleum engineer.

In the unfiltered data, people reported these behaviors at rates far above the population baseline. Yet, every benchmark came back into alignment with actual population rates once both systems were active. Without them, that same panel would have produced a picture of the American public that looked nothing like the American public.

What Verisoul Adds

The threats you can’t see from survey behavior alone

One finding in our study stood out to me. Among the respondents Verisoul flagged, 11.5% were physically located in countries including Syria, Bangladesh, Nigeria, and Colombia. And this was despite passing the panel’s own location verification. More striking: 3.6% of the fraudulent participants caught by Verisoul had also passed Sentry’s behavioral evaluation.

These weren’t careless actors making obvious mistakes. They were sophisticated enough to behave like legitimate respondents throughout the survey. The only way to catch them was through device-level forensics, which is the kind of persistent identity tracking that Verisoul was built to do.

This is the core argument for a combined approach to screening: each system catches something the other misses. Behavioral analysis can tell you a lot about how someone is engaging with your survey. It can’t tell you whether the device they’re on is real, whether their location is genuine, or whether they’ve taken your survey seventeen times from seventeen different accounts. Verisoul can.

The question isn’t whether fraud exists in your data. It’s whether your tools are sophisticated enough to find it.

The Bottom Line

What this means for you

The data from this study shows what that combination produces: cleaner attention check results, better open-ended responses, fewer extreme speeders, and benchmark rates that actually reflect the population you’re trying to study.

Online research only works if the people in your sample are who they say they are, located where they say they’re located, and genuinely answering your questions. That bar is harder to clear than it used to be. The Sentry + Verisoul combination is how we’re meeting it.

Want to see what your data is hiding?