For years, researchers have used attention check questions to catch people who aren’t paying attention during surveys. These questions often include instructions like, “Select ‘Strongly Agree’ to show you’re paying attention.” While once effective, these checks have become easy to spot—and easy to beat.
Today, online survey fraud is more sophisticated. Bad data doesn’t just come from distracted participants—it comes from bots and professional fraudsters who know how to appear attentive without actually providing thoughtful or truthful responses.
What’s Wrong with the Old Way?
Traditional attention checks are now so common that they no longer serve their purpose. Many people have learned to recognize them—and even share “how-to” guides online. Fraudsters can often pass these questions without reading anything carefully. Automated tools and AI can also be trained to spot and correctly answer them.
What does that mean for your data? It means you could still be getting poor-quality responses from people (or bots) who figured out how to cheat the system.
A Better Approach: Catching Acquiescence Bias
At CloudResearch, we’ve developed a new kind of fraud check that’s much harder to game. Our questions are designed to catch a behavior called acquiescence bias—the tendency to say “Yes” to everything.
This “yea-saying” habit is common among people trying to qualify for as many surveys as possible. They assume agreeing will keep them from being screened out. We use this to our advantage by asking about things that are either highly unlikely or completely made up. For example:
- Visiting a location that doesn’t exist
- Using fictional products or services
- Claiming to have done something extremely rare or impossible
Attentive participants are unlikely to say “Yes” to these. But fraudsters—especially those trying to fly through a survey—often will.
These questions look natural and blend in with other items in a survey, making them harder to spot as checks. They can also be updated or localized to fit different countries and cultural contexts, which helps avoid overuse and ensures continued effectiveness.
Best Practices: Use Multiple Checks and Allow Leeway
We recommend including several fraud check questions spread throughout a survey—not all grouped in one spot. To avoid mistakenly flagging honest people, it’s best practice to allow some room for error. For example, when implementing 5 of the recommended questions below, you might flag a participant only if they fail two or more of these unlikely or fake items. This helps balance fraud detection with fairness and keeps you from mistakenly discarding good responses due to a simple misunderstanding or accidental click.
Examples below:
Have you recently visited any of these local dart shops to purchase a dart board?
- Darting Bliss
- Tully’s Bully Shop
- Spike Shot Nation
- WinningCorner
- None of the above
Have you seen any live music shows in Shea Stadium, NY in the last 2 years?
Have you recently purchased food from any of these small food chain stores?
- Toasty Cravings
- Sal’s Harvest Delights
- Savory Sizzling BBQ
- Spicey Nicey Eats
- None of the above
Within the last 3 months, have you discovered a lost manuscript written by a famous author in your attic?
Have you hired any professionals from the following freelance platforms or service providers in the last year?
- Freelancealot
- Servestry
- Findery
- Giglance
- None of the above
Sentry: Built on Better Fraud Detection
Our Sentry system uses this new approach to help researchers stop fraud before it enters the survey. It screens participants using behavioral analysis and advanced technical checks, catching bots and bad actors early—without requiring extra work from researchers.
Sentry is utilized in some capacity across all CloudResearch products. However, if you’re fielding participants from other sources or platforms, Sentry is also available as a standalone tool to help improve data quality—no matter where your sample comes from. Contact us to learn more about how Sentry can support your research.
Bottom line: Traditional attention checks don’t cut it anymore. To protect your data, you need fraud checks that are smarter, subtler, and designed for how survey fraud actually works today. That’s the standard we follow at CloudResearch—and it’s one we believe every researcher should adopt.
