Avoiding False Positives: Survey Data Protection is a Balance


If you’re like me, you’re sick of hearing about the “New Normal” that COVID has inflicted upon us. Some examples are very obvious, while others are not so much. Respondents in survey research are living through subtle changes that impact survey data in concerning ways. In short, changing consumer behaviors have exposed real flaws in outdated or simple data protection tools. As a result, too many good respondents (aka “false positives”) are blocked while actual fraudsters are missed.

Living in data collection every day allows us to see noteworthy trends in data protection and quality. Below are a few key observations that will help build a more robust online survey strategy.


1. While certain fraud approaches are growing in prevalence, others are declining as efficacy wanes.

2. Newer, complex fraud approaches have been discovered right here in the USA.    

3. “False positives” are increasingly prevalent as common detection approaches do not understand new consumer behavior in this new normal.  

While all trends are notable. Point 3 is the focus for this piece.


With our Data Quality as a Service product, CleanID, OpinionRoute has participated in scores of studies designed by clients to measure the effectiveness of various fraud prevention tools. Through these experiences, we’ve noticed that the pace of change in normal consumer behavior has accelerated so quickly during COVID that “new normal” everyday events are being incorrectly flagged as malicious behavior by older or less sophisticated systems. Let me share a few examples:


VPN usage

Work-from-Home has exploded during the pandemic in white-collar professions. A summer Stanford study estimated that as many as 42% of the labor force worked from home full-time. Companies have adapted to this reality while raising their technical security game for remote devices. Virtual Private Networks (VPNs) are commonly used to protect company data on devices logging in via household ISPs like Comcast and AT&T. Since fraudsters often use virtual networks to mask their actual device location and markers, this is a common misread for fraud flagging. Missing this nuance means your survey could systemically exclude 42% of the labor force. It’s happening right now, and for some researchers, it’s drastically skewing the insights. Fortunately, CleanID understands and accommodates this nuance.

Mobile Usage

When using a mobile phone, your IP address often routes through a cellular network that identifies location differently than your actual geography. For example, as I write this, my cell phone is in Cleveland, OH, but my IP points to Knoxville, TN. CleanID understands this. However, other tools don’t, which introduces a massive bias potential.  

Insufficient IP Blacklist Usage

IP Addresses often get renewed or refreshed in time, and it’s similar to how pay-as-you-go cell phone numbers often jump back into rotation. Any effective fraud prevention system must account for this by ensuring any blacklist check it uses is comprehensive, current, and global. This represents an uncomfortably high level of bias if mismanaged.  

In our collective zeal to protect data and ensure only the best data quality, researchers must constantly investigate the relevancy of all approaches and tools.  The balance is to stay ahead of emerging threats while avoiding the systemic exclusion of large population segments. Let’s keep out the cheaters, but let’s also make sure real consumers can have their voices heard in our surveys. You can have both!

Insights & News

News and Perspectives for the constantly evolving market researcher.

About OpinionRoute

Learn more about the team committed to redefining survey insights delivery.

Want to hear more?

Let us show you how to take your quantitative research to a new level.

About OpinionRoute

We deliver accurate data by utilizing our expertise in online survey sampling and proprietary technology solutions to simplify research processes. This enables clients to scale and researchers to stay ahead in a dynamic and competitive market.

Contact Info

hello@opinionroute.com 216-282-0793

Headquarters: Cleveland, OH

© 2024 OpinionRoute, LLC. | All Rights Reserved.