Recommendations from a Fraudster — Part 6: The Last Recommendation…for Now

Being Right vs. Getting It Right

By: Rich Ratcliff, Chief Trust Officer

A few months ago, I was walking a Whitehat team member through how we evaluate respondent risk. I was confident. We had signals. We had calibration. I was, in my head, being right. He let me finish, then said, “Did you know that cheap proxies rotate or drop without warning?” I had spent years on respondent risk and never built a simple end-of-survey IP comparison. One question. One sentence. One realignment of my perspective, that quick.

That was one of more than 50 conversations like it. Each one stopped me. The ones that made it into this series were the subset an MR Agency could act on in the same week. The rest, the majority, became updates inside our platform. Different audience, same lesson, which is the recommendation this series has been building toward.

I used to want to be right about fraud. Now I just want to get it right. Those sound like the same thing. They are not.

The “Be Right” Trap

Single-phase fraud prevention tools claim to have the right filter. They catch the bad devices, the bad IPs, or the bad open-ends. Often, they do. But no single tool is right, because fraud does not arrive in a single phase. If a tool stops a bad device, a good device with bad intention takes its place. Every confident, single-source claim of “right” tells fraudsters which thread to pull next.

Stacking more tools inside the same phase is not the answer either. The instinct is honest; one tool is not enough. The execution is contradictory: you do not trust any single partner to be right, while trusting all of them to elegantly fit together. None of them engineered their software for that. You end up with overlapping signals, conflicting risk scores, and a pipeline no one fully owns.

The existence of an organized fraud ecosystem, with boards, marketplaces, and answer keys, is market evidence that no single tool, and no stack of tools, is the source of truth. Fraudsters show up wherever the industry is over-confident. What gets caught in this posture is the amateur. The professional stays silent in the data, completes the survey, and gets paid.

The “Get It Right” Posture

Getting it right looks different from being right. It is harder.

For an MR Agency, getting it right is not about designing or owning a tool that “sees” fraud. It is about running a data quality system across every phase of the project. The objective is not “did we catch fraud.” The MR Agency’s job is: “did we deliver clean insights to the end client.”

Inside that system, calibration is the actual work. This is where never-zero lives. We treat zero detected fraud as a sign we are not looking hard enough, not as a win. The bar moves. The fraudsters move. The system has to move with them.

These recommendations are what getting it right looks like in motion:

• Part 1 flagged that fast and instant payouts are the number one exploit. The fix is a payout window.
• Part 2 named tracker links that sit on fraud boards as “annuities.” The fix takes 15 minutes per wave.
• Part 3 showed a respondent whose IP, ISP, and country all flipped between screener and complete. The fix is one comparison.
• Part 4 covered path cloning, partial speeding, and the comprehension check at the end. The fix is three small edits.
• Part 5 named the room market research never built, the one with windows into respondent behavior. The fix is to plug into someone who already has the windows.

None of these required a new platform. They required someone to stop, listen to what a fraudster was actually doing, and put some additional logic in the right place. Worth saying out loud: any of these recommendations can go latent. Today’s clean fix is tomorrow’s exploit. The list above is a snapshot, not a finish line or even close to it.  They are tips, tricks, and easy shifts with the hopeful strategic ‘systems’ approach in mind.

When AI Enters the Room, the Paradigm Will Change

Every recommendation in this series came from human fraudsters using human-speed exploits. They open tabs. They share links. They consult a cheat sheet. The industry catches up, eventually.

AI does not move at human speed.

The example I would put in front of any researcher is AI-generated open-ends. Today’s professional fraudster is not pasting “It was a good survey” into the comprehension check. They are generating a topical, grammatical, on-tone open-end response and either typing it themselves or using AI with human entry cadence.  As mentioned above, The Part 4 check is already starting to expire.

Agentic survey-takers and synthetic personas are right behind. The “be right” posture cannot keep pace. The signal set tuned against a fraudster on a laptop is not the set you need against an agent that can read the page, plan, and respond. Getting it right in an AI environment means assuming the next Whitehat conversation will, again, stop us mid-sentence. And then doing the work.

Closing the Series

Looking back at these five posts, the recommendations themselves are almost laughably small. Change a link. Compare two IP addresses. Rotate a screener. Ask the respondent what the survey was about. Embed a behavioral monitoring partner.

That is the punch. These are not hard fixes. They are missed fixes.

They are missed because survey fraud prevention has quietly become a cybersecurity discipline, and cybersecurity is not where most MR Agencies are staffed. That is not a weakness; it is a reason to utilize a data quality system fit for purpose.

That ownership is real work. If the majority of an Agency’s QA happens after the survey closes, professional fraudsters are already on their way to the client. Forensics matter, but they run after the room has been broken into. Getting it right looks like healthy blocks pre-survey, healthy monitoring in-survey, and healthy objective forensics post-survey combined with subjective subject matter expertise by the researcher. With that system in place, an MR Agency is not absorbing fraud and hoping post-fielding QA catches it. They are catching it on their terms, with confidence, and handing the client data and insights they can stand behind.

So the last recommendation, for now, is this: stop trying to be right about fraud, and partner with people whose only job is to get it right with you, one Whitehat conversation at a time.

That has always been the work. With AI looking to enter the room, it is the work that matters most.

Thanks for reading the series. If any of this caused you to pause, that was the point. If you want to talk through where your current setup may be drifting, you know where to find me at rratcliff@opinionroute.com.

Click here to start the of Recommendations from a Fraudster series from the beginning.