How Do You Calculate Feasibility?
Blog | September 3, 2025
By: Terence McCarron
Recently, a podcast interviewer (an MRX outsider) told me that the content coming from our industry screams “FEAR”. I found this interesting, but also concerning.
It does feel like there’s a trust bubble that burst lately, doesn’t it? Particularly relevant when we think about sample vendors and data quality. Many MRXers are thinking twice about everything they are told.
Questioning, doubting, constantly on guard.
Here’s what I mean. Ever have one of these thoughts lately?
- Are these really human respondents?
- Is my sample vendor lying to me?
Though not new, this mistrust, misalignment has amped up this year. I’m on a mission to calm the waters and re-establish relationships and connections in the process.
An Overlooked Symptom
Through my years of investment and experimentation in solving MRX problems, one process is overlooked, but is at the heart of the Trust dynamic. MRXers go through a daily ritual that I feel has lost its purpose.
Have we lost our way on Feasibility? Let’s walk through a familiar process.
- MRA gets an RFP from client that’s ideal for Online Quant.
- Send Specs to a selection of sample partners to assess budget.
- As you work on the other pieces, within 1-48 hours, vendors reply with a “feasibility” report and a CPI.
- MRA picks ideal partner from the bids, finalizes and sends proposal.
It all starts with a promise made. Ever wonder how this gets calculated?
For years, I never thought twice about it. Now, I’m constantly obsessing over it. I genuinely have no clue how some vendors come up with their numbers. Many make no sense at all.
In 2005, the process was mathematical and explainable.
Target profile panel counts X Average Response Rate X Apply Incidence Rate = Feasibility. This worked really well. This only faltered as panels started inflating their panel counts in marketing. Hard to argue you had 10million panelists and a 15% response rate, but can’t get a 20% IR census quota job done.
Then, in programmatic 2015, it became:
Run click counts for past 4 weeks for the demo profile. Apply a single study allocation assumption. Apply IR. Then you have feasibility. Not great, but at least it’s a formula.
Today it feels like this:
If the client needs 200, say we can do 200 as fast as possible. NEVER be the one to say no. At LEAST promise “best efforts!”
The modern-day approach has produced ridiculous promises made to our team. Here are a few examples. All of these were promises/commitments from sample companies.
- N=1200 #1 top HR Exec for a 20 min survey, 30% IR among target in company with 20K+ employees. (Universe of US companies that size is around 500.)
- N=1000 ADHD diagnosed teens, untreated. 45 mins survey.
- Panel Size of 3,000,000 US-based IT DMs, able to deliver 30,000 completes on any single project (of course with enough field time). (Universe of IT DMs in US is ~450K in the US).
On one bid, I drew a line with a client in a way I rarely do. I begged him not to bid on a project that was an impossible ask. He challenged me right back: “What will happen if I sell it and just use someone other than you?“ Fair question. Here’s what I predicted:
There will be 1 of 2 possible outcomes.
- You soft launch, and the sample vendor quickly realizes the bid was a terrible mistake.
Outcome for my client: Difficult spot for him with his end-client, probably relationship damaging. - You soft launch, the vendor realizes the jam they created, then quietly “adjusts” and gets it done without a word.
Outcome for my client: He would be delivering fraudulent data. A close corollary to this is when a job is stalled out for 5 days and then, after applying pressure, the vendor magically gets the final 100 completes in 2 hours.
Internally at OpinionRoute, we talk about having a “gut check” on every bid before we run numbers. We believe that having a contextual understanding of all respondent types gives us a great start in avoiding project disasters. We do a ton of desk research, we calculate “gettable” percentages in the broader sample market, and then we apply study specifics.
Start from a place of context, and your fear of the unknown will drain quickly. You’ll have a fresh lens to spot which vendors are the pretenders.
It’s just another way we trust, but verify.
Want to share war stories on ridiculous vendor bids? Let’s chat it through as we build a new way together to Work Better!