We sat down with Marc for a Q&A session to discuss the topic of data quality and his advice for businesses that are looking to conduct sound consumer research in a market saturated by online consumer panels and other traditional methods.
Q: What’s the current state of consumer research?
A: I think consumer research is one of those fields that’s taking some time to evolve. We’ve seen great technological advances that have significantly impacted the way consumers behave nowadays, and consumer research needs to be agile to catch up with those behaviors.
With the advent of social media and the mobile phone, researchers gained unprecedented access to consumers everywhere, on a massive scale. That’s a goldmine of real-world data that calls for new ways to access and harvest it. New methodologies on how to survey consumers who are now almost constantly on social media are emerging, Potloc being one of the pioneers to do so. In the end, what should matter for businesses is the quality of the data they obtain from their studies.
Q: Could you talk a bit more about data quality? What are the factors that businesses should be looking for when conducting their research?
A: Data quality, and the research methodology used to collect the data, is the most important thing businesses should be looking for when conducting their research. As it is the data and the insights extracted from the right respondents that will drive business decisions and ultimately impact their results. With bad data comes bad insights. Bad insights lead to bad business decisions, and that leads to bad results.
My main concern today as a researcher is that data quality is often overlooked as many businesses try to get a sample of respondents at the most affordable price, and –unfortunately, many data providers are willing or even forced to play this game.
While price is important and will never be taken out of the equation, my advice to businesses would be that they should be more curious and pay more attention to the research methodology used to collect respondents.
Businesses should be asking their providers that use traditional methods (phone surveys, online consumer panels, intercept interviews, etc.), how respondents were recruited to be part of the panel (randomly by phone vs. self-selection); if and how much respondents are incentivized to answer the survey; what are the processes put in place internally by the data provider to guarantee that the data quality is high (how does the panel provider identify phenomenons such as ‘speeders’, ‘straight-liners’, ‘duplicates’, inconsistencies between answers, etc.); and also what is the expertise of the data provider in terms of questionnaire design regarding the client’s industry.
I will personally always lean towards a smaller sample of respondents made of ‘better’ respondents than the opposite. What is the purpose of having a low (theoretical) margin of error if the data collected is poor?
Q: We’ve all heard many horror stories about online consumer panels: From professionalized respondents to how they struggle to target niche populations. Could you paint a picture -based on your professional experience, on what’s the quality of the output data coming from panels?
A: I would say that there are huge discrepancies in terms of the quality of output data coming from panels. Some of them are extremely careful with how they build their "community", have thorough processes to guarantee that data quality is high, and employ questionnaire design experts that are verticalized by industries. Others don’t. Just like in any other industry, there are good and bad apples.
However, there are in my opinion 2 practices that most of the panels out there share and that unfortunately prevent them from achieving the highest data quality they could get:
Q: How do you ensure high data quality in your everyday work here at Potloc?
A: Related to what we have just discussed, here are the concrete things we do today at Potloc to ensure high data quality:
Thanks to these actions, our processes put in place internally to check data quality are always enforced, usually discarding very few respondents from the sample collected. Ever since I joined Potloc, I’ve always been impressed –and surprised, with the depth and richness of the qualitative comments we gather when asking optional open-ended questions. There is always a WOW moment with our clients that other methods seem to overlook.
Q: What are the 3 top pieces of advice you can give to businesses looking to conduct meaningful consumer research?
A: Always ask the right questions and be curious about the way your research will be conducted:
Q: Where do you think we are heading in terms of methodologies to conduct consumer research? Online panels have been around for a while, and before them, we had phone and intercept interviews. What’s the next big thing?
A: Social Media, definitely. Just to put in perspective: some of the largest consumer panels out there have up to 30M panelists. There are 3.8 billion social media users around the world and all projections see that number climbing steadily. So, naturally, consumer research has to evolve to tap into that audience in a way that allows it to collect data coming from real people, buying and using products in real-world situations.
Today, at Potloc, we only see social media as a middleman, helping us to get in touch with our target populations (We push our surveys through ad platforms such as Facebook Ads Manager. Then once people click on our ads, they are redirected to a survey platform where they complete the questionnaire). But there are so many other ways to conduct consumer research on social media such as using listening tools to run sentiment analysis on users’ comments and much more.
Q: Talking about social media, how would you compare what we do here at Potloc with what online panels do?
A: The approach is quite different. On top of keeping our surveys to a 6-7 minutes limit and not giving incentives to respondents upon completion, there are 2 other things that come to mind: