Surveys are one of the most widely used types of primary data collection within market research. Market researchers and users seeking reputable insights about their consumers trust this well-established type of research to make smart business decisions.
Despite seeming like a straightforward methodology, market research surveys can be complicated, and users can sometimes experience confusion or suspicion when examining survey results. This can occur when actual results contradict expectations, reveal missing demographics or disagree with other surveys covering the same topic.
How can survey data be made more consistent and useful for understanding consumers?
Common challenges with survey data
There are two main sources of miscommunication that can occur with market research surveys. First, a survey may not be the right solution for the question being asked. So, we must carefully consider whether it can properly answer our research question. Second, there can be a fundamental misunderstanding or lack of visibility in sample design and creation. It’s important to know the sample design approach when interpreting and contextualising your survey data.
Why is sample design so important?
While effective questionnaire design and using the right methodology for the right question is important, there needs to be more focus on better understanding the sample design. There are many ways to create a survey sample, but depending on the approach chosen, there might be vastly different results.
Probability sampling
A probability sample is when the true population has an equal chance of being selected to be part of a study. Probability sampling is the “gold standard” of survey methodologies and the predominant sampling approach for academic research.
Non-probability sampling
Non-probability sample is when you do not have the guarantee that everyone in the population has an equal chance of being surveyed. This might be because there is no way of finding all the people in the population that you want to observe.
Within non-probability sampling, there are also several sub-types to consider. Convenience samples are samples that are selected because they are accessible to the researcher. A purposive sample is one where samples are screened for certain criteria and selected based on what is thought to be more fitting to be part of the study. Snowball sampling is typically used when the population is so small that researchers use one respondent to find others that are similar. Finally, quota sampling is a technique used when the researcher puts quotas around certain criteria, like demographics that are well-known to the population, to make the sample representative of that population.
In these types of non-probability sample types, not everyone has an equal chance of being part of the study. For that reason, results from non-probability samples, if unacknowledged and uncontrolled for, could be biased toward a subset of the population. However, non-probability samples and studies are pervasive within market research. Their accessibility and agility make them an ideal method for researchers and business decision-makers who do not have the luxury of time and resources. As such, this methodology is not going anywhere anytime soon, but this does not mean we cannot re-think its approach and use.
Our white paper ‘How to Make Consumer Research Work for you: Rethinking Best Practices’ will reveal common misconceptions, discuss different survey sampling techniques and recommend best practices on how to leverage and get the most out of your consumer surveys.