How to be sure enough: identifying the needed amount of interviews
Several factors impact the right sample size for your project: the amount of subsegments results should be split into, what margin of error you can accept, your budget, and your reporting needs. In this post we highlight what considerations to take.
Yes! Sample size matters. While there is no one size fits all, it is not (only) how big it is, it is also how you are going to use it: the sample size determines validity of your insights, not least when results are split into subsegments.
Thus, to determine what sample size you need, you must have a plan for the reporting - and the importance of the business decisions you want to make based on your research: sample size should be highly correlated with the impact your conclusions will have for your business.
For instance, when strategic decisions like brand position or customer segmentation needs to be undertaken, your sample size needs to be higher than when you are doing research for more tactical and operational decisions like package test or concept evaluation.
The basics on samples and uncertainty
A sample is a subset of a full population. Thus, it is not necessarily a representative sample of the entire population but it should always be representative of your target audience.
Uncertainty and significance
Whether a research outcome is a random error in data due to a limited sample size or significant due to a real change in consumers' opinion, behavior, perception, belief, or attitude makes a real difference for the value of your insights. When something is significantly different, it means you have eliminated the error margins as being of importance for the change, and your conclusions are more certain - as are your subsequent action options.
Neglecting this can have big consequences for the outcome of the business decisions made and, hence, the storytelling needs to stay true to the data and uncertainty.
Calculated margins of error at 95% confidence level with p=.5 for selected sample sizes
Determining the right sample size for your research project
The sample size you need correlates with a) the impact the conclusions you are about to make will have for your business, and b) the lowest level of analysis; the amount of subgroups you will divide the survey respondents into in your reporting.
Thus, it is a good idea to sketch out the research objective to identify the ideal sample size: is it top line penetration figures or getting an in-depth target group understanding? The art here is to find the sweet spot between depth of analysis, time, and budget.
While there is no one number to always go for, there are rules of thumb:
Less than 300 interviews is never a good idea
The sample size should double if your results are broken down into more than two subgroups. Thus, four sub-groups in your reporting should have at least 600 respon-dents in your sample, eight splits should have 1,200 respondents, and so forth
Last, but not least, you need to ensure at least 150 respondents for each subgroup if you know from the onset that you want to analyze re-sults as a split between two or more groups. This is ensured by your sampling strategy and screening questions.
Do you want more guidance on mobile research?
Check out our Guide to conducting good mobile research and get inspired on how to evolve your quantitative market research to a mobile world.