A Simple Guide to Choosing the Right Sample Size for Your Survey

A well-conducted survey is one of the greatest tools that a marketing department can employ. Surveys point out the need for a product or service, demonstrate the shortcomings of the competition, bolster the expertise of the company, or simply increase visibility among potential stakeholders—all in an empirically credible way. The catch, however, is that surveys aren’t credible by default; each one must be carefully planned and executed to ensure that its findings are applicable to the real world.

Such diligence is necessary for the whole surveying process, from ideation to analysis, but there is one question that stands out above all the rest when journalists and stakeholders judge the quality of a survey: how many people were surveyed? Too small a sample, and the results are unusable. Too large a sample, and the company has wasted valuable resources. So how can one tell too small from too large? Here is a simple guide to answering that very complex question.

What Is a Representative Sample?

To begin with, a more basic question needs answering: why do surveyors sample in the first place? Say that a company wants to determine the opinions of Texas schoolteachers—that is, the population—on technology in the classroom. Since there are around 350,000 schoolteachers in Texas, it is practically impossible to survey the whole population. The company will instead want to survey a more manageable number of teachers—that is, a sample.

The problem is that a survey of any 15 Texas schoolteachers will not necessarily represent the opinions of the 350,000 taken together. In fact, there is a very low chance that it will accurately capture the diversity of the whole. So before results can be generalized to the entire population, the sample must first reflect the characteristics of that population—that is, it must be representative. A representative sample is generally randomly selected and of a minimum sufficient size.

How Small Is Too Small?

The smallest sufficient sample size depends on three factors: the size of the population, the desired margin of error, and the desired confidence level. The latter two terms may sound technical, but the ideas are quite simple. Margin of error is the leeway allowed for results from the sample to vary from the true characteristics of the population. Confidence level refers to the likelihood that the results will actually fall within that margin of error.

Consider again the example of the schoolteachers. If, in fact, 65% of Texas schoolteachers approve of digital standardized testing, and the sample has a 5% margin of error with a 95% confidence level, one would expect the findings to fall between 60% and 70% around 95% of the time. It is best practice to have a 5% margin of error at maximum and a 95% confidence level at minimum.

Once the margin of error and confidence level have been decided, these can be combined with the size of the population in a mathematical formula to determine the minimum sample size. The same can also be done in an online calculator, fortunately, such as this one from Qualtrics. Using the same figures as before, the calculator yields a sample size of ​​384 randomly selected Texas schoolteachers. It is worth noting that this is only a minimum; a larger sample will increase the accuracy, and it is favorable among journalists and readers to sample at least 500 people.

Advanced Tip: Quota Sampling and Post-Stratification

Companies are likely to encounter the terms quota sampling and post-stratification when working with professional survey vendors. In order to understand these concepts, it is first necessary to understand the limitations of random sampling; by nature, the demographic makeup of a truly random sample is likely to vary from that of the population. Around 30% of the sample may consist of male teachers, when in reality only 24% of Texas schoolteachers are male. Quota sampling and post-stratification are ways of counteracting such discrepancy.

Quota sampling does this by including a certain number of subjects from every subgroup, proportional to the size of each subgroup within the population. By setting a quota for female schoolteachers according to what percentage of texas teachers are known to be female, the surveyor minimizes the likelihood that either males or females will be over- or underrepresented.

If the survey researcher did not use quotas when designing the survey, post-stratification can achieve a similar effect after the survey has been conducted. This is done by identifying and reversing biases caused by overrepresented demographics. An analysis team would compare the demographic makeup of the sampled schoolteachers with that of the real-world population before weighting responses from certain regions to minimize any imbalance.

Bottom Line

Conducting a worthwhile survey requires precision and care. If a company fails to meet the basic ground rules for selecting a representative sample, all of the money, time, and effort poured into the execution of a survey can go utterly to waste. Those who undergo the small measures of diligence here discussed, meanwhile, will both steer clear of statistical pitfalls and ensure that they get the most credible numbers they possibly can for their investment.

Enjoy this blog on selecting a sample size? Be sure to check out our other blogs on useful and interesting public relations topics, like how to design a newsworthy survey.

While we may employ generative artificial intelligence for research purposes, all content published by Razor Sharp Public Relations is written directly by our team.