As a professional marketer, you want to know what your customers think and feel about your brand. Customer feedback energizes and motivates your thinking when designing your marketing programs. There are numerous formats at your disposal when asking questions. Just to name a few, there are:
- “Yes” or “no” questions
- Open-ended
- Rating (e.g., 1 = dissatisfied, 10 = very satisfied)
The more you know about customer psychographics and behavioral drivers, the more engaging your messaging and customer experience is likely to be.
The big question is this: “How many respondents do you need to connect with to receive meaningful feedback you can rely on?” The challenge of hitting on the right sample size can be tricky.
For example:
- Suppose a B2B supplier (let’s call him X) has one mega-size enterprise Fortune-500 client (i.e., constituting a market segment all on its own).
- X may only need to talk to two or three decision-makers inside the business to get the lay of the land and clues as to the next best steps.
On the other hand, when your market size is in the hundreds of thousands, two or three opinions are unlikely to represent anything close to the majority. And if they do, luck is supremely on your side. The bottom line is that calculating sample size is critical when conducting market research because it’s also the foundation of insightful surveys. They erase waste and guide strategy if the field of investigation is large enough. Reliable data is a company’s most valuable asset, bolstering investments in your product development, supply chain, pricing structures, and promotions.
What do we mean when we talk about sample size?
Sample size covers the completed response numbers your survey generates. The term “sample” implies that it’s not feasible to speak to every customer and prospect — just enough of them to represent the population. There’s a massive gulf between a representative sample and one that misses the targeted segment’s core motivations.
Market segmentation tells us that customers who share key characteristics generally behave the same way. That’s why a sample with enough respondents in the target population to pick up the commonalities (emotional and cognitive) works well. If your offer is innovative (not part of the customer experience yet), you probably want to know how the majority will react to it. Again, if the sample size is representative, it may conceivably provide the answers you’re looking for.
What’s the formula that a sample calculator works around?
Here it is — a complex combination of variables:
- N = Market size
- e = The margin of error (as a % expressed in decimal form)
- Z (score) = The confidence level you are looking for (generally ranging from 80% to 99%)
Here are the “gives” and “takes” when deciding on the sample size that works for you:
- The margin of error varies inversely with the sample size. When one goes up, the other travels down (and vice versa).
- Your sample size dictates the level of confidence you can expect. The more of the latter you want, the bigger the sample must be.
How to calculate the ideal sample size
Define your target market
The first thing you need to know is your target market’s size (i.e., N above). In B2C marketing, N envelopes possibly thousands of people, even hundreds of thousands. After analyzing the demographics, it may make sense to break the overall market into more defined segments and extract a sample for each (aligned with a smaller number in each case).
For example:
- Let’s say that Hispanic females age 18 to 35 is your bumper market (numbering in the millions)
- Dividing them down into cities like Miami, Philadelphia, and NYC creates reduced (but still significant) populations.
- If urban segmentation is still insufficient to provide an affordable sample size survey, apply more demographic constructs. Education, religion, and family size constructs deliver more definition and focus — as long as the numbers in each whittled-down population make ROI sense.
Define your confidence level
When you give your marketing team the sample results as a motivator for projects and activities, you want to express your confidence in them. The industry average confidence level should be around 95% for any given margin of error (i.e., deviation from the mean). The smaller the sample size, the wider the error margin is likely to be. As strange as it sounds, there are instances where companies accept substantial error margins (see below).
As a responsible marketer, you need to take the variables through a process of trial and error until you get down to a sample size that:
- You can afford to survey.
- Represents a market population that aligns with:
- Substantial buying power.
- A realistic market share for your company.
- Reflects a margin of error you can address with agility if it impacts performance.
- Gives enough confidence to put resources behind a full-blown marketing program to achieve demonstrated benefits.
Sample sizes in different situations
High-confidence, strongly representative samples with minimal error margin are most common when surveying employees:
- The total population is the entire staff complement — not millions of active buyers. Frequently, you may only be interested in people’s opinions in the marketing division, HR, or production.
- Whatever it is, small samples do the trick most of the time.
- SMB (with relatively small staffing) can get to grips with emotions and thoughts reliably by speaking to only a handful of employees.
Conversely, a company like Coca-Cola delving into all consumers’ motivations on the East Coast will likely require a sample size in the thousands. As a rule of thumb, a company should aim for as large a sample as you can afford to survey for any situation to get the essential accuracy.
Do statistically significant sample sizes matter?
There’s no easy answer to this question. In some instances, survey sampling can deliver relevant information even if the sample size calculator tells you the results are not statistically significant. When you get into deep emotions and interview a few typical customers, the opinions and attitudes may reveal a lot, giving your team different perspectives. On the other end of the spectrum, where YES or NO are the only possibilities connected to a critical question (like will you vote for me), statistics kicks in big-time. Here’s a look at the relevance of the sample size calculator in different survey situations:
HR and employee satisfaction pulse surveys
The more employees who respond to the survey, the better it gets. However, whatever input you derive from the feedback is valuable. We would rate the statistical significance importance as a 5 out of 10 (moderate.)
Customer satisfaction surveys
Reliant on more extensive samples than HR surveys, but at the same time, whatever emotions you define around the brand (even with a small sample) can be helpful. We would rate the statistical significance importance as a 7 out of 10.
Healthcare, education, and mass-market surveys (like the Coca-Cola survey example above)
These are crucially dependent on sample significance. Application in tandem with good segmentation exercises creates balance, but numbers count here to follow through with confidence. We would rate the statistical significance importance as a 9 out of 10.
Conclusion
The sample size is one aspect of market research that companies need help with. Sogolytics and other professional companies like it know how to use a sample size calculator to establish the right mix of variables to provide data you can use. Sogolytics takes the bias and “false hope” syndrome out of sample sizing. Moreover, they have the skills and talents to get the most out of surveys with employees and customers. Check your calculations with them before making the next move.
