Hello everyone. Last time, I wrote that “fact collection” is necessary at first for problem solving, and that fact collection can be broadly divided into “external research” and “internal research.” And there were “secondary research (public information collection)” and “primary research (original research)” in external research, right? This time, I would like to write about the survey planning (generally called as survey design) when conducting the primary research.
When conducting primary research, I think there are many so-called “customer surveys”, so I will write on that assumption. To design customer surveys, it is necessary to decide items such as “1. Survey objectives” -> “2. Target customer segment” -> “3. Analysis method” -> “4. Number of samples”.
1. Clarify “Survey objectives”
It is NG to just list the question items, saying, “I’m going to do a customer survey, so let’s think about what we want to ask.” Even in the situation, I think that the answer results would come out, but we don’t know how to use that data, and nothing would happen after the result report meeting. In order to prevent this from happening, let’s clarify “what is the purpose of the survey” at the survey design stage.
Examples of survey objectives include the recent trend of “improvement of CX (customer experience)”, and the long-standing popular ones of “measurement of customer satisfaction”, “measurement of NPS (net promoter score)”, and “customer needs survey in products/services development”. I think there are many other things.
2. Define “Target customer segment”
Customer segmentation is an in-depth field (so I won’t go deeper into it here), but I think it can be broadly divided into “demographic segmentation (e.g. segmented by attributes such as age, gender, area of residence, etc.)” and “psychographic segmentation (e.g. segmentation by psychological attributes such as ‘fashionable consumer’, ‘frugal’, ‘conservative’, etc.)”.
When actually defining it, in the case of B to C, I think that we often need to contact to research companies that has research panels, but there are attributes that are “easy to obtain / difficult to obtain” within the panel, so it is realistic to consider the segmentation that can be taken. In that sense, I think demographic segmentation would be the basics.
In the case of B to B, I think that it is difficult to find the targets by using the research panels of research companies in many cases. If your company has already conducted a related customer survey, you can use it, but if not, you may need to coordinate with your marketing or sales department to plan a customer survey.
3. Consider “Analysis methods”
This is related to “1. Survey objectives”, but it is necessary to consider “analysis method” at the stage of survey design. This is because the actual questionnaire of the survey would be changed depending on the method used. Here is a summary of common analysis methods according to major research objectives.
There are many other existing analysis methods, but I think the above ones are relatively useful. So I would like to write these in order from the next time onwards.
4. Decide “Number of samples”
What is the appropriate sample size for customer surveys? I think many of you have this question when you do customer surveys.
Actually, there is a general formula for calculating the number of samples. If you are interested, try to google it by using keywords such as “sample number calculation”. You may find so many pages, so let me put only one link here. The point of this formula for calculating the number of samples is that the required number of samples changes depending on what percentage you set the two parameters, “Confidence Level” and “Margin of Error”. It is common to set as Confidence Level = 95% and Margin of Error = 5%.
With this Confidence Level = 95% and Margin of Error = 5%, the required number of samples (depending on the size of the population) is roughly 400 people. It is useful to remember this number of 400 people. Practically speaking, it is common to analyze multiple segments in one survey (e.g. men in their 20s, women in their 20s, men in their 30s-40s, women in their 30s-40s,,, etc). At this time, if the total number of samples is 400, and if there are 4 segments, 100 people in each segment would be fine. But if there are 8 segments, each segment would have 50 people, so the survey results might be somewhat not enough. By the way, when it comes to public opinion polls by major newspapers in Japan (the population is the entire Japanese population = about 126 million), I think there are many surveys with a sample size of about 1,500 to 2,000 people.
Statistically, there are various theories such as at least 20 or 30 people, but from my experiences, it would be better to have 100 samples for each segment. For example, if there are missing values (blanks) in the sample data, deleting the sample including blanks might impact the analysis results considerably. This is because the larger the number of samples, the more stable it will be, even if some samples are removed.
And, the rest is a balance with the budget 🙂 The larger the sample size, the higher the cost of the research company.
That’s all for this time, and I would like to continue from the next time onwards. Thank you for reading until the end.