How long should my survey be?
One of the main reasons for quitting the survey is that the questionnaire is too extensive.
Methodological ethics dictate that the respondent must be informed in advance about the time that is required to complete the survey. It is also expected to provide a progress indicator that clearly depicts at which point of the survey process the respondent is currently situated.
The estimated time required to complete the survey can, in general, be divided into the following categories:
- Very short surveys take up to 2 minutes to complete. In such cases, the length does not have an effect on users quitting the survey, nor is length an initial obstacle;
- Short surveys take from 2 to 5 minutes. They require a certain amount of respondent motivation, which can be achieved especially with interesting topics;
- Medium-long surveys require from 5 to 15 minutes to complete. In addition to interesting topics, respondents require an additional motivation (e.g. affiliation to the same group, association, class, place, etc.), encouragement or incentive (e.g. survey results, monetary incentives, etc.). This is also the duration that respondents to the surveys they are interested in, agree to participate in advance (if asked or informed). Longer duration requires serious and explicit incentive factors;
- Long surveys (15 - 30 minutes) require more intensive and direct ways of encouragement. It is important to carefully consider how to motivate respondents and how to ensure that they do not quit the survey prematurely. Long surveys are not uncommon in the survey industry, especially in online panels that usually reward participation with points or money (e.g. 5€);
- Very long surveys take more than 30 minutes of the respondent’s time. For such surveys to achieve reliable levels of quality and response rate, intensive methodological consideration and knowledge are required in regards to motivation, incentives and encouragement.
For all surveys that take more than 2 minutes to complete, one should check the following:
- Are all of the questions really required? Experience shows that most of the included questions are not analysed later. Because researchers often become too involved with the research topic when designing the questionnaire, they believe that even too specific questions are relevant;
- Did we consider the principle of randomized blocks that allocates less important questions to different subgroups that are assigned only to specific respondents? The majority of surveys contain an unnecessarily large amount of units for questions that are not of key importance. Especially for scale questions (1-5, 1-7) smaller samples are sufficient – it is suitable to use only a half or a quarter of the entire sample (more details >>).