Quality index (reminder tracking)

In the 'Dashboard' – 'Reminder tracking' tab, we can monitor where the respondents triggered the alerts in the questionnaire. There are four types of alerts: alerts for mandatory questions, numbering alerts, sum alerts and validation alerts. Using alert tracking, you can calculate i.e. quality index, which tells us how quality our questionnaire is, and consequently we can improve it with its help. The quality index (Haraldsen in Callegaro, 2015) is calculated using the formula below:

If we want to use the alert tracking feature, we must first turn on the collecting of advanced para data. We do this in the 'Edit' – 'Settings' tab, where we choose option 'Advanced paradata'.

We have two options – display by respondents and display by variables.

DISPLAY BY RESPONDENTS

In this view, triggered alerts are displayed by respondents. The row records the number of times the respondent triggered a certain type of alert (in Figure 1 we can see that the respondent 3 triggered the following alerts: 2 hard warnings at sum, 1 hard and 1 soft at number, 5 hard and 1 soft on mandatory questions and 1 hard at validation). At the end of the table it is written how many of the possible alerts that the respondent could trigger and the number of alerts triggered (how many different types of warnings are possible). The number of all possible warnings that the respondent could trigger is in our example 8, and the respondent triggered 6 types of alerts.

In the last line (coloured blue), it is written how many alerts can be given depending on the type of warning (in Figure 1, it can be seen that for sum alerts there is 1 hard warning, 1 hard and 1 soft warning for number alert, for mandatory questions 3 hard and 1 soft warning and for validation one hard warning). Both respondents together triggered 9 different alerts (6+3) and the number of all types of alerts is 6 (counting the number of different types of warnings – so even with hard warnings of mandatory questions are only counted once).

DISPLAY BY VARIABLES

There is also a display of variables – here we can see how many and which alerts each question has on the survey. In the bar, we can see how many times an alert was triggered, and at the end of the line, it is written how many alerts a variable has. Thus, in Figure 2, we can see that Q1 has one possible warning (a hard warning of the mandatory question); Q2 also has one warning (a soft warning of the mandatory question); Q3 has two warnings (number alert and mandatory question alert); Q4 has a soft warning, Q5 has a hard validation warning and Q6 has two warnings (a hard warning of sum alert and a hard warning of mandatory question alert).

In the last line (coloured blue), we can see how many alerts have been given depending on the alert type – in our case, we have a hard warning of sum alert, a hard and soft warning of number alert, a hard and soft warning of the mandatory question alert and a hard warning of validation alert – total of 6 different types of alerts.

Functionality is available for administrators.