Question(able) Format: Missing ≠ No
Being able to distinguish missing responses from ‘no’ responses is an important but often overlooked issue in survey design and analysis. A missing value and a response of ‘no’ represent two different circumstances: the former indicates a nonresponse—a respondent refused to answer several items, or maybe they chose to skip a question altogether; while the latter signifies the absence of an attribute—a ‘no’ response.
Let’s consider an example to illustrate this point. The traditional ‘select all that apply’ question format only records the presence of an attribute (i.e., ‘yes’ answer choice). For instance, this ‘select all that apply’ question:
Which work-based learning experiences
have you participated in? (Select all that apply.)
would generate the following data:
And most people would recode all missing values in the data set to zero (0) for analysis. However, because respondents could only identify which work-based learning experiences they participated in, by recoding the data in this way, we cannot distinguish between data that are truly ‘missing’ and a ‘no’ response.
To avoid confusion between data that are missing (for some reason or another) and ‘no’ responses, choose an alternative question format that contains two (or more) answer choices that can be applied to all items. In our case, the ‘select all that apply’ question could be transformed into a 'yes-no matrix' question, where respondents are asked to evaluate the same series of items, but must select either ‘yes’ or ‘no’ for each statement:
Please indicate "Yes" or "No" to the following..
I have participated in...
| Yes | No | |
|---|---|---|
| A cooperative education program | ||
| An internship | ||
| A job shadowing or work-site visit experience | ||
| A mentoring program | ||
| A community service event | ||
| A school-based enterprise |
The data generated from the 'yes-no matrix' question are easy to interpret: a one (1) is a ‘yes’; a zero (0) is a ‘no’; and blank values represent missing data:
As a bonus, we now also have more data to draw conclusions from and act on. That is, using the 'yes-no matrix' question format, we can examine experiences respondents participated in (answered ‘yes’ to), and those respondents did not participate in (answered ‘no’ to).
A missing value is different from a ‘no’ response and zeros (0) should NOT be substituted for a missing value. When developing a survey, be mindful of how question format may affect data analysis and interpretation.
Question format not only determines the type of data that are generated, but it also limits the type of analysis that can be conducted.
