Preparing to Write Survey Questions

Choosing Your Data Type

Qualitative

Data that is verbally based (words and concepts).

Strengths

Weaknesses

Offers insight into research questions.

May identify emerging trends in the data not previously considered by the researcher.

Provides more direct representation of subjects’ responses.

Requires multiple stages of data analysis.

Introduces researcher subjectivity in data analysis (see Survey Data Analysis Guide).

Quantitative

Data is numerically based (numbers only).

Strengths

Weaknesses

Allows for the direct application of statistical models such as ANOVA or t-tests (see Survey Data Analysis Guide) to identify general trends and patterns.

Potential for large data sets.

Limits insights to what the data shows statistically.

One survey data type is not necessarily better than another. As summarized by Ahmad (2019), “Quantitative data can help to see the big picture. Qualitative data adds the details.” What type of data you need is going to be dependent on what you are trying to analyze.

Choosing Your Question Format

Open-Ended Questions

Open-ended questions are those in which a survey respondent can generate a unique response using their own words. These are seen predominantly in qualitative data surveys. These types of questions are particularly useful when information is needed about individual-specific context that might not be accounted for in a multiple-choice type closed-ended format. A key benefit of open-ended questions is that they allow for respondents to give personalized responses that are not confined to the choice selection set by the researcher. This is simultaneously a key disadvantage, however as it introduces the need for qualitative coding and in turn, introduces a new source of error.

Some examples of open-ended survey questions would be:

What was your impression about working in groups?
Describe any study techniques you found to be beneficial.

Open-ended survey questions are generally useful for small sample populations. Though these questions offer great insights, on a large sample population scale they are often unfeasible due to the administrative planning and analysis required. If you want the insight from open-ended questions but are working with a large respondent pool, a small sample population can be given an open-ended question-based Pilot Survey to obtain information and inform the design of close-ended question surveys for your larger populations.

Closed-Ended Questions

Closed-ended questions are those in which the response options are limited and provided with the survey. These can be used in qualitative data acquisition as well as quantitative. Some of the main benefits of closed-ended questions are the reduction of the need for communication skills on the behalf of the respondent and the ease of analysis. Conversely, some of the main disadvantages include lack of depth in responses and lack of emergent insights (Hyman and Sierra 2016). These question types help to eliminate sources of error in the data analysis by reducing or eliminating the need to code free response answers; however, the data obtained will be limited to response options generated by the researcher. Closed-ended survey questions can often be evaluated statistically and are easier to use when evaluating large sample sizes, which encourages studies that are more generalizable. Some examples of closed-ended survey questions would be:

Did you enjoy working in groups?
- Yes
- No

Select any and all of the following study techniques you used for this course.
- Flashcards
- Rewriting Notes
- Group Study
- None of these

Thinking Ahead

Often, qualitative data are linked to open-ended questions while closed-ended questions are paired with quantitative data. The reality is that open-ended qualitative questions can be converted into quantitative data and conversely closed-ended quantitative questions can be used to glean qualitative data. The key is the coding, so write the questions in whichever way will give you the data you are most needing to see while keeping in mind the logistical elements that come along with delivering each type to the population you are studying.

References

Ahmad, S., Wasim, S., Irfan, S., Gogoi, S., Srivastava, A., & Farheen, Z. (2019). Qualitative v/s. Quantitative Research- A Summarized Review. Journal of Evidence Based Medicine and Healthcare, 6(43). https://journals.indexcopernicus.com/api/file/viewByFileId/916903.pdf

Hyman, M., & Sierra, J. (2016). Open- versus close-ended survey questions. NMSU Business Outlook, 14(2), 1–5.

This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.