Survey Design

See how to design a high-quality survey by understanding when to use specific question types, how to write questions, and how to format and organize the survey.

Challenge

Designing a high-quality survey can be challenging when trying to determine when to use specific question types, how to write questions, and how to format and organize the survey.

Solutions

Item Writing

The guidelines for writing quality survey items can be categorized as either closed or open-ended questions. All questions should be as specific as possible and use words that virtually all respondents will understand.

Before you begin to collect data, you should always review your questions, addressing content, cognition, and usability.

• Content. Are the questions relevant to the research topic?
• Cognition. Do respondents understand the questions being asked?
• Usability. Can respondents complete the survey easily?

Testing the questions with outside parties is also helpful. Some aids include:

• Expert Reviewers: See Experts and Communities for a list of offices on-campus who can help you plan, review, and edit your survey.
• Field Tests: Presenting your survey to a focus group or trial population will help you anticipate every answer and point out the strengths and weaknesses of a survey.
• Behavior Coding: Use a third party to monitor the interaction between the interviewer and respondent in a survey field test and look for problems in the questionnaire or its administration.

Survey Formatting

When it comes to formatting your survey, there is no right or wrong answer. However, a survey that is disorganized can affect the respondent’s understanding of certain items and consequently, their responses. It is important that your survey is thoughtfully constructed to elicit responses that are reliable and valid. Below is a list of visual and organizational components to consider when formatting your survey.

Visual components

• Blank Space: Blank space allows respondents to focus on one item at a time and reduces fatigue throughout the survey.
• Answer Design: Answer spaces should be indented and inserted underneath the item as opposed to the end of it so respondents do not have to move their eyes across the whole page or screen.
• Images: Use images sparingly as they can draw attention and distract the respondent. If they must be used, keep them consistent in size and shape.
• Color and Contrast: Create a focal point by including a lightly shaded background for the item and white space for the answer.
• Orientation: Paper surveys should be printed in portrait orientation rather than landscape as this is the most conventional and familiar format.

Organizational components

• Item Order: Interesting items should be presented at the beginning of the survey and all items should be easy to complete. Objectionable items should be put at the end of the survey to reduce the threat of low response rates.
• Similar Questions: Similar types of items or questions should be presented in the same question format.
• Multiple Items on Screen: Including multiple items, on-screen sometimes results in faster completion times and less missing data. Therefore, you should organize similar items on the same screen.
• Alignment: Vertically align-items if there are multiple items on the screen.
• Paging vs. Scrolling: Some research has found few differences between paging and scrolling surveys. However, scrolling surveys tend to have a higher rate of non-substantive responses.
• Matrix vs. Expanded Format: Matrices should be used sparingly or at least kept simple. The task of matching columns to rows is complex and respondents are more likely to miss items when they are arranged this way.
• Radio Button vs. Text Box: Using text boxes instead of radio buttons often results in missing data. However, if the respondent chooses to fill out the text box, it often results in higher quality data.
• Progress Indicator: This is generally recommended unless the survey is long and including one would exacerbate the perceived length of the survey.