Qualtrics: Designing Accessible Surveys

Use the Qualtrics New Survey Taking Experience (NSTE) when selecting your Qualtrics layout in the Look and Feel menu. Survey question types in the NSTE are all WCAG 2.1 AA compliant, including the Matrix Likert question type. The NSTE is also optimized for mobile devices, providing a better experience for respondents on smartphones and tablets.

However, please note that there are some more complex question types that are currently unsupported with the NSTE, and there are other question types that will not be migrated to the NSTE layout. For these reasons, we recommend users do both of the following: 

1. Become familiar with these unsupported/incompatible features now, planning to use Qualtrics’ listed alternatives
2. Enable the NSTE layout before building out a survey so that you don’t run into issues later. 

Old layouts (i.e., Flat, Modern, or Classic layouts) are only guaranteed to meet up to WCAG 2.0 standards. If you have to continue to use an old layout (i.e., Flat, Modern, or Classic layouts) for a methodology related reason, make sure that you:

• Select question types in Qualtrics that meet accessibility guidelines.
• Select question types that work well on mobile devices.
• Use Qualtrics ExpertReview to see if Qualtrics flags any of your question types as not meeting accessibility standards.

Important Note: The Qualtrics ExpertReview accessibility checker only checks if the types of questions you use are accessible. It does not check accessibility in terms of how the questions look or feel. To make sure your survey is fully accessible, including text formatting and how people move through it, please review the extra tips and information on this page.

• Update your survey’s display name underneath the survey options menu to something descriptive of your survey. The survey display name is how your survey shows up in browser tabs, social media posts, and search results, when applicable.
• Update your survey’s description underneath the survey options menu to something descriptive of your survey.

• Use section headings to organize survey content within survey pages and within large blocks of text (e.g., instructions, consent forms). Please note that each question should be a heading level. You can find section headings in a Qualtrics question’s Rich Content Editor. See the ODA’s Qualtrics Accessibility guide for more details on using section headings in Qualtrics surveys.
• Be as descriptive as you can about where a respondent is within the survey using text. This can include things such as adding section progress Text / Graphic questions throughout the survey, adding manual or auto question numbers, and telling people how many questions are in each part of the survey.
• Update survey button text to ensure that the button that continues to the next screen and the button to go back have text descriptions such as Next and Back.  
• Choose simple, predictable movement for various elements on your survey pages when applicable. For example, we recommend selecting None for the page transition setting under the Look and Feel menu. 

• Use unambiguous fonts for normal text. Within Qualtrics, we recommend Verdana or Tahoma. Arial and Helvetica are also acceptable.
• Choose a font size that will be comfortable for your potential survey respondents to read.
• Use left-aligned text whenever possible. However, if you are writing in a language that is written from right to left, use right-aligned text.
• When emphasizing text, use keywords like “Note” or “Important” in addition to bolding.
• Avoid using all caps for emphasizing information.
• Use italics for emphasis only in conjunction with other formatting options (e.g., boldface type).

See the ODA's guidance on text formatting for more information and resources. 

• Ensure that the text in your hyperlink is descriptive of the location of where the link will take a user.
  • Inaccessible example: “Check out this Wikipedia page on cats: ​ https://en.wikipedia.org/wiki/Cat.”
  • Inaccessible example: “Click here to find out more about cats.”
  • Accessible example: “Check out the Wikipedia page on cats.”
• Avoid multiple occurrences of the same link in the content of a survey page whenever possible. Duplicate links increase the number of links on a page through which screen reader and keyboard-only navigation users must navigate.
• Ensure that links stay visually distinct from other text. Make links in the main content look like links, and make sure other things don’t look like links.
  • No other text should use underlines for emphasis. Browsers underline links by default, and users understand what that means.
  • The color of links should be distinct from other colors used for text emphasis.
  • Avoid styling links to look like buttons.
• Set links to open pages in the same tab. In Qualtrics, links should open in the same tab by default. If someone wants to go back to the survey after clicking a link, they can use the Back button on their browser. If they prefer to open the link in a new tab, they can right-click the link and choose that option from the dropdown menu. However, please see the WCAG page on opening new windows and tabs from a link only when necessary for additional context.

See the ODA's guide on accessible links for more information and resources.  

• Ensure an adequate contrast ratio between background and foreground colors; this includes text color vs. background color, button color vs. background color, etc. You can review your contrast ratios using the WebAIM contrast checker tool.
• Never use color as the only means of conveying information.
• Although not a hard requirement, we also recommend setting a contrast ratio that is not too high. Black text on a white background can be stark and fatiguing, especially for those with light sensitivity issues (e.g., Meares-Irlen Syndrome, Scotopic Sensitivity Syndrome). It’s often better to have a dark gray text on a white background. We like gray with the hex code #595959 for text on a white background. You can also set the Foreground Contrast equal to Medium within the look and feel menu.
• Use solid backgrounds for text. Text on non-solid backgrounds is harder to read, which means increased cognitive load for your respondents.

Always insert tables using Qualtrics’ built in Table option within the Rich Content editor.

• Under the Table Properties menu, specify Headers.

See the ODA's guide on accessible tables for more information and resources. 

When creating bulleted lists or numbered lists, always use the bulleted list or numbered list functionality within Qualtrics' Rich Content Editor so that assistive technology (e.g., screen readers) will read the list correctly. Avoid manually creating bullet points or numbered lists.

When inserting images into your Qualtrics survey, always add descriptive alternative text to your survey image under the image properties menu. ODA’s Qualtrics Accessibility Google doc guide provides a step-by-step guide on how to update a Qualtrics image’s alt text. 

See the ODA's guide on alternative text for more information and resources.

When inserting media such as video or audio, either through Qualtrics’ insert media feature, or through embed codes from sources like YouTube or MediaSpace, always ensure that this media meets accessibility standards. This includes captions for video content and transcripts for audio-only content.

See the ODA’s guide on accessible Video and Audio for more information and resources.

• Your survey’s reading level should reflect the reading level of those who will be responding to your survey. General surveys of the United States population should aim for no higher than an 8th grade reading level.
• Provide a mechanism (e.g., accessible tooltip, additional description) for identifying specific definitions of words or phrases that are unusual, idioms, or jargon.
• Avoid abbreviations and acronyms unless they are a common abbreviation that all of your potential survey respondents will know.
   

In order to ensure that all survey takers have enough time to complete survey questions, the Auto-Advance setting of the Qualtrics timing question type should generally be avoided. 

Issues with the Show Timer

If you're using timing questions, it's also better to avoid use of the Show Timer option. The big moving timer can distract some people and make it harder to answer the questions, especially for those with attention issues like ADHD, or people who need more focus to read, like those with dyslexia or different reading comprehension levels.

If you are a researcher who plans to use the Auto-Advance feature or the Show Timer option as part of your study design, consider how these elements might unintentionally exclude certain groups of participants. This exclusion could influence the diversity of your sample and conclusions drawn from the data. We recommend writing this into your study design when submitting to the Institutional Review Board (IRB). 

While Qualtrics' reCAPTCHA question type aligns with WCAG guidelines by being screen reader-friendly, it's important to acknowledge that certain individuals may still struggle with reCAPTCHA tests for reasons unrelated to vision.

Recommendation: Instead of using a Qualtrics reCAPTCHA verification question, you can enable reCAPTCHA within your Survey Options menu. By doing this, Qualtrics will quietly assess participant responses in the background, flagging suspicious responses in an embedded data field. While checking these flags requires more effort for the researcher, it eases participants’ cognitive load and enhances survey accessibility for those who may find reCAPTCHA challenging.

Attention checks are a tricky subject. While researchers often want to implement attention checks to help them sort ‘bad’ quality data from ‘good’ quality data, many forms of attention checks can have issues related to participant burden and cognitive accessibility. It is important to consider how any given attention check method may affect a given survey’s group of potential respondents, as people with certain demographic characteristics may be more likely to fail attention checks (Anduiza & Galais, 2016). 

Because of this, our recommendation is to keep your attention checks covert. That is, survey respondents should not know that the attention check is happening. Examples of covert attention checks include:

• Tracking survey timing - Using Qualtrics' timing question to determine how long a respondent spent on a page. If a respondent spends not much time on a page, it’s an indicator that they were not paying close enough attention to the survey content.
• Response inconsistency - Determine if a respondent’s answer to two or more survey questions are inconsistent with one another. For example, if someone answered the question, “Do you think of yourself as closer to the Republican Party or to the Democratic Party?” with “Closer to Republican“ but then answered “Which of the following best describes your political leanings?“ with “Extremely Liberal," that would be an indicator that the respondent was not paying attention.

Considerations when Designing Overt Attention Checks

Overt attention checks are attention checks that the survey respondent would actually notice while taking the survey.

1. Ensure that your attention check measures attention and not other things, like memory or education level. Don’t ask participants to answer a knowledge question they may not know, even if it seems obvious to you. Don’t ask participants to recall something that previously happened in the study, as that is a measurement of memory as well, not just attention. For more information, take a look at these examples of good and bad attention checks.
2. Your attention check should have an extremely clear and correct answer. When creating an attention check, think about whether or not there is any room for a participant to interpret your question in a different way than intended.
3. Avoid attention checks that are intentionally tricky, as they may confuse your participants at best. Additionally, participants generally dislike them.
4. If participants are to be disqualified after failure of an attention check or multiple attention checks, the attention checks should be placed earlier in the study. This is especially true if there is a monetary incentive to complete the survey. This ensures that you don’t waste the participant’s time and they don’t waste your time.
5. Consider placing attention checks in the questions themselves instead of in the survey instructions or question instructions. Participants see survey instruction items at different levels of importance, suggesting that respondents may view reading instructions as an option used when further clarification is needed, instead of necessary instruction (Shamon & Berning, 2020).
6. Experienced survey takers (e.g., MTurk respondents) may be actively searching for attention checks. Some even have browser extensions that highlight words commonly related to attention checks.
7. Passing attention checks, by themselves, is not a sufficient stand-alone method for detecting insincere respondents (e.g., Kennedy et al., 2021). Don’t just rely on a single attention check to determine who is not taking your study seriously. Someone may accidentally fail a single attention check but perform well on other measures of data quality.
8. Just introducing attention checks has been shown to inadvertently affect survey results (e.g., Agley, Xiao, Nolan, & Golzarri-Arroyo, 2022) , but findings on this are generally mixed (e.g., Gummer, RoBmann, & Silber, 2018, Abbey & Meloy, 2017). Whether or not overt attention checks affect participant behavior in a survey study likely depends on context.

The Accessibility of Digital Content and Information Technology policy will be enforceable by April 24th, 2026. In the meantime, University units should work to ensure that their digital content and information technology meets the accessibility standard set by the policy by the April 24, 2026 implementation date. Until then the standards described in Administrative Policy: Accessibility of Information Technology are in effect.

Contact the Office for Digital Accessibility (ODA) with any questions related to the accessibility of your Qualtrics survey. You can also request consultations with the ODA. 

To make sure your Qualtrics survey meets WCAG 2.1 AA accessibility standards, start by following the checklist on this webpage when you create your survey.

We also suggest becoming very familiar with the 7 Core Digital Accessibility Skills listed on the Office for Digital Accessibility’s (ODA) website. These skills will help you understand why the tips outlined on this webpage are important.

Please note that you are ultimately responsible for programming and making sure your survey is accessible.

No, a Back button (i.e., previous page button) is not required for accessible survey navigation. We recommend choosing whether or not to have a Back button based on the specific purpose of your survey. 

The New Survey Taking Experience (NSTE) is set on a per survey basis, much like when you choose specific layouts or themes for individual surveys. The NSTE is not set up as the default layout, so you will need to manually choose the NSTE in the Qualtrics Look and Feel menu for your survey. 

To determine whether or not your survey is using the NSTE: 

1. Navigate to the Look and Feel menu for your Qualtrics survey.
2. Look for the Layout submenu.
   1. If you do not see a Layout menu, you are already using the NSTE.
   2. If you see the Layout menu, click into it and select the Try out the new survey taking experience option. Accept the prompts to move to the NSTE. 

3. If you are successfully in the NSTE, you will no longer see the Layout submenu within the Look and Feel menu. 

Check if there’s a similar question type available in the NSTE that meets your needs. Qualtrics also provides a list of alternative question types for those that don’t work in the New Survey Taking Experience.

We recommend using question types that aren’t part of the NSTE only if skipping them would seriously harm your project’s research methodology.

Yes. There are a few primary changes that will need to be considered if you need to create JavaScript with a survey that uses the New Survey Taking Experience: 

1. jQuery: jQuery is no longer included by default within the New Survey Taking Experience, but vanilla JavaScript still works.
2. Embedded Data: If you are using JavaScript to set embedded data:
   1. Replace the .addEmbeddedData method in your code with the .setJSEmbeddedData method.
   2. In the Survey Flow menu, add the prefix ‘__js_’ to an embedded data name used to store JavaScript-set embedded data. For example:
      1. If your JavaScript is: Qualtrics.SurveyEngine.setJSEmbeddedData('Num_Sports', 1);
      2. Then your embedded data name in the Survey Flow should be: __js_Num_Sports
3. CSS: If you have JavaScript that updates the CSS of different elements in your survey, please note that CSS id and class names within the New Survey Taking Experience are generally different from the id and class names used in the old Qualtrics layouts (i.e., Flat, Modern, or Classic layouts).

Important Note about JavaScript in Qualtrics surveys: Please note that it is your responsibility to ensure that the JavaScript you use in any Qualtrics survey meets WCAG 2.1 AA standards. 

Yes. The CSS id and class names with the New Survey Taking Experience are generally different from the id and class names used in the old layouts (i.e., Flat, Modern, or Classic layouts). You will need to adjust your custom CSS accordingly.

You're looking to update your longitudinal survey that's currently collecting data and will continue to do so through April 2026 and beyond. This is a common challenge, and it's great you're thinking about accessibility now!

To update accessibility for longitudinal surveys, we recommend making the lower-risk edits outlined below right away. Then, we recommend thinking through the higher-risk edits and creating a clear plan for tracking and analyzing responses from before and after the modification. 

Lower-risk Survey Edits

We consider lower-risk edits one that won’t disrupt data collection or the meaning of your survey. These include the following (the edit suggestions listed are described in more detail in previous sections of this webpage): 

1. Add alt text to images: Adding alt text to Qualtrics images helps screen reader users without changing how the image looks.
2. Specify headers and summaries within tables: Specifying headers and summaries within tables helps screen reader users without changing how the table looks.
3. Add headings to existing survey text: Adding headings to existing survey text improves assistive tech support without changing the word content of your survey. Only do this when the existing text is meant to be a heading. Use headings as a way to organize sections of questions, much like you use headings in Google docs. Each individual question does not need to have a heading.
4. Use color contrast with accessibility in mind: Improving color contrast makes text easier to read without changing how the survey works. If color carries meaning, use lighter or darker shades of the same color to meet contrast guidelines.
5. Modify a link’s target to open on the same tab: Make sure links do not open to a new tab or window. Qualtrics has the correct default behavior when you first create the hyperlink.
6. Format bulleted or numbered lists correctly: Updating your lists to use the bulleted list or numbered list functionality within Qualtrics' Rich Content Editor helps screen reader users without getting rid of the bulleted list. 

Higher-risk Survey Edits 

For any changes that might impact the interpretation of your data, make sure you have a clear plan for tracking and analyzing responses from before and after modifications for accessibility. 

• Update a link to be descriptive: If a link is an integral part of a question (e.g., "Click this link and then answer..."), changing the link text could fundamentally alter the question and the resulting data. Updating language may also require an update to the Institutional Review Board (IRB) if you are working on academic research.
• Update to the New Survey Taking Experience: Updating your survey to the New Survey Taking Experience (NSTE) will change the formatting, available question types, and custom code (i.e., CSS or JavaScript functionality of your survey. If you need to update to the NSTE, we recommend:
  1. Creating a copy of the currently running survey,
  2. Updating the layout of the copied survey to the NSTE,
  3. Updating and thoroughly testing the new NSTE survey copy, including generation of test data,
  4. Switching out the old survey link for the new NSTE survey link when ready
     • Tip: If your old survey link is embedded in a lot of places, you can consider having your old survey link forward to the new NSTE by putting an End of Survey element at the top of the survey flow and using the Redirect to a URL option. Don’t forget to publish changes!
  5. Additional step: If you are using the Qualtrics API, remember to also update any IDs in your script(s) to the new survey’s IDs. 

In Summary

Focus on implementing accessibility features that won't disrupt your ongoing data collection or alter the meaning of your questions. For any changes that might impact the interpretation of your data, make sure you have a clear plan for tracking and analyzing responses from before and after the modification. 

We recommend sticking with old layouts (e.g., Flat, Modern, Classic) for longitudinal studies only if switching formatting mid-data collection pose a significant risk to your project’s research methodology. If this is the case, it is your responsibility to provide an accessible alternative that works well within the context of your study. If you have any questions related to your specific survey accessibility challenges, please contact the Office for Digital Accessibility (ODA).