What is it?

A survey is a research method used to collect quantitative data about users' thoughts, feelings, and behaviours related to a product or service.

Surveys are useful when you need to collect data quickly, you want to validate your hypothesis on a large population to make a decision about a particular feature, solution, or experiment, and you have limited UX resources—it is much faster and cheaper to run a survey than conduct dozens of interviews.

Why is it important?

Surveys are great for collecting a large amount of data from a wide range of user groups, giving users the flexibility of providing their answers on their own time. Surveys typically ask specific, close-ended questions to answer ‘when, where, what’. If your team is more interested in open-ended questions such as ‘why and how’, use qualitative research methods instead, such as interviews, moderated usability testing, and job-shadowing.

When to use it?

There are two main phases of the design process when surveys are typically conducted, Discovery and Evaluate. When in the Discovery phase, a generative survey type is ideal. When in the Evaluate phase, an evaluative survey is best.

Generative survey

Generative surveys are called “generative” because they generate insights that help you learn about your target audience by providing insight into people’s characteristics, thoughts, motivations, demographics, behaviours, goals, concerns, and pain points. Use generative data to create personas, journey maps, and service blueprints, which are powerful ways to communicate the findings to team members and stakeholders so that you can identify and prioritize specific areas where you might be able to provide value.

We recommend using qualitative research methods whenever possible to generate insights in the Discover phase (such as interviews and job-shadowing), but surveys can be better than nothing. Do not use generative surveys to evaluate the effectiveness of a feature or product.

Evaluative survey

Evaluative surveys are used to determine the success or failure of a specific feature and identify potential improvements, which means they are conducted post-implementation. It is important to conduct evaluative surveys for products that are in production so that users can give feedback that is grounded in the context of their day-to-day lives.

How to use the template

When building any type of survey, it is a good idea to include screener questions. Screener questions will help you identify different types of users that map to your personas, and potentially different usage scenarios as well. Depending on the goal of your survey, you might ask screener questions that give you demographic information, how often someone uses a specific feature of a product, where they use a product or service, what their role or job title is, etc.

The survey template we created includes screener questions and general questions designed to evaluate a product feature for an internal app: Evaluative Survey Template. Feel free to add, remove, or change any of the questions to cater to your specific situation.

Tips

When designing an evaluative survey, there are a few general principles to follow to make sure the survey is reliable, non-biased, and inclusive.

Use inclusive language

Sometimes we use words that can unintentionally exclude segments of the population based on attributes such as gender, race, and religion, and this is considered exclusive language. Inclusive language removes these stereotypical attributes so that the word can apply to everyone. For example, we can use the term ‘flight attendant’ instead of ‘stewardess’.

For more guidelines on inclusive writing, refer to Government of Canada Inclusive Writing guide and the GBA+ guidelines.

Ask non-leading and unbiased questions

Leading and biased questions can make a respondent answer questions in a way that they normally wouldn’t answer, or can push them towards a particular answer because it sounds like there is a ‘right and wrong’ answer. This will result in misleading and unreliable survey data. For example, instead of asking “how good is this feature?” (which is biased in that assumes the feature is good to begin with), a better question could be “On a scale of 1-5, with 1 being not useful and 5 being very useful, how useful is this feature to you?”.

Collect the least amount of personal information possible

It can be tempting to ask a lot of demographic questions as screener questions because we often think the more data we have, the better. It is important to be respectful of participants' data and only ask for the minimum amount of personal information based on your research objectives. For example, you might be interested in learning how a feature is being used by inspectors remotely on job sites. In this usage scenario, gender or race likely doesn’t impact the user experience when someone is trying to perform a safety inspection on a moving ship, so don’t ask users to provide that information.

By only asking for the minimum amount of personal information, participants will usually feel more open, honest, and trusting, which improves the quality of the data and strengthens the relationship between the product team and their users.

The GC Task Survey

The teams behind the Canada.ca Design System (Digital Transformation Office and Principal Publisher) created a survey component to collect user data. The GC Task Survey is a “continuously running web intercept survey that asks questions related to three key elements: task success, ease, and satisfaction.” It can be implemented on any Canada.ca website and has a standard set of seven questions. The only thing you have to do is choose and write the tasks.

Visit the GC Task Survey website for more information, including how to contact the DTO and PP for implementation help.

Resources

Generative versus evaluation research - Usertesting.com

Types of user research methods - Userinterviews.com

70+ Great User Testing Questions To Ask Before, During, & After User Tests

Beyond the NPS: Measuring Perceived Usability with the SUS, NASA-TLX, and the Single Ease Question after tasks and usability tests

Usability Testing Questions Rules and Examples

https://www.canada.ca/en/women-gender-equality/gender-based-analysis-plus/apply-to-work.html

https://www.noslangues-ourlanguages.gc.ca/en/writing-tips-plus/inclusive-writing-guidelines-resources

GC Task Success Survey