Methods

Interviews

Very useful for gathering first-hand accounts of users' opinions, stories, and experiences; favours depth of data over amount of data.

What people say they do and what they actually do can differ; somewhat time consuming; can be expensive.

Surveys

Large number of participants allows for statistical analysis; allows for rapid and remote data collection; low cost and reusable; often used to validate findings from other research methods.

Participants may not be truthful; survey bias (only certain people types tend to fill out surveys); finding and connecting with participants can be tricky.

Focus groups

Allows for a large amount of data in a short time (1–2 days); group setting creates an environment where participants are more open to sharing their thoughts and experiences; provides insight to compare and contrast participants.

Can have negative group dynamics if there is a dominant personality; can be expensive/difficult to coordinate.

Diary study

Can collect a large amount of user data over a long period of time; allows researcher to see how people react while ‘in the field’; gives longitudinal usage data (compare usage after one week and then after one month, is there a learning curve?).

Participants may not be truthful; often requires a long period of time; relies on participants being diligent and filling out the journal as agreed without missing days.

Task analysis

Step-by-step analysis of user actions to identify optimal flow and the tools required to support their activity; behavioural approach that sees how users actually use a product/service.

Research is purely behavioural; can be tricky to setup testing environment.

Ethnography

Participants are observed in the natural setting of the ‘context of use’; most objective measure of context of use; allows for capture of non-articulated and tacit information.

Observer effect (participants may change their behaviour because they are being observed); can be expensive (travel costs).

Contextual inquiry

Combination of ethnography and interview; helps understand workflows, communication processes, tools used, physical and cognitive requirements, and cultural influences.

Observer effect (participants may change their behaviour because they are being observed); has to be done in-person.

Card sorting

Informs the IA by uncovering on how users organize information; cost effective and fast; open card sort helps identify language used to describe groupings; closed card sort validates groupings and language.

Insights are narrow.

Usability testing

Used to determine pain points (start of research) and then validate IA and interaction design changes; includes A/B testing, heat maps, first-click test, etc.

Can be expensive; research is often performed in a usability testing lab so participant is removed from their actual environment/context of use.