Overview

Observation is in many ways similar to several other methods, such as think-aloud technique, task analysis, hard case technique, and scenarios. The core of observation is observing what someone does while they are doing it (in the case of direct observation) or inferring what they have done from the evidence they leave behind (indirect observation). Direct observation may be undisclosed (where the person being observed does not know that they are being observed) or disclosed (where the person does know that they are being observed). Undisclosed observation requires more stringent ethical clearance than disclosed observation.

In this tutorial, we focus on disclosed direct observation.

Advantages of observation

• Inexpensive
• Useful for seeing what really happens
• Good for finding out the reasons for what happens

Disadvantages

• Observation usually gives fragmented insights, not a clear overview

What you need:

• An activity for the participant to perform
• The tools, materials etc needed for the task
• A recording device and/or paper for notes

• You won’t be able to keep accurate handwritten notes in real time while also observing what’s happening.
• People are usually OK with being audio recorded, and quickly stop paying attention to an audio recording device.
• People are usually self-conscious about being video recorded, and this usually lasts through the recording session.

• Explain what you would like the participant to do.
• Give them a quick demonstration involving an activity from an area that’s very different from the one they’ll be doing, to avoid bias.
• Make it clear that there are no right or wrong answers.

Tip: Don’t let them see the material until you’ve finished your demonstration, or they’ll start doing the task  before you’ve finished explaining.

• Resist the temptation to talk, make suggestions, or help; you’re trying to find out about the participant’s knowledge, with as little bias or steering as possible.
• Make a note on paper or mental note of any significant actions that would not appear on the recording.
• If they are silent for more than the agreed time, use a non-directive prompt, such as: “Could you tell me what you’re thinking about?”

• Once you have identified the key actions and concepts, you can collect quantitative data using various approaches.
• Timelines let you see what is happening when; for instance, how often people hesitate or swear when they try to use a particular feature.

You can use the outputs from observation (e.g. concepts identified as important) as inputs for other methods, for instance within visual analogue scales. For example, if the participant often looks uncertain during the task where they are being observed, you might choose to use a visual analogue scale question about how clear or how difficult the task is.

Observation can be used to identify places where hesitations and errors often occur, so that the causes can be designed out of the system (whether the system is a software, bureaucratic, or physical system).

Copyleft Hyde & Rugg 2021