Jump to:

• Best Practices for Facilitating
• Resources Needed for User Testing
• Running a User Test

Best Practices for Facilitating

Facilitators play an important role in explaining the test beforehand, walking through the test with the participant, and taking notes for analysis.

• Before the test:
  • Institutional Review Board: Prior to your testing, you should follow your institution’s policy for working with human subjects; usually called the Institutional Review Board (IRB).  If you plan to publish your research results you must go through your IRB. This will promote the safety and well-being of human participants, ensure adherence to the ethical values and principles underlying research, and alleviate concerns by the general public about the responsible conduct of research.
  • Consent: Always get consent from a test participant. This is an example template for a consent agreement.
  • Data Privacy: Have a data protection plan for personal identifying information. This includes personal names, identification numbers, email addresses, telephone numbers, or birth dates. Consult your institutional research department about institutional privacy policies.
  • Inclusive design: It is critical to understand different types of user capabilities and how they affect product interaction in general. The Inclusive Design Toolkit by the Government of Ontario  offers some design cards can help you identify interactions and processes. Learn more about Liberatory Design to embed in your user testing as well.
• During the test:
  • If during testing a participant decides to not complete a test, that’s okay.
  • Make sure it is clear that participants understand that they are helping us; there are no wrong answers.
  • Stick to the time limits for each activity.
  • Make sure to ask participants to think aloud while completing test activities. This helps note-takers understand participants’ thoughts as they occur and as they attempt to work through issues they encounter. It will also elicit real-time feedback and emotional responses that are important for knowing if an interface works or doesn’t.
  • Take good notes: capture as much detail as possible. Record what they say out loud as they move through the tasks. This will help with analysis later.
  • Capture subjective metrics you notice during the test. Note things like “they hesitated a while” or “they looked around before clicking,” etc.
  • Be neutral during a test. Simply watch and listen to participants. When they ask a question, reply with “What do you think you would do?” or “I’m more interested to know what you think or would do.”
  • Don’t jump in and help. This will bias the results. If someone wants help, either reassure them that they can’t make any mistakes, or end the task and move on to the next one.
• When the testing ends:
  • You can demonstrate any workflows or programs the participant had questions about if it seems appropriate.
  • Inform them of any follow-up that may come, if applicable. 
  • Thank them!

Resources Needed for User Testing

The resources you will need to conduct user testing depends on what you have available at your institution external to who will be coordinating user testing. Consider these three types of resources: test participants, recruitment incentives, and technology.

Test Participants

You will need at least 5 participants. Depending on your testing topic, your focus might be one specific group (ex: Undergraduates, Researchers, etc.) or be more broad (ex: Faculty focused or student focused).  Who you choose should depend on context and what you are testing (ex: course materials search for undergrads; data management support for researchers). If your topic is broad and reaches many users, you might try for 5 in each user group. For guidance on interacting with users, reference the Working with Test Participants component of this guide. Sometimes you will not be able to get all the users you need. Remember, any user testing is better than none at all

Recruitment Incentives

Recruitment incentives are a key way to get test participants to engage in your study. Incentives for test participant recruitment also depends on your overall budget, and amount of time for each test.

Examples

• Gift cards ($5-50) for a grocery store, campus bookstore, etc.
• Cup of coffee or cold drinks on hot days
• Food, such as pizza, doughnuts, or candy
• Library swag, e.g. awesome mugs and pens
• Extra credit in a course (optional if collaborating with instructors)

Technology

Basic technology needed to perform testing:

• Laptop computer with mouse (for accessibility) and wireless internet
  • Desktop optional, if you have a setup/ location for users to test
• A range of updated web browsers (Firefox, Chrome, DuckDuckGo)
• External microphone
• Video screen capture
  • Zoom is the most common tool (Pro tip: turn off the webcam so it only captures the screen clicks and audio.)
  • Other options include: Quicktime, TechSmith Capture (formerly Jing), Camtasia, SnagItEye Tracking software (optional)
• Want a deeper dive, try eye tracking software to track where the user looks, instead of just where they click.

Assistive Technology

You should always try to do user testing with someone who uses an assistive technology device. It helps to make systems more inclusive to all types of abilities. It is recommended to at least test with a screen reader. Below are some recommendations and how-tos:

• List of screen-readers endorsed by the American Foundation for the Blind
• Common screen readers and text-to-speech software, including guides for use

People use assistive technology to support accessibility needs beyond sight, such as for cognitive, hearing, movement, vestibular, and other disabilities. Assistive technology could take the form of screen readers, screen magnifiers, color contrast, speech-to-text, text-to-speech, communication support, keyboard-only navigation, and more.

Running a User Test

Determine where to conduct testing and get your resources in order

Figure out if you are going to approach running your test formally or using the Informal / Pop Up method outlined below. It’s preferred to test people in their natural information-seeking environment, but if you want to run tests in a quiet space, that’s welcomed as well.

Recruit Test Participants

How you get test participants to do your test requires you to do outreach. Think about the following before you try to engage them:

• Where are your users?
• How do you reach them?
• What do you need to communicate to them?
• What do you have to offer them?
• What library workers, archivists, faculty, and other academic staff or contacts do I know that can help me spread messaging?

More resources on recruiting test participants can be found from the Recruiting Participants resources section.

Informal / Pop Up Method

Pop up or more informal on the spot testing is a useful method depending on what you are testing.  Walking around the space, approaching tables of users, offering a snack or food and asking if they have time for a quick question (oral or on an index card) is one method. Tiny Cafes, where you pop out a table, offering food/drink, for a quick test or question is another. Similarly, the Hallway method is another creative, on the fly testing method. In the university environment, if students are your users of interest you can often set up your user study in a cafeteria, library, commons room or hallway and select people as they walk by. (See Selecting and Recruiting User Test Participants for more information.)

• Prepare testing space and computer
• Approach a person and ask if they would like to participate in user testing; if you are looking for a specific type of user, ask if they meet those specifications. Another option is to have a table and sign welcoming them to stop by for a snack and feedback so they approach you.
• Provide an overview of what you’re testing and why.
• Be prepared to engage in testing right away

Formal Recruitment

• Reach out to potential subjects by email, phone, email lists, social media, flyers, advertise in student newspapers, dorm rooms, department newsletters, screens around campus, and word of mouth through your student employees.
• Make sure test participants know the intention and purpose of the test.
• Schedule a time(s) and place to conduct testing.
• Select specific people that match the group(s) to be tested. Confirm with them the location and time. Assume some people will not show up so consider adding drop ins or extra people.

Perform the test

• Make sure location is reserved and testing instruments are set up for testing. Try a practice session to make sure it is all working correctly.
• Introduce the participant to the test and have them sign consent agreements.
• Give the participant tasks to complete and make sure they explain and talk through what they are thinking and doing while they perform the task.
• Watch and take notes about the participant’s actions and comments.
• Give the participant their incentive once the test is completed.