We encounter usability every day: from coffee machines to mobile phone apps to ticket machines. In human-computer interactions, usability is crucial for the user experience. In the worst cases, users are confronted with negative emotions. For example, if someone wants to buy a transport ticket in Washington, they are likely to find themselves in front of one of the ticket machines shown above.

Quelle: Twitter@RianVDM

Due to the large number of chaotically arranged markings and labels, it is anything but intuitive to use. It can be assumed that inexperienced users will have difficulties buying the right ticket and will also experience negative emotions such as frustration or stress. However, there is a simple way to avoid bad user experiences: usability tests.
The aim of a usability test is to detect problems and weaknesses in the use of a software application or device. To do this, a test person completes typical use cases (tasks) with the help of the test object. Five rounds with one test person each are sufficient to identify about 85% of all usability problems of the test object (Nielsen, 2000). Usability tests with fewer test subjects are more likely to produce incomplete results, while usability tests with considerably more test subjects usually only lead to duplications of observations. The amount of work involved is therefore manageable compared to the effects of poor usability.

The right planning

At the beginning of planning a usability test, the aim of the investigation should first be clarified. The most important questions are:

• Which test object is to be examined and with which objective?
• Which user groups are relevant for the test?

Once these questions have been clarified, a decision is made on a suitable test setting. The quickest way is hallway or guerrilla testing. Here, people who are nearby are questioned spontaneously and briefly, usually employees from one's own company or people on the street. The classic usability test in the laboratory requires considerably more preparation. The laboratory here refers to a room with a prepared workplace. In this form, test persons are sought in advance and appointments are made for the usability test. During the test, the test person and the interviewer sit together at the prepared workstation while observers watch through a one-way mirror or via livestream. This form of usability test can also be conducted remotely, e.g. if the test persons are scattered regionally. In this case, the interviewer and the test person meet in a web conference and the processing of the use cases takes place via screen sharing. Many 3rd party providers of remote usability tests have recently emerged. However, since there is no interviewer present who can ask more detailed questions in case of problems, this form of usability testing is only recommended to a limited extent.

The exact procedure of the planned usability test is recorded in an interview guideline. Such a guideline can also be also helpful in ensuring all project participants develop the same expectations of the test and its contents in advance. In the interview guideline, all questions and tasks are recorded and the time schedule is determined. During the test, the interviewer follows the guidelines so as not to forget anything.

Conducting the usability test

Once the appointments have been made, the interview guidelines have been agreed upon and the technology has been prepared, the interview can begin. Since usability tests have only been possible remotely in recent times and this form will probably remain popular in the future, we would like to use this test setting for our best practice.

At the beginning of the agreed appointment, the interviewer and the test person meet in Microsoft Teams, Zoom or any other suitable tool for web conferencing. The interviewer starts recording the screen and sound with the consent of the test person. The recorded video is particularly helpful for reviewing critical points later during the evaluation. In addition, it is also possible to cut together a highlight video from the individual videos of the usability tests. This provides all project participants with a quick overview of the results. Such videos are especially effective if eye or mouse tracking is used during the usability test. For usability tests that are conducted remotely, however, these test methods can only be realised with considerable additional effort.
Once the recording has started, the interviewer and the test person begin with the usability test. Usually, a preliminary interview takes place first. This serves to obtain important information about the typical context of use and is also a kind of warm-up for the test person.
After the preliminary interview, the processing of the use cases begins. The interviewer presents tasks that the test person has to complete without help. It is essential that the test person thinks out loud and speaks their thoughts without hesitation. In this way, ambiguities and potential for improvement can be found more easily. It is also important that the test person interacts as naturally as possible with the test object – just as they would in the context of use. Meanwhile, the interviewer asks questions from the guideline that fit the context. Spontaneous questions from the interviewer are also acceptable and often very useful. These should be simple, closed, not hypothetical and not suggestive, so as not to influence the test person.
Once the processing of the use cases is completed, a follow-up interview usually takes place. The interviewer asks questions about the previous interaction with the prototype. The aim is to once again address any operating hurdles and to ask about the test person's overall impression. The follow-up interview is the conclusion of the usability test with this test person. The interviewer now thanks the test person for their help, ends the recording of the test session and says goodbye. This procedure is repeated until all test persons have participated. The usability test is now completed and the evaluation of the results can begin.

Evaluate observations

Observation of a usability test is a prerequisite for its evaluation. Usually, project participants or other UX team members assume this role. This can take place directly during the tests via a livestream or on the basis of a recording of the conversation.

The observations should be written down on sticky notes. It is important to assign a colour to each test person and to write an extra sticky note for each observation. It is irrelevant whether the sticky notes are physical or virtual. In a remote setting, we recommend using a digital collaborative tool such as Miro or Google Docs. There, all observers can easily collect digital sticky notes on a virtual board. Alternatively, if the observation does not take place remotely but in the office, sticky notes can be written on and attached to the walls.
Once the sticky notes have been collected in one place, they can be sorted and clustered. We recommend firstly clustering sticky notes with the same or similar observations. Duplicate sticky notes (same observation, same colour) can be safely sorted out. By clustering the sticky notes, it is possible to identify those places where problems have occurred repeatedly and also to identify problem patterns and causes. The colours quickly reveal how many test persons are affected by a certain problem. This shows which interactions lead to problems more often and which observations are rather isolated incidents.

When the clustering is complete, a list of all observations can be made from the sorted sticky notes. The individual problems should then be prioritised in terms of the number of occurrences as well as the severity of the subject's limitation. Afterwards, initial solutions can be added to the list for the respective problems. Thus, at the end of the usability test, an overview with all problems, their severity and possible solutions will have been created. This list can then be used to decide which of the problems shown should be solved and which approaches exist to solve them.

Properly conducted usability tests offer a comparatively easy way to identify the majority of usability problems that occur. Negative user experiences are thus avoided and the attractiveness of the product is considerably increased. Good usability has become indispensable for contemporary products and contributes significantly to the reputation of a company through positive user experiences. It paves the way to a barrier-free future.

Sources:

Nielsen, Jakob (2000): Why You Only Need to Test with 5 Users, in: www.nngroup.com

[online, retrieved 14.03.2022]