In our daily lives we often become agitated if something is difficult to use. In fact, unless there are no alternatives, we tend to start looking for other solutions that work better. On top of this, if a product is designed in a way that does not make sense to us, we will likely complain about it to others.
However, something interesting occurs when we make the mental move from user to creator, particularly in the area of software design and web development. There’s a strong tendency to forget about or undervalue the experiences of those who actually use the product. Our focus becomes on how to solve a problem; we spend a lot of energy just to make something work. We focus on the inner functionalities of the sites and software we create. This is something that also occurs for those working on front-end applications. Front-end designers often have something of the artist in them. The look and feel is something that engross us, but somehow there’s a fear to let someone else control how we design something, lest someone else try and manage our artistic expression.
We often do try our best to make user-interfaces that work well and are intuitive, however we are often too close to what we create to recognize that something that makes sense to us does not necessarily translate to others. When a user has a hard time understanding how to use a product, or if they find it clunky or awkward, this is not their fault. We ignore user reaction at our own peril: bad user experiences can mean death for your site/software product, etc.
What is Usability anyway?
A professional approach to usability is not simply thinking about ourselves using a site that we have built. It involves integrating a testing process throughout the development life-cycle of any project. While we may do plenty of functionality and unit testing, with usability, we are testing our software from the perspective of an end-user who has no relationship to the creation of the product.
Usability requires:
- An understanding of the importance of user perspective. Who are the people using the software or the website?
- An assumption that we know nothing. We can never assume we know what others are thinking. Often what seems easy or obvious to us may not be for others.
- A continuous integrated testing of aspects of a project throughout its life cycle.
- Working usability into all stages of the design/build process. It is easy to think we have addressed a problem simply by changing something, but without continued testing, we have no way of knowing whether our changes had any impact, or if so, that they were positive for the user.
Types of Usability Studies
There are a number of different methods for assessing the usability of a site or product. Below is an explanation of some of the most commonly used ones. Each one has its value, and can be done independent of the others. Performing at least one of these forms of testing is better than none at all, however, a combination is recommended, if not all of them. Some have merits that are more beneficial in some stages of a study than others.
Please note that this is far from a comprehensive list of the tools used in usability analysis, but it should provide an introduction.
Hallway Testing
Hallway testing is pretty much what it sounds like. It involves setting up a testing environment in a hallway or corridor that receives a lot of traffic and recruiting passers-by to spend a few minutes of their time to run through a prototype website or software application. This can occur within a target organization, or it can consist of something as simple as pulling a co-worker (someone who is not part of a project) in to look at a project during any stage of development.
It is one of the most informal methods of usability testing. While it may not be as good as extensive usability testing, it can provide some useful results.
Pros:
They are easy to set up, and can provide some valuable feedback, particularly if you don’t have the time or resources to conduct larger usability studies.
Cons:
The informality and public nature of these can make the results unreliable. The fact that they lack structure can also increase the chances that important flaws may be missed.
Focus Groups
During the Discovery-phase of software/website development process, setting up focus groups to discuss and discover what end-users will be looking at in a product is a key part of usability testing.
Usually these sessions are made up of potential stakeholders in the project, particularly the potential end-users. Focus groups can serve a number of functions.
- Identify the needs and wants of end-users.
- Brainstorming sessions to help create initial list of topics, ideas that can go into a project.
During these sessions, users are generally asked to go into some detail about how they actually perform a task that the development intends to assist. The loose, non-structured format of these is designed to get end-users to discuss features that they are frustrated with – in existing software, as well as features that they wish or want to exist. They are informal, but are typically run by a moderator to help the group maintain focus.
Pros
They can be extremely useful during the Discovery-phase of a website or software project, as end-users can describe what it is they want in a product, as well as problems that they may see in existing competitive products.
They can be helpful at providing a loose set of topics which can be used for next-step testing, such as for card-sorting exercises. They are good with creating a number of topics which can be later used for more integrated testing. If these sessions are guided well by an expert observer, valuable information can be obtained.
Cons
They are not, particularly good as a standalone segment in usability testing. While they can offer some value, it is not recommended to use these alone. While these are very useful at gauging the way that users say that they use a tool, evidence has shown that it does not necessarily reflect the way that they actually use them. What people say and do are wholly different things. People also have a tendency to be influenced by “group-think.” They respond to what someone else says, and not necessarily what they think or do on their own. Using focus groups in later stages of development is not recommended as they are unlikely to provide useful information about a product in progress.
Observation/Usability Testing
This is probably the most well-known form of usability testing, and one of the most well-known and effective methods of Usability testing is the observation method; in fact, these terms have essentially become synonymous. These tests are highly structured, with preset questionnaires, and individual users are observed, typically within a laboratory environment, navigating a site or using software and their behavior and actions are recorded. The goal of user-observation is that instead of relying on what people say they like or don’t like about a site or software, we can observe how they actually use it in simulated real-world situations.
There are a number of ways of doing observation, and these can vary largely due to budget
These include:
- Blind Testing: this involves the testers watching from a hidden location (such as through a one-way mirror). There can be a tester in the room providing instructions so that they can respond to questions, or the user can be given a written list of tasks.
- Eye Tracking: These are devices that will identify where a user’s eyes move on a web page, and can identify whether or not the appropriate items are being viewed. While highly effective in identifying how users look at a website, it can be prohibitively expensive for some agencies.
- Over-the shoulder observation: the observer stands over the shoulder of the participant and records a user’s reactions to instructions. This can be a more interactive approach.
- Secondary monitor: Useful in lower-budgeted environments this involves attaching a second monitor to the participants computer, and the mouse movements can be viewed from across a longer table. While not fully non-intrusive, it has the advantage of removing some of the nervousness associated with the “over-the-shoulder” approach.
Within these sessions there should be at least two or more observers, one to provide questions, and the others to record observations, who takes notes, while paying attention to difficulties a user has, where they click, etc. The information gleaned during these sessions is then typically applied to a rubric with some scoring given to individual tasks rating the ease of use.
Pros
Within a well-controlled environment, results are easily compared. They are highly structured, so with a large enough n users interactions with websites can be well-monitored and measured.
Cons
Laboratory environments are somewhat poor at mimicking the real world. The largest difficulties in observation is the fact that many users react differently when they sense they are being observed than when they are on their own. It is important to be aware that users can become nervous when being observed which can interfere with the accuracy of the results, so the over-the-shoulder approach is not recommended.
Card Sorting
A useful approach after identifying the basic areas that goals of the software are meant to address, is to use a card sorting exercise. This segment is of most use to the designers when they seek to create user interfaces for users to be able to self-navigate a website or piece of software, and to organize data in ways that users think.
Card sorting testing involves creation of simple index cards, with a listing of the major and minor features of a website or piece of software, and presented to user in an unsorted stack.
Users are then requested to organize or order the cards into groupings that make sense to them. They are not provided top-level headings; this is left up to the user. Users are also provided a set of unmarked cards where they can write down any sections they feel are missing.
The value of this exercise is not only does it help the designers organize information into sections, it can provide perspectives of how users actually think and can help designers adapt their designs to this information. It also helps identify pieces that may have been overlooked which can then be relayed to the developers to be created.
Card sorting:
- lets users organize information.
- enable the testers to discover unexpected arrangements of data.
- can help identify user-requested features.
Pros
Card-sorting can give a picture for how people tend to organize information for themselves. It can be extremely useful when structuring a site, or designing an interface for software. Also of value is the ability to identify areas which had not been previously considered within the site; it can lead to useful information regarding new areas which should be developed.
Cons
Card sorting is not generally oriented around tasks. While it can be useful in organizing information, it can result in users placing things where they make sense in a categorical sense, but not in a way that may be most efficient for completing an action. There can also be a lack of consistency in the results. You may find that the same user might arrange information differently in a different testing session.
A/B or Split Testing
A/B testing is primarily associated with websites and not with software testing
It is most often done as part of marketing campaigns where users are randomly given two different versions of a web page, and through the use of analytics on the site, results are measured. If one provides better results, then it becomes more permanent.
This is something that can be performed as an ongoing basis and can typically be used for under-performing sites or segments of sites. To be able to do this, requires a somewhat agile system, wherein pages can be generated quickly in short periods of time.
Ideally A/B tests are best performed over time, and can really only be measured with a meaningful n, t to help reduce the likelihood of false positives or negatives. To gain accurate measurements, it is extremely important to make sure that the sample groups are as similar as possible, and that there are no other factors interfering with the test, such as the time of day/year. For this reason it is best to run 2 or more tests simultaneously.
Pros
A/B tests are excellent for measuring actual user behavior. The data is in real-time, in real-world situations. Results are easily quantifiable, and actions tend to be very clear.
Cons
Split testing can be somewhat costly. Also given the fact that you are testing with a live site, you could accidentally cost actual business, and can turn off customers if the wrong approach is taken.
More Resources on Usability Analysis
As mentioned earlier, this is by no means a comprehensive list of forms of usability analysis. The space of this article is not enough to cover them all. More information can be found here:
A Note about Sample Sizes
While a larger n is generally considered better, this can vary widely depending on the nature of the target market. For instance if you have a very limited target market, such as a small subset of a group of professionals within a larger field, it may be difficult to gather a large enough sample size to meet ideal social scientific standards. However, inability to gain a large enough sample should not be considered a reason to not perform these tests.
Through years of analysis, usability expert Jakob Nielsen discovered that the rate of discovery of new issues in design were reduced significantly beyond a low number of users. Test subjects tended to identify the same problems, and the likelihood of finding unique problems decreased drastically beyond 5 test subjects.
He may have oversimplified a little, as his observations were made in earlier stages of the web (before 2.0) when sites were significantly less complicated, but the underlying point is that it’s worthwhile to test the impressions of a few people than none at all.
One point that should be stressed: it is important to make sure the people being used in the test are appropriate for the subject matter. Often there may be different aspects of the site or software being used by separate people, so ideally you would wish to gather a representative sample for each segment of use-population. If we define the n necessary to effectively test the software, and x is the number of groups, the number of tests should be x(n).
Stages of Usability Studies
Usability research is not something that happens at one point in the development process. The value is best realized by integrating into all stages of development including
Discovery-phase
In this stage is where you gather information about the market, understand what users want, and what sort of functionality they want, and to help identify the behavior of your target market. In other words, what sort of expertise does this group have? How do they normally gather information or perform the tasks that this software or website will help them accomplish?
Take a look at our post on the importance of discovery phase here.
Types of studies: Focus Groups, Contextual interviews, Task Analysis, Surveys
Specification stage/pre-design
During this phase you already have a relatively good idea of what is going into the product. Working with end-users here can help you understand how to organize it and to make sure you have all of the elements that you need prior to actual design or development.
Types of Studies: Card-sorting, Prototyping
Wireframe Testing
At this point some loose wire-frames have been created. The site hasn’t gone into full development, however there is some basic functionality that users can work with. Some usability analysis here can give some indication whether the design-plan is appropriate, and can also head off major problems in usability before the design and development begins.
Types of Studies: Usability/Observation Testing, Hallway Testing
Mid-design Testing
Much of the initial stages of the product has been built. It may not have all of the bells and whistles yet, but it is largely functional. Testing at this stage is extremely important as enough functionality exists to give users a genuine sense of how the site will look. If major problems are identified at this point, changes can be made to arrangements of content, and possible look and feel issues can be addressed.
Types of Studies: Usability/Observation Testing, Hallway Testing, Use Cases
Late Stage Design
At this point of the development, the product is nearly complete yet not yet released. If the site has progressed well and has adapted to information collected in earlier analysis, most of this should be a matter of making some last minute changes. However, as can occur, sometimes larger issues can be identified here, and some triage can occur regarding which issues should be handled immediately and which should be scheduled for next-version releases.
Types of Studies: Usability/Observation Testing, Hallway Testing, Expert Review, Contextual interviews,Contextual interviews.
Ongoing
Once a product has been released, information should regularly be collected regarding user impressions, and real-time activity to plan for future enhancements and to help identify bugs which should be prioritized. Real-time market activity can be measured as well as customer satisfaction
Types of Studies: A/B Testing, Surveys, Usability/Observation Testing
Conclusion
While the expense of usability testing is something that can be outside the scope of a project’s budget, there are ways to incorporate some of the principles into the development and design process. Ideally designers and developers should feel empowered to stop the development at any time if a problem arises. Within Agile project management frameworks, this can be possible, particularly within Lean/Kanban approaches.
One of the key factors of understanding usability is to respect that if one person has a problem with using something, it should be taken seriously, quite simply because many people do not say anything when they have a negative experience; they simply move on. If one person says something, there’s a high likelihood that there are many others who are thinking/experiencing it.
Even if testing is difficult to create or to get target market, it never hurts to at least do some hallway testing. Because it is quite common for people working on a project to become too close to it and not spot problems, quite simply because they know how it is supposed to work, it’s a good idea to do some testing of peers throughout a project; bringing in someone who is not directly part of the design, development, or project management teams. Beyond this, it is a very good idea to do some basic usability testing, even if limited. It is understandable that extensive testing can seem like a large expense, however there are some easier methods, particularly the secondary monitor approach.
Without at least some rudimentary testing you risk creating a product which does not meet the needs of the end users. If the users dislike what you’ve created, unless the product is crucial to their daily activities and they have no alternative, this is akin to failing. Without testing, you have no reliable way of knowing whether or not you have created a viable product.
Another important point is that we need to never treat a site/product as “done” even after it is built. To understand whether it is working well, it is necessary to track behavior, gauge user perception and change only in response to what is learned.
