To be or not to be? That is a question and a constantly repeating situation. People often have to face some choices which are hard to make. Designers’ job is also full of situations when two solutions seem right but you need to make a choice and not stumble. When it happens, a creative team runs tests for both to find out which solution is the best. One of the popular methods is called A/B testing. The article describes the facets of A/B testing methodology and shows how designers apply it for UX improvement.
What is A/B testing?
A/B testing, also known as split testing, is a method of comparing two versions of a digital product in order to find out which one performs better. A creative team divides users into two groups and each of them is shown different variants. One half sees the A version, the other – B. Such an approach helps to determine a more profitable solution.
The differences between A and B options can vary from the smallest to the big one. For example, it can be a color of a small element or a different placement of all the components. The effectiveness can be measured by different criteria such as page views, clicks, number of subscriptions, or sale leads. The choice of criteria for measurement depends on the goals a company or a creative team has established. In some cases, it may be good to combine these metrics to achieve deeper insights into all aspects.
How can designers use A/B testing?
UI and UX design for a product is always created to gain certain goals. Designers thoroughly choose all UI elements as well as plan clear and pleasant interaction and navigation system for effective user experience. However, what once worked well, may not have the same effect after a while. Business goals constantly change during the time a product functions, so the design requires some improvements.
A/B testing is an approach helping to make design changes carefully so that users wouldn’t feel inconvenient. Designers can collect the needed data and metrics while people continue to use an app or website.
So, what do designers get from A/B testing? First of all, it’s a good way to learn the behavior patterns and peculiarities of the target audience. By testing different versions of UI, designers can see what kind of influence each solution has on users and which one performs more effectively. Moreover, A/B testing assists to remove bad design decisions if there were ones and replace them with something more profitable.
In addition, A/B testing can reduce conflicts between designers and clients. It often happens that the opinions of both sides differ and no one wants to step aside. To solve such a kind of conflict, designers can run A/B testing for two variants which a creative team and a customer see as the best solution.
As we said above, differences between versions A and B can be either small or big. Here is a list of elements that designers often choose for A/B testing.
- CTA buttons (their placement, size, color, or copy);
- Headings and subheaders;
- Images on landing pages;
- Presentation of clients’ form on websites;
- Copy (length, placement, and content);
- Offer display;
- Videos (presence or absence).
A/B testing is not a one-time operation. Designers can test different elements one by one and then combine gathered information to check which of their solutions is more user-friendly and goal-oriented.
A/B testing process
If you never tried A/B testing before, it’s never too late to experiment. Besides, experimenting with new approaches can help to open new opportunities. Moreover, A/B testing is not a complicated method. It can be accomplished easily if you take the following steps.
Step 1. Gather data and rates.
The first aim of A/B testing is optimization. It can be revenue optimization, user experience improvements or just a product upgrade as a whole. That is why the first step before you run A/B testing should be data collecting. Analytics results can help to understand what aspect needs optimization the most. Pages with low conversion rates or CTA buttons with a small number of clicks can be the first targets.
Step 2. Set the goals.
To make the improvements work right, designers need to determine what they expect to gain from them. For example, it can be more subscribers for a blog, more time spent on a website, or more clicks on CTA buttons. Setting the goals you can track the effectiveness of your decisions.
Step 3. Build a hypothesis.
This step is needed so that designers could use the gathered information for future improvements. After the goals are set, you need to think of the reasons why the new solutions will work better. Research on the target audience helps to generate some hypotheses about what can improve the current version. It will guide designers when the testing ends and will give deep insights into the target audience preferences.
Step 4. Create A and B versions.
When designers have decided what changes they want to implement, it’s time to bring them to life. There are many free and paid tools helping to divide the users’ stream into two groups. It may be a good idea to QA the testing at the beginning to make sure everything goes as it should be.
Step 5. Go testing.
It is the most pleasant part of A/B testing for a creative team because it’s time for users to work. People use an application or visit a website and everything they do is measured and transformed into useful data.
Step 6. Results analysis.
The experiment lasts for a certain time and when it’s complete designers work on results. All the data and metrics from the two versions are gathered and compared. Based on the results designers decide which variant performed better and is capable of accomplishing the goals that have been set at the start.
Benefits and limitations of A/B testing
There is no perfect methodology that would suit any situation. However, each approach is unique and brings certain advantages which others can’t. Let’s see what A/B testing can offer to designers and what are its limitations.
Benefits
It measures the actual user behavior. People use a product without knowing they are a part of the experiment now. That’s why they act naturally which helps to identify the model of their behavior quite accurately.
It is a cheap method. A/B testing doesn’t require applying expensive tools and hiring people who’d like to test a product. All you need is two versions of the experiment and software which would randomly divide users into groups.
Quality content. A/B testing is a good way to find out what content is valuable and works best for a product.
User-centered design. A/B testing is absolutely oriented to users’ needs and their behavior patterns. After the experiment designers get the data helping to make a product even more user-friendly.
Fast and easy analysis. A/B testing is usually focused on a few certain metrics. So when the results are delivered, it’s not that hard to determine which version performed better.
Everything can be tested. Not all the approaches allow testing really small things such as buttons color, fonts or headings placement. However, even such little elements can have a great impact on users’ behavior and their perception of the design.
Reduced risks. Some mistakes may cost a lot if they are made at the important stages. A/B testing can reduce the risks of losing money because of the wrong design decisions. By testing several versions of design at the early stages, designers make sure UI elements work well.
Limitations
A/B testing works only for fully implemented design. You can’t test just CTA buttons in UI without copy content or any other component because the results won’t be the same. Everything should be as a fully accomplished product ready for usage.
It doesn’t tell you why. A/B testing is an effective method of gathering certain data and metrics showing what works well or not. But if you are looking for the answers to why it is so, you need to do user research that will go deeper into the peculiarities of the target audience.
It can’t tell you what is wrong. Continuing the point above, we need to add that A/B testing isn’t meant to identify the problems in design. It only can show which improvements work effectively.
If the design goes into real usage without proper testing, there is a high risk of its low effectiveness. That’s why designers are recommended to make testing an essential part of the UX process. The choice of approach is a matter of circumstances and designers’ preferences. Stay tuned!
Recommended reading
A/B Testing, Usability Engineering, Radical Innovation: What Pays Best?
Putting A/B Testing in Its Place
Precious Errors: Testing iOS Mobile Applications
