In which cases you don’t have to do A/B tests
05.09.2016

A/B testing is considered almost the main tool for optimizing the conversion. In fact, A/B tests are not always necessary. In this article you will know in which situations it is better not to do split testing and focus efforts in other areas.

A/B tests are not for all

As you may know, split testing is useful thing. However, for some companies it’s too early to carry them out. Roughly speaking, if you have fewer than 100 transactions (purchases, subscriptions, leads, etc.) in a month – forget about A/B tests. And there’s nothing offensive in such situation: hundreds of small businesses and startups are simply too small for A/B testing.

Of course, you may disagree with this. Suppose, you all the same conducted testing and got the result of 240 against 260 conversions. The superiority of one option is 8%. It would seem that everything is clear – you have to implement the winning version. But here’s a problem – the received data is not enough to make a choice, because you still have too little traffic to reach statistical significance (a quantitative index, which shows that the test result can be considered reliable).

Expenses for testing

Also, thinking about split testing, we can’t forget about the cost of optimization. We are not talking about tools – there are many free programs. Think about the cost of time spent on:

  • thinking, what should be tested;
  • development of a test option, especially if you decided to test something unusual;
  • launching the test using CSS and javascript/jquery;
  • QA (Quality Assurance).

Now let us return to that 8%. You have implemented your winning option and received 140 leads instead of 130. Can you talk about the increasing of ROI (return on investment) in this case? Hardly.

In determining the sample size, keep ROI in mind and convert percentage difference of conversions into money. By the way, there are plenty of free tools that will help determine the sample size based on the desired conversion. For example, this one.

Are micro‐conversions important?

You can say that micro‐conversions are also important. But is there a connection between micro‐conversions and the growth of profit? You can have huge traffic and thousands of visitors, but at the same time you will have very small amount of requests and orders, and the shopping carts will be empty.

If you want to achieve success in business, do not waste time on the micro‐things – micro‐studies, micro‐tests and micro‐conversions. Think about more important stuff and engage in something that will bring real profit. For example, work on your product quality. Or on the usability of your website. Or study unit‐economy, in other words, the calculation of the profit or loss per customer, to see whether there is a place to grow and attract more customers. Or launch a blog to gather the audience that in the future may become your customers.

Testing is not the main optimization tool

Split testing is a great way to confirm the hypothesis. And even if you don’t have enough data to obtain statistically reliable results, you can conduct heuristic analysis – evaluate the usability and other qualitative studies. For example:

  • communicate with clients;
  • conduct user tests;
  • conduct a poll;
  • explore the map of clicks and the behavior of visitors of your website with the help of Google Analytics.

Time is much more valuable resource than many think. It’s your decision, how to spend it. Our advice: as long as your business only grows, leave A/B tests alone and focus on really useful things.

If you are not sure what marketing activities you need to carry out to increase the conversion of your resource – you can write to SoftwareGarage. We will gladly provide you with free consultation and prompt how we can be useful for you.

Also, subscribe to receive latest news and publications!

Newsletter