Take your A/B testing to the next level

One of the basic but most effective forms of experimentation while building a product is A/B testing.

The concept is to create two alternative versions of some element of your product, and then see how well they perform according to some metric such as conversion or retention. Once the experiment concludes, you make a final decision on the variation to implement.

This can refer to text, the onboarding flow of your product, or even the color of a button.

It sounds very trivial, however those simple changes might significantly improve your conversions and what follows, revenue.

If, for example, you own a webshop and you choose the checkout flow where an additional 5% of users are dropping out, or the color of the call to action button converts 7% less, this can cost you a good portion of potential revenue.

How can you perform A/B testing?

Usually, the first product you would reach out to is Optimizely. It’s said to be the most known tool for experimentation.

However, if you are a small or medium sized company, you will probably not be able to use it on your product. If you approach them as a non-enterprise client, you will most probably be advised to try Google Experiments first and when you grow, come back to them.

Furthermore, their platform cannot experiment directly on the metrics that are actually crucial to any website or app: retention, virality, engagement and others.

This means that although you can A/B test your conversion rate, you have no way to know if your better converting alternative will lead to less users coming back over time.

Of course, with some custom and complex code implementation you could still measure retention on other platforms, but this involves changing and re-publishing your app/website on every change in your goal metric.

This is why, while building our A/B testing Insight for Stormly, we made it possible to quickly iterate A/B tests and use ad-hoc conversion, retention, session or viral measures as the goal of your experiments. You can even experiment on the portion of users that had more than $100 in revenue for example, without the need for any custom code.

Our A/B testing Insight will help you:

• Show which A/B version is the winning one and with how much significance.
• Experiment on event conversion, retention, sessions, revenue, viral coefficient and much more.
• Automatically determine sample size and how many days your experiment should run.
• Show conversion lift (relative conversion improvement), and help decide if the economic impact is big enough to implement the new variation in your website or app.

Nowadays the speed at which you can launch new experiments is crucial for many online businesses, with Stormly you can stay ahead of the game.

Furthermore, according to our own research, conversion is usually not the best proxy for the bottom line. More often it’s retention and virality you should be looking at.

To start running A/B tests with Stormly, all you have to do is connect your data to the platform and pick A/B Testing Insight from our Marketplace. See the Set-Up Data option in your projects page for more information.