A deeper Refresher on A/B Testing

This article might clear all your doubts about A/B Testing. Also it will help you to learn it step by step with suitable example. It’s time to ramp our skills up a notch.

What is A/B Testing? A/B Testing is a method of testing one change on a page to see how it performs compared to the original. The change can be anything change the colour of the button or font of the text. A/B testing, also known as split testing, is an experiment in which you split your audience to test the smallest change of the newly developed feature and determine which performs better over a specific period of time. And then observe the metrics of both situations.

This article might clear all your doubts about A/B Testing. Also it will help you to learn it step by step with suitable example. It’s time to ramp our skills up a notch. Software testing might helpful to detect the bug in earlier level of development.

What is A/B Testing?

A/B Testing is a method of testing one change on a page to see how it performs compared to the original. The change can be anything change the colour of the button or font of the text. A/B testing, also known as split testing, is an experiment in which you split your audience to test the smallest change of the newly developed feature and determine which performs better over a specific period of time. And then observe the metrics of both situations.

How does A/B Testing work?

The first and foremost step of the test requires an understanding of your current metrics. Search for areas that need improvement and target one of those at a time. The targets may be time spent on your site, clicks on a button, sales or e-mail signups.

After that, we have to determine what needs to be altered and tested. To evaluate how effective a change is, you have to separate one variable and measure its performance otherwise, you won’t be sure which one was responsible for changes in performance. If more than one change is done at one time tracing the effect of that variable on the performance will be impossible to find out.

Now we’ll have to find out the alternative to meet our expected target which means we have to formulate the hypothesis.

Then the process of splitting the audience into two equal groups takes place. Make two copies of your site, and apply the change to only one of them. The one who can see the site with changes deployed is known as Challenger and the one who can see the site without the changes deployed is known as Control. The sample size should be taken at random.

Now we’ve to run the test for a sufficient amount of time. Changes don’t take place overnight so be patient and give sufficient time to the test. After that, we’ve to analyse the metrics of both sides viz. Challenger Side and Control side. Matrices are the real measure to see whether the deployed feature is helping us to reach our target or not. Metrics is nothing but the analysis of the behaviour of users toward the website.

If the metrics of the Challenger side is in increasing order that suggests that our deployed feature is behaving well and now we can deploy it for all of our users if not then deploying this feature will be a disaster and the website will lose its users.

Let’s take an example of juice-shop.herokuapp.com/#

For an instance, we want to increase the clicks on OWASP Juice Shop Button. That’s the ultimate goal of our test for that the hypothesis we have taken is that we’ll change the colour of the button from white to Red and see the matrices of both parties.

Now after forming the hypothesis we have to make two copies of our website in which the first copy will be the actual website without the changes embedded into it often called Version “A” and the second copy will be the website with the changes embedded into it often called as Version “B”. In our case, the change is the different colours of the OWASP Juice Button.

The random sample size of users is taken and we have to part them into two equal groups and select a particular time at which this test is going to be conducted. Suppose we’ve taken the runtime for this experiment as 30 days so for the next 30 days the Version “A” users will see the actual website(the button as white) and Version “B” users will see the website with changes(the button as red).

How will we find out that we should deploy the change on our website? How can we make sure that the given change is not affecting our click-through rate of the button? For that, we’ll study the statistics of both the matrices after 30 days and analyse if we can deploy the subsequent change in our website or not. If the metrics of Version “B” is showing that by changing the colour of the button more users are getting attracted towards our website and ultimately increasing the click-through rate of a button we can apply this feature now for all of our users and if the metrics of Version “B” is showing that by changing the colour of button website is losing it’s original users too then after knowing this fact applying this feature in our website will be the biggest mistake. Let’s contact us and make a more discuss about how A/B testing helps your businesses.

After that, we have to determine what needs to be altered and tested. To evaluate how effective a change is, you have to separate one variable and measure its performance otherwise, you won’t be sure which one was responsible for changes in performance. If more than one change is done at one time tracing the effect of that variable on the performance will be impossible to find out.

Now we’ll have to find out the alternative to meet our expected target which means we have to formulate the hypothesis.

Then the process of splitting the audience into two equal groups takes place. Make two copies of your site, and apply the change to only one of them. The one who can see the site with changes deployed is known as Challenger and the one who can see the site without the changes deployed is known as Control. The sample size should be taken at random.

Now we’ve to run the test for a sufficient amount of time. Changes don’t take place overnight so be patient and give sufficient time to the test. After that, we’ve to analyse the metrics of both sides viz. Challenger Side and Control side. Matrices are the real measure to see whether the deployed feature is helping us to reach our target or not. Metrics is nothing but the analysis of the behaviour of users toward the website.

If the metrics of the Challenger side is in increasing order that suggests that our deployed feature is behaving well and now we can deploy it for all of our users if not then deploying this feature will be a disaster and the website will lose its users.

Let’s take an example of https://juice-shop.herokuapp.com/#/

For an instance, we want to increase the clicks on OWASP Juice Shop Button. That’s the ultimate goal of our test for that the hypothesis we have taken is that we’ll change the colour of the button from white to Red and see the matrices of both parties.

Now after forming the hypothesis we have to make two copies of our website in which the first copy will be the actual website without the changes embedded into it often called Version “A” and the second copy will be the website with the changes embedded into it often called as Version “B”. In our case, the change is the different colours of the OWASP Juice Button.

The random sample size of users is taken and we have to part them into two equal groups and select a particular time at which this test is going to be conducted. Suppose we’ve taken the runtime for this experiment as 30 days so for the next 30 days the Version “A” users will see the actual website(the button as white) and Version “B” users will see the website with changes(the button as red).

How will we find out that we should deploy the change on our website? How can we make sure that the given change is not affecting our click-through rate of the button? For that, we’ll study the statistics of both the matrices after 30 days and analyse if we can deploy the subsequent change in our website or not. If the metrics of Version “B” is showing that by changing the colour of the button more users are getting attracted towards our website and ultimately increasing the click-through rate of a button we can apply this feature now for all of our users and if the metrics of Version “B” is showing that by changing the colour of button website is losing it’s original users too then after knowing this fact applying this feature in our website will be the biggest mistake. Let’s contact us and make a more discuss about how A/B testing helps your businesses.