Understanding AB Testing Analytics In NotifyVisitors

  • Created : Apr, 20, 2021
  • Last Updated : Jan, 18, 2022

What is AB Testing

A/B testing is an effective method of optimizing web pages and improvising conversion rates. It is an effective practice of comparing two or more versions of a webpage to determine which one is performing better. The variants of the a/b testing campaign are then displayed to the users and the statistical analysis is used afterward to decide which variation is performing better for a specific conversion goal.

You can create an AB Test and edit the elements of a web page to optimize conversion rates with the help of an effective software tool.

NotifyVisitors enables you to create an AB test and further allows you to analyze the insights of the campaign such as the goal conversions, total visitors on your ab testing campaigns, etc. All in all, you can have a campaign overview as well as you can check out the metrics of each variation and determine whether you have achieved the specific goals or not.

Read this article to know in-depth information about AB testing campaign analytics in NotifyVisitors.

Undestanding AB Testing Analytics in NotifyVisitors :

AB testing analytics allow you to determine the optimized version of your campaigns. You can compare and analyze which version is performing better. Consequently, you send the best-optimized version to the audience to procure the determined goals such as better revenues.

Below are the steps to view the ab testing analytics in notifyvisitors dashboard :

  • Navigate to the NotifyVisitors dashboard> Configure > AB test.

configure ab test

  • Click on the AB test in the left side navigation bar and view all the campaigns you have created. Go to the ACTION category to view the analytics for any campaign.

ab testing analytics

  • Click on the campaign whose analytics you want to view and evaluate.

NotifyVisitors ab testing analytics is bifurcated into three categories :

  1. Summary
  2. Daily report
  3. Heatmaps

Let us take a look at each of the categories to know the ANALYTICS better.

1) SUMMARY :

You can evaluate the overview of the campaign under this category. Know the total visitors, conversion, and conversion rates on your campaign as well as on each variation. Simultaneously you can view the insights about the goals you have determined for your campaigns. i.e. for the control as well the variations. Consequently, analyze the leading variation.

Define the variables of the SUMMARY :

First of all, define the time frame of the campaign for which you want to view the analytics.

  • DATE FILTER: Select the start date and the end date for the campaign.
  • SEGMENT: Select the device from the drop-down menu, for which you want to track the campaign analytics.
  • BASELINE: Select the control or any variation whose analytics you want to view.

ab testing filters

Click on the SUBMIT button.

How to define the variables of CAMPAIGN OVERVIEW :

  • Variation: View the variations you created, i.e. the control version and the other variants. Here we have created two variations :
  1. Desktop_ open_ an_ account.
  2. Without shadow
  • Users: View the total number of visitors on the control as well as on the variants. These are the unique users who are viewing the control or any variants. That means, if a user is loading the page multiple times, he will be counted as one. Also, you can view the unique users’ percentage.

Note: Divide the number of total visits by the number of unique visitors to calculate the percentage of unique users on control as well as any variant.

  • Goal conversions:  Refer to the below image to view the total number of visitors on control as well as on your variations individually against the goal conversions.

For example, The “Right submit” Goal conversion has (12) total number of users for variation 1. That means, 12 users were converted on this predetermined goal (Right submit) for variation 1. The same applies to the rest of the goal conversions.

Goal conversions

How to define the variables of GOAL CONVERSION overview :

Here you can view the total number of visitors, conversions, conversion rate, and the improvement possibility for the LEADING VARIATION as well as for the least performing ones.

For example - We had defined 7 goal conversions for the campaign we created. We defined the goal conversions as Right Submit, Top Submit, top (Open Account), Right (Open Account), form click, live chat buttons, and main lead form submit.

Goal conversion analytics

  • Visitors: The total number of visitors (the count of each visitor who loaded the page multiple times) for your goal conversions for the leading variation as well as the least performing ones.
  • Conversions: The number of visitors who achieved the goal conversion.
  • Conversion rates: Determine the conversion rates by simply taking the number of conversions and dividing that by the number of total ad interactions that can be tracked to a conversion during the same time period.
  • Improvement: This is the median improvement you can expect over the baseline if you implement the variation. The 'best case' and 'worst case' values represent the 99% credible interval where improvement is likely to be contained.

Note: The above definitions apply for evaluating the metrics of other goal conversions as well.

Point to Remember :

You can also view the campaign information to have a better understanding of the campaigns’ status, the age of the campaign( the time frame since its running, the segment ( the segmented visitors whom you showed the campaign), Page URL (on which your campaign is running). View the total number of visitors, total variations created, and the number of goal conversions.

campaign information

2) DAILY REPORT :

You can view here the following key metrics for your predetermined goal conversions for control as well as the variations individually : 

                                                daily report         

  • Users: View the total number of visitors on the control as well as on the variants. These are the unique users who are viewing the control or any variants. That means, if a user is loading the page multiple times, he will be counted as one. Also, you can view the unique users’ percentages.
  • Conversion rates: Determine the conversion rates by simply taking the number of conversions and dividing that by the number of total ad interactions that can be tracked to a conversion during the same time period.
  • Expected conversion rate: This is the median conversion rate you can expect from the variation. The 'best case' and 'worst case' conversion rates represent the 99% credible interval where the conversion rate is likely to be contained
  • Confidence interval: The confidence interval measures uncertainty around improvement. Stats Engine provides a range of values where the conversion rate for a particular experience actually lies. It starts out wide, as Stats Engine collects more data, the interval narrows to show that certainty is increasing.
  • Probability to Beat control: Probability to Beat Control (CTBC) determines the statistical significance and shows the probability that the variation page will perform better than the control version.
  • Conversions/visitors: The ratio of the number of conversions to the total number of visitors.
  • Absolute potential loss: It depicts the percentage possibility for the improvement of conversion rates. If your Absolute Potential loss is 2% and the expected conversion rate is 10%, it means you still have a chance to improve this conversion rate and increase it to 12%.

Note: Here you can select the time frame for the campaign, choose the device, the baseline, and the goal conversion for which you want to view the analytics. That means, here you have the power to view the analytics for any goal conversion individually by simply selecting them from the drop-down menu.

How to define the variables of probability density functions :

  •   Probability density: View the statistical significance for the likely or possible outcomes for the ab testing results out of the current results. Evaluate the success possibilities for the control version or other variations.

probability density functions

  • Chance of being the best:  It determines the probability of a variation to perform better than all other variations including control.

              chance to being best

3) HEATMAPS :

View here the AB testing heatmap analytics for the control as well for the variations. The two powerful tools can help you optimize the conversion rates and see surprising results. With Heatmaps, you can analyze why a variation did not perform better.

Below are the metrics you can determine under this category :          ab testing metrics

  • Users:  View here the total number of visitors on your control version or variants.
  • Page views:  Determine the total number of page views. The repeated views of a single page are also counted.
  • Clicks: See here the total number of clicks on the goal conversions for your control as well as the variations.
  • Avg. time spent:  Evaluate the average time spent on the website by taking the total no. of variation views against the total amount of time (in minutes) spent on it.
  • Engagement rates: Determine the engagement rates by taking the total time spent by the users against the total number of page views.
  • View heatmap: View here the scrolls, hovers, and clicks on the ab testing heatmaps on the control version as well on the variations individually.

Scrolls :

Hovers :

Clicks - 

 

  

Note: Define the time frame, the segment, and the baseline for the campaign for which you want to evaluate the analytics and determine the best-performing version.

Conclusion

Keep an eye on the ab testing analytics to view each and every aspect of your campaigns whether it is in terms of the total users, conversions, or simply the time spent on each variation. Every aspect matters in determining the best performing version to elevate your business growth.