Boost your App Revenue: Unleash the Power of A/B Testing with Apphud

Let's take a closer look at what the A/B testing tool represents in the Apphud service, how to use it correctly, and the advantages it offers.

Experiments in Apphud is a feature that includes A/B testing of paywalls.

With A/B tests in Apphud, you can gain a better understanding of your audience and their preferences (as each niche has its own), allowing you to offer products that have higher demand and earn more revenue.

Apphud Experiments provide the following capabilities:

• Run A/B tests on 2-5 variations of paywalls with different sets of products and prices.
• Modify the appearance of the paywalls.
• Define your target audience based on your specific needs by creating a custom audience.
• Analyze the results using a wide range of metrics such as views, trial/purchase conversions, sales, proceeds, ARPU, ARPPU, etc.
• Run multiple A/B tests simultaneously.

Conducting experiments (A/B tests) using Apphud has several important advantages:

     1.  No coding

You can modify paywalls from the Apphud dashboard without involving costly development resources or releasing the app on the App Store. Conducting experiments allows you to avoid wasting time and resources on implementing ineffective changes that could negatively impact the development of your mobile application. Here's how it works.

You create a separate container (variation) for each group of users, and the functionality of the paywalls includes:

• Set of products
• Custom JSON

By having custom JSON, you can test not only purchases but also the appearance of the paywalls.

Example 1: You can create two different onboarding experiences in your project and switch between them based on a field in the custom JSON.

Example 2: You can show or hide specific app features based on a field in the custom JSON.

json example:

{

“close_time” : 5.0,

“selected_index” : 1,

“onb_type” : “video”

}

where:

onb_type - one of the onboarding types.

selected_index - the preselected product index on the paywall.

close_time - the delay time before displaying the Paywall close button.

     2.  Risk Reduction

Experiments help mitigate risks associated not only with development efforts but also with implementing changes in the app's code, such as visual modifications to the paywall, which can negatively impact purchase conversions from paywall displays. In case of a negative outcome, you would need to make changes to the paywall again and release the updated app on the App Store. This process takes time during which you may lose your audience. Therefore, A/B tests provide a beneficial opportunity to assess the effectiveness of different paywall modifications on real users.

   3.  Optimization

Conducting A/B tests allows you to optimize your application by improving the user experience, and increasing purchase conversions, and other metrics.

   4.  Obtaining Objective Insights

Experiments provide you with objective data on the effectiveness of changes, enabling you to make decisions based on factual results rather than assumptions or intuition.

However, conducting experiments should be based on a sufficient number of users and data to ensure high metric sensitivity and obtain objective results.

There a plenty of factors you can modify and test in Apphud:

• Subscription price, 
• Availability of a free trial period, 
• Appearance and information on the paywall, 

and many more.

You can run a test with multiple variations of paywalls, such as 5, each featuring different products and prices. Alternatively, you can compare just two variations of the paywall, determine the winner, and then create a new experiment comparing the new variation against the previous winner.

You can test your hypothesis by using the Trial subscriptions graph in the Events section when segmented by Paywall. This will show the expected increase in subscriptions. Additionally, you can observe the results directly in the experiment using the CR Purchases metric (Conversion to Purchases).

Audience tips

Apphud provides extensive capabilities for working with audiences. You can use both the standard set of audiences and create custom ones, utilizing various parameters and combinations (location, specific event occurrence, paying/non-paying, current subscription status, and many others).

Here's an example of how you can create a custom audience.

If we are targeting traffic from contextual advertising for new users in the USA, we can create a custom audience called New USA users in the Users section and select it from the list of audiences.

To narrow down the audience, just create a custom audience called New USA users in the Users section with the following criteria: Country by IP = USA and User lifetime = New users. Once created, select this audience from the list of audiences for your experiment.

The audience section in Apphud Experiments

Apphud allows for the analysis of the following metrics during and at the end of the experiment:

• Views - a number of overall variant paywall views (every repetitive view is counted).
• Marked Users - a number of unique users, marked (related) to a particular paywall variant. May not view the paywall.
• Affected Users - a number of unique marked users have seen the paywall.
• Trials - a number of started trials.
• CR Trials - conversion from paywall view to trial start.
• CR Trial-Purchase - conversion from a trial to an in-app purchase.
• Purchases - a number of initial purchases (non-renewals) and trials are included.
• CR Purchases - conversion from paywall view to purchasing.
• Last Purchase - last in-app purchase date for the variant.
• Sales - the total amount billed to customers for purchasing in-app purchases from the paywall variant. Sales = Gross Revenue - Refunds.
• Proceeds - the estimated amount you receive on sales of subscriptions. It excludes refunds and Apple’s commission.
• Refunds - a number of purchases refunds.
• ARPU - Average Revenue Per User. Calculated on a cohort basis. The cohort is users, who have installed the app and were marked to the paywall variant.
• ARPPU - Average Revenue Per Paying Users. Calculated on a cohort basis. The cohort is users, who have installed the app and were marked to the paywall variant.
  

For a more detailed analysis, we recommend collecting feedback from users using our Rules tool. With Apphud, you can configure push notifications and display a survey screen when a specific event occurs (e.g., Paywall payment canceled) directly from the dashboard, without the need for development (!). Afterward, you can analyze the feedback from your customers within the dashboard as well. For more information about this tool, please refer to the article Understanding Rules in Apphud in our blog.

P-value

To measure your A/B test results statistically, we use the p-value, or probability value that is expressed as a number between 0 and 1.

This indicator allows us to determine whether the difference in results between paywall variations is due to actual changes or simply the result of chance.

A small p-value (0 - 0.05) indicates that the likelihood of the results occurring by chance is low. The smaller the p-value, the more reliable the determination of a winner.

A large p-value suggests that the observed difference in metrics could be attributed to chance. In such cases, it is not advisable to trust the apparent winner. One can either wait for a sufficient sample size of the audience or establish a new hypothesis and initiate a new experiment.

Once the winner of the experiment is identified, the winning paywall can be applied to the remaining users in the specified experiment audience, and the revenue growth can be monitored.

A/B testing is a powerful tool that can help you optimize your app's performance and improve user engagement. With Apphud, you can easily set up and run A/B tests, track user behavior, and analyze data to make informed decisions about your monetization strategy. By experimenting with different variations of your paywalls, you can identify what works best for your users and continue to increase your app revenue. 

So why not unleash the power of A/B testing with Apphud today and see the results for yourself? Sign up now!

Jane

Head of Business Development at Apphud

10+ years of experience in Project Management and Business Development. Jane began her professional journey as a Sales Manager. Over time, she successfully established herself as Product Owner, and BizDev Lead.