Analyzing Firebase A/B test results in Apphud

Firebase A/B Testing helps you to test your app’s UI, features, and push notification texts on some small audience before rolling out to the wide public. We will concentrate on classic Remote Config tests.

Remote Config – is a cloud key-value hash map, which usually loads on app launch. You can store app logic parameters and any values that affect your app's UI or logic. I think almost every app developer used to work with Remote Config.

While creating A/B tests users can set exposure in percents, variants, and their weights. However, the most interesting part is the goal metric. You can choose between crash-free users, retention, purchase revenue metrics, and others. The most important metric in most cases is Purchase Revenue.

Purchase revenue and in_app_purchase

The results table is quite simple. You can view Total revenue and ARPU values among each variant. But that's it. This is the maximum you can view in Firebase.

In Firebase A/B Testing:

• You can't view ARPPU
• Can't segment users across countries, app versions, OS versions, and other parameters.
  
• Firebase only tracks revenue while the app is in the foreground; it doesn't send trial conversions and renewals while the app is inactive.
  
• You can't view charts – Proceeds, Conversions, and others among each variant.
  
• Refunds are not counted.
  

By adding just a few lines of code you can send remote config values (which includes experiment values) to Apphud. And view results in Apphud Charts.

Proceeds

User Properties is a custom key-value hash map attached to a user. Values can be set once, can be incremented, or updated. You can set properties after the app launch or at any time.

Apphud.setUserProperty(key: .email, value: "user4@example.com", setOnce: true)  

Apphud.setUserProperty(key: .age, value: 31)

Besides built-in fields, you can also send your values, including Firebase remote config values.

Limitations:

• Value must be one of: Int, Float, Double, Bool, String, NSNumber, NSString, NSNull, nil.
• Key must be 30 characters maximum.
  
• Each user may have a maximum of 50 attributes.
  

While creating A/B tests you can add as many parameters as you wish in each variant. When creating a new A/B test add a new parameter with a key as a unique test name and values as each variant name. See the screenshot below.

Parameter in variants

In the screenshot, there is a paywall_type parameter that is being tested with the v1 and v2 values as variants.

Assuming both Apphud and Firebase SDKs are initialised:

RemoteConfig.remoteConfig().fetchAndActivate { _, _ in  
  DispatchQueue.main.async {  
    RemoteConfig.remoteConfig().allKeys(from: .remote).forEach { configKey in  
      let configValue = RemoteConfig.remoteConfig().configValue(forKey: configKey)  
      if let string = configValue.stringValue, string.count > 0 {  
        Apphud.setUserProperty(key: .init(configKey), value: string, setOnce: true)  
      }  
    }  
  }  
}

In Charts you can add filter by clicking Where. Choose your test name and enter the value:

Filter in Charts

and view your charts:

View charts

All done!

 Bonus 1: Filter and view user pages exposed in the experiments

On the Users page, there are also Filters where you can filter out users from the experiment with user properties filters applied.

Bonus 2: Export these users in CSV

There is the Export Users button on the Users page where you can export your users in CSV with user properties filters applied.

So, let’s summarize – if you use Firebase for A/B-testing, give Apphud User Properties a try! You can get really helpful insights and enrich your experiment results. Have a great day!

To get more insights regarding subscription app revenue growth read the Apphud Blog.

Ren

Co-founder at Apphud

Ex iOS app and game developer. 11 years in the industry since iOS 3. More than 50 apps are in the background with 4 exits. Entrepreneur and traveler.