How to run A/B test for mobile paywalls in Adapty?

8 min read

Share

With the release of iOS 13, Apple set the trend for the privacy of user data. More recent seismic changes, related to developers’ access to IDFA, will affect traffic purchases, the work of internal unit economics analytics systems, and mobile marketing as a whole.

Traffic volumes in mobile apps should not be impacted. However, the traffic itself will become less targeted, and the number of irrelevant users will grow. In the context of this shift, personalisation of onboarding and payment screens (paywalls) will have the maximum impact on the quality of monetisation.

Targeted A/B tests are a tool for finding the best deal for a segment of your app users. In a post-IDFA environment, they are becoming the most effective way to keep mobile targeting accurate. In this article, we will cover how A/B tests are done in Adapty.

Getting started with A/B testing

To use A/B testing for paywalls in Adapty, you need to install the Adapty SDK. Choose any that suits your platform and install it through your package manager. All SDKs are entirely open-source.

We recommend that you immediately integrate your traffic attribution service and transfer additional information to your users so that later it will be easier to work with users on your dashboard.

Please call the Adapty.logShowPaywall(paywall) method: it increments the number of users who see the paywall. This will allow you to calculate statistics in tests more accurately.

Install Subscription Status URL in your App Store Connect:

App Information page in App Store Connect

Check the installation with a test purchase in the Sandbox on your device, as purchases won’t work correctly using the purchase simulator. If everything is alright, you will see a new profile inside Adapty and events in the Event Feed.

<img src=“spacer.gif” alt=“”>
Profiles inside Adapty and events in the Event Feed.

Purchases and A/B Tests: step-by-step

Structure of purchases and A/B tests in Adapty

Structure of purchases and A/B tests in Adapty consist of four main components:

  • Access Level
  • Products
  • Paywalls
  • A/B tests

Access Level

Access level is a marker showing whether a person has access to premium features. Access levels are used if there are screens and functionality inside your application that are only available after activating the subscription.

With just a line of code, your developer can check whether to show the user a paywall or go straight to displaying functions. This is how you do it:

Adapty.getPurchaserInfo { (purchaserInfo, state, error) in
   if error == nil {
       // “premium” is an identifier of default access level
       if purchaserInfo?.accessLevels[“premium”]?.isActive == true {
           // grant access to premium features
       }
   }
}


There can be several access levels. Tinder uses three subscription tiers and, accordingly, three access levels: Plus, Gold, and Platinum.

Plans in Tinder mobile app
Tinder Platinum is only available to Plus and Gold members.

Products

Imagine your app being released for both iOS and Android. Each platform has logically identical purchases, for example, a monthly subscription that opens up the same functionality.

Product IDs are usually difficult to read and recall, so working with them in analytics isn’t very convenient: you need to remember inconvenient IDs for each platform.

With Adapty Products, you can collect logically identical products from App Store and Google Play into one product, making it convenient to work in analytics, create paywalls, and conduct A/B testing.

To create a product inside Adapty:

  1. Copy App Store and Google Play Product IDs;
  2. Paste them into the appropriate fields in the Product;
  3. Set the correct period and access level.
Product section in Adapty

Note that technically, lifetime subscriptions are one-time purchases. For analytics to be measured correctly, we recommend checking the appropriate flag:

Product section in Adapty

Adapty supports Apple Promotional Offers: just add the offer ID to the product, then link it to the paywall.

Once you generate products, you can move on to creating paywalls.

Paywall

Paywall is a crucial entity within Adapty. Technically, that's an array of products + optional JSON + visual rector (editor).

<img src=“spacer.gif” alt=“”>
Paywall as an array of elements

Comparing Adapty paywalls and product array in Firebase:

  1. In Adapty, paywalls are a logical place to purchase. In reality, the number of paywalls usually wouldn't exceed the amount of 5 to 10 pieces. Most likely, the main areas of purchase are onboarding, settings and somewhere else inside the application: so you can create either three separate paywalls, therefore achieving maximum customisation, or make just one for every screen.
  2. You can modify the paywall (that is, the set of purchases) on the fly; you do not need to release the application or ask the developers to do something about this.
  3. Paywalls have versions. All product changes are versioned and analytics is available for each version. You don't need to create hundreds of JSON files in Firebase to get lost in them later on.

The paywall has an SDK ID, which is the only thing that needs to be hardcoded on the device.

Creating Paywall in Adapty

Paywall's anatomy

Adapty Paywall has three main components:

  1. Shopping set. Add up to five products to the paywall. Optionally, you can add an Apple Promotional Offer. Order of purchases can be changed: this affects how they are displayed and received in the application.
  2. Custom JSON. Namely, JSON with any additional fields that you might find useful in your application. For example, if you want to change pictures or texts, you just need to agree with the developer on the data exchange format and fill in the JSON.
  3. Visual editor. If you prefer configuring the screen as fast and independent as possible rather than designing it with developers, you can use our editor. Right now, there is a template that works best with a one-offer screen.

Primary user case in Adapty paywalls is to receive purchases from the service and place them on your screen. Developer himself makes the layout and design of the screen, and purchases are received through Adapty. During the design process, developers lay out key elements, such as text and images, which can then be modified using custom JSON.

This scenario gives you great flexibility and allows you to stay within your style while using a paywall of any visual complexity.

Paywall analytics

Over time, any paywall accumulates stats:

  • Subscription purchases
  • Trial subscription activation
  • Subscription cancellations

In Adapty, you can see all these metrics, and some others.

Available paywall metrics in Adapty

To ensure that statistics are read correctly, use the Adapty.logShowPaywall(paywall) method. Call it precisely the moment the user opened the corresponding paywall.

Adapty shows metrics both at the paywall level and at the product level in paywall.

A/B tests for paywalls

Suppose you send a paywall to production and at some point begin wondering: How do you compare multiple offers? How do you show a paywall to a specific user group? Of course, you’d want these tasks to be solved while avoiding a mobile application release and writing additional code.

With Adapty, you can do this in two clicks. Convert paywall to A/B paywall test (remember that paywall in Adapty is a logical place in the application), then add purchase options.

When converting the paywall to A/B test, the A/B test’s SDK ID remains the same as that of the paywall. Due to this, you don’t have to release the application.

You can think of a paywall as a card and an A/B test as a deck of cards. Your app will show a paywall in any case, so the necessary card is accidentally pulled out of the deck and sent to the device.

Testing different periods with Adapty

You can only create an A/B test based on a paywall. We believe that this logic allows you to save data accurately as well as avoid many excessive paywalls.

Anatomy of A/B Test in Adapty

Internally, the test consists of:

  • SDK ID (A/B test ID), which was passed from paywall;
  • Lots of segments and sets of paywalls.

A/B tests include the ability to target the test audience using Segments, groups of users with the same attributes.

By default, the test is created with one segment: all users. Add as many internal paywalls as you like inside this test. The internal paywall is the same paywall but without the SDK ID.

Within the segment, you can make the desired distribution of weights. For example, if you are unsure of your experimental solution, you can allocate 10% of your traffic to a new paywall.

Setting balance weights in Adapty

If you opt to finish the test, Adapty will show you a list of all paywalls and offers to choose the best one. After that, all internal paywalls will go to the archive under the Inactive button, and the test will turn into a paywall again.

A/B test analytics

Just as in the paywall, the test results can be monitored live at the level of internal paywalls and products. Metrics remain the same.

However, when choosing two paywalls, Adapty conducts a statistical test for significant differences between paywalls.

Different Periods for Paywalls in Adapty

A/B tests boost your growth

We see that with our A/B tests, companies are increasing their revenues up to 40% within a few months. After integration, setting up paywalls and A/B tests does not require any code and can be done directly by marketers. They can compare paywalls, change price tags, manage payment screens without app releases, and analyze your business’s health deceivingly effortless.

With 14-day trial and five strings of code to get you going, Adapty is worth checking out. Implement in-app subscriptions, purchases restore, receipt validation in your app using our mobile SDK. Adapty SDK is available on any platform you'd need and requires less than a dozen strings to get you going.


Grow in-app subscription revenue

Start for free
Or book a call with us
SCHEDULE A DEMO
February 1, 2021
Vitaly Davydov

More reads you might like

GO Back to the blog