Paywall A/B testing guide, part 4: how to run experiments in Adapty
Updated: October 11, 2024
11 min read
Adapty is a great choice if you’re looking for a convenient framework. Apart from A/B testing it can offer detailed in-app subscription analytics, cohorts analysis, integrations with 3rd-party systems, and more. In this article we’ll cover the main steps in integrating Adapty with your app and setting up an experiment.
This is the final article of our paywall A/B testing series. If you missed our previous articles on marketing and technical aspects of running experiments, you can find them via the links below:
- Paywall A/B testing guide, part 1: how to approach split-testing and avoid mistakes
- Paywall A/B testing guide, part 2: what to test on the paywall
- Paywall A/B testing guide, part 3: how to run A/B tests on paywalls and why it can be difficult
- Paywall A/B testing guide, part 4: how to run experiments in Adapty
Getting started with A/B testing
To use A/B testing for paywalls in Adapty, you need to install the Adapty SDK. Choose any that suits your platform and install it through your package manager. All SDKs are entirely open-source.
We recommend that you immediately integrate your traffic attribution service and transfer additional information to your users so that later it will be easier to work with users on your dashboard.
Please call the Adapty.logShowPaywall(paywall) method: it increments the number of users who see the paywall. This will allow you to calculate statistics in tests more accurately.
Install Subscription Status URL in your App Store Connect:
Check the installation with a test purchase in the Sandbox on your device, as purchases won’t work correctly using the purchase simulator. If everything is alright, you will see a new profile inside Adapty and events in the Event Feed.
Purchases and A/B Tests: step-by-step
Structure of purchases and A/B tests in Adapty consist of four main components:
- Access Level
- Products
- Paywalls
- A/B tests
Access Level
Access level is a marker showing whether a person has access to premium features. Access levels are used if there are screens and functionality inside your application that are only available after activating the subscription.
With just a line of code, your developer can check whether to show the user a paywall or go straight to displaying functions. This is how you do it:
Adapty.getPurchaserInfo { (purchaserInfo, state, error) in
if error == nil {
// “premium” is an identifier of default access level
if purchaserInfo?.accessLevels[“premium”]?.isActive == true {
// grant access to premium features
}
}
}Products
Imagine your app being released for both iOS and Android. Each platform has logically identical purchases, for example, a monthly subscription that opens up the same functionality.
Product IDs are usually difficult to read and recall, so working with them in analytics isn’t very convenient: you need to remember inconvenient IDs for each platform.
With Adapty Products, you can collect logically identical products from App Store and Google Play into one product, making it convenient to work in analytics, create paywalls, and conduct A/B tests.
To create a product inside Adapty:
- Copy App Store and Google Play Product IDs;
- Paste them into the appropriate fields in the Product;
- Set the correct period and access level.
Note that technically, lifetime subscriptions are one-time purchases. For analytics to be measured correctly, we recommend checking the appropriate flag:
Adapty supports Apple Promotional Offers: just add the offer ID to the product, then link it to the paywall.
Once you generate products, you can move on to creating paywalls.
Paywall
Paywall is a crucial entity within Adapty. Technically, that’s an array of products + optional JSON.
Comparing Adapty paywalls and product array in Firebase:
- In Adapty, paywalls are a logical place to purchase. In reality, the number of paywalls usually wouldn’t exceed the amount of 5 to 10 pieces. Most likely, the main areas of purchase are onboarding, settings and somewhere else inside the application: so you can create either three separate paywalls, therefore achieving maximum customisation, or make just one for every screen.
- You can modify the paywall (that is, the set of purchases) on the fly; you do not need to release the application or ask the developers to do something about this.
- Paywalls have versions. All product changes are versioned and analytics is available for each version. You don’t need to create hundreds of JSON files like in Firebase to get lost in them later on.
The paywall has an SDK ID, which is the only thing that needs to be hardcoded on the device.
2024 subscription benchmarks and insights
Get your free copy of our latest subscription report to stay ahead in 2024.
Paywall’s anatomy
Adapty Paywall has two main components:
- Shopping set. Add up to five products to the paywall. Optionally, you can add an Apple Promotional Offer. Order of purchases can be changed: this affects how they are displayed and received in the application.
- Custom JSON. Namely, JSON with any additional fields that you might find useful in your application. For example, if you want to change pictures or texts, you just need to agree with the developer on the data exchange format and fill in the JSON.
Primary user case in Adapty paywalls is to receive purchases from the service and place them on your screen. Developers themselves make the layout and design of the screen, and purchases are received through Adapty. During the design process, developers lay out key elements, such as text and images, which can then be modified using a custom JSON.
This scenario gives you great flexibility and allows you to stay within your style while using a paywall of any visual complexity.
Paywall analytics
Over time, any paywall accumulates stats:
- Subscription purchases
- Trial subscription activation
- Subscription cancellations
In Adapty, you can see all these metrics, and some others.
To ensure that statistics are read correctly, use the Adapty.logShowPaywall(paywall) method. Call it precisely the moment the user opened the corresponding paywall.
Adapty shows metrics both at the paywall level and at the product level in paywall.
A/B tests for paywalls
Suppose you send a paywall to production and at some point begin wondering: How do you compare multiple offers? How do you show a paywall to a specific user group? Of course, you’d want these tasks to be solved while avoiding a mobile application release and writing additional code.
With Adapty, you can do this in two clicks. Convert paywall to A/B paywall test (remember that paywall in Adapty is a logical place in the application), then add purchase options.
When converting the paywall to A/B test, the A/B test’s SDK ID remains the same as that of the paywall. Thanks to this, you don’t have to release the application.
You can think of a paywall as a card and an A/B test as a deck of cards. Your app will show a paywall in any case, so the necessary card is accidentally pulled out of the deck and sent to the device.
You can only create an A/B test based on a paywall. We believe that this logic allows you to save data accurately as well as avoid many excessive paywalls.
Anatomy of A/B Tests in Adapty
Internally, the test consists of:
- SDK ID (A/B test ID), which was passed from paywall;
- Lots of segments and sets of paywalls.
A/B tests include the ability to target the test audience using Segments, groups of users with the same attributes.
By default, the test is created with one segment: all users. Add as many internal paywalls as you like inside this test. The internal paywall is the same paywall but without the SDK ID.
Within the segment, you can make the desired distribution of weights. For example, if you are unsure of your experimental solution, you can allocate 10% of your traffic to a new paywall.
If you opt to finish the test, Adapty will show you a list of all paywalls and offers to choose the best one. After that, all internal paywalls will go to the archive under the Inactive button, and the test will turn into a paywall again.
A/B test analytics
Just as in the paywall, the test results can be monitored live at the level of internal paywalls and products. Metrics remain the same.
However, when choosing two paywalls, Adapty conducts a statistical test for significant differences between paywalls.
A/B tests boost your growth
We see that with our A/B tests, companies are increasing their revenues up to 40% within a few months. After integration, setting up paywalls and A/B tests does not require any code and can be done directly by marketers. They can compare paywalls, change price tags, manage payment screens without app releases, and analyze your business’s health deceivingly effortlessly.
With a 14-day trial and five strings of code to get you going, Adapty is worth checking out. Implement in-app subscriptions, purchases restore, receipt validation in your app using our mobile SDK. Adapty SDK is available on any platform you’d need and requires less than a dozen strings to get you going.
Recommended posts
Tutorial
Money
Trends-insights
22 min read
January 23, 2024
