A/B test

Are you looking to boost your in-app purchases and subscription revenue? One effective way to optimize your offerings is through A/B testing. With Adapty you can easily create and manage A/B tests for different pricing strategies, subscription lengths, trial durations, and more. In this guide, we'll walk you through the steps of creating A/B tests in the Adapty Dashboard and analyzing the results to make data-driven decisions about your in-app purchases and subscriptions. Whether you're new to A/B testing or looking to improve your existing strategies, this guide will provide you with the tools and insights you need to maximize your app's revenue potential.

warning

Be sure you send paywall views to Adapty using the .logShowPaywall(). Without this method, Adapty wouldn't be able to calculate views for the paywalls within the test, which would result in irrelevant conversion stats.

A/B test types

Adapty supports two types of A/B tests:

• Regular A/B test: An A/B test created for a single placement. This is useful if:
  • You have only 1 placement in your app.
  • You want to run your A/B test for only one placement even if you have multiple placements in your app and see economics changes for this one placement only.
  • You want to run an A/B test on old users (those who have seen at least one Adapty paywall).
• Crossplacement A/B test: An A/B test created for multiple placements in your app. This means the same variant once determined by the A/B test will be consistently shown across different sections of your app. This is useful when:
  • You want to use the same variant in multiple sections, such as Onboarding and Settings.
  • You want to evaluate your app's overall economy, ensuring that A/B testing is conducted across the entire app rather than just specific parts, making it easier to analyze results in the A/B testing statistics.
  • You want to run an A/B test on new users only, i.e. the users who have never seen a single Adapty paywall.

warning

Crossplacement A/B tests are only available for native iOS and Android SDKs starting from v3.5.0. Support for cross-platform SDKs is coming soon.

Users from previous versions skip them and can only become a part of regular A/B tests.

Key differences

Feature	Regular A/B Test	Cross-placement A/B Test
What is being tested	One paywall	Set of paywalls belonging to one variant
Variant consistency	Variant is determined separately for every placement	Same variant used across all placements
Audience targeting	Defined per placement	Shared across all placements
Analytics	You analyse one placement	You analyze the whole app on those placements that are a part of the test
Variant weight distribution	Per paywall	Per set of paywalls
Users	For all users	Only new users (those who haven’t seen an Adapty paywall)
Adapty SDK version	Any	⚠️ 3.5.0+
Best for	Testing independent changes in a single placement without considering the overall app economics	Evaluating overall monetization strategies app-wide

A/B test selection logic

As you may have noticed from the table above, cross-placement A/B tests take priority over regular A/B tests. However, cross-placement tests are only shown to new users — those who haven’t seen a single Adapty paywall yet (to be precise, getPaywall SDK method was called). This ensures consistency in results across placements.

The logic Adapty follows when deciding which A/B test to show is outlined below:

Regular and cross-placement A/B tests are displayed in separate lists, which you can switch between.

Crossplacement A/B test limitations

Crossplacement A/B tests guarantee the same variant across all placements in the A/B test, but this causes several limitations:

• They always have the highest priority in a placement.
• Only new users can participate, i.e. the users who have not seen a single Adapty paywall before (to be precise, getPaywall SDK method was called). That is done because it's not possible to guarantee for the old users that they will see the same paywall chain, because an existing user could have seen something before the test has been started.

Creating A/B tests

When creating a new A/B test—whether it's a regular or cross-placement one—you need to include at least two paywalls. Each paywall is assigned a weight that determines how traffic is split during the test.

For example, if one paywall is given a weight of 70% and the other 30%, the first paywall will be shown to roughly 700 out of 1,000 users, and the second to about 300. The total weight across all variants must equal 100%.

The key difference between regular and cross-placement A/B tests is that cross-placement tests allow you to show paywalls in multiple placements. These paywalls can be different or even partially the same depending on what you're testing. In cross-placement tests, weights are assigned per variant, not per paywall.

Setting up your A/B test this way allows you to effectively compare different paywalls and make smarter, data-driven decisions for your app’s monetization strategy.

To create a new A/B test:

1. Open the A/B tests section from the Adapty main menu.

   

2. Click the Create A/B test button.

   

3. In the Create the A/B test window, enter a Test name. This is required and should help you easily identify the test later. Choose something descriptive and meaningful so it’s clear what the test is about when you review the results.

4. Fill in the Test goal to keep track of what you're trying to achieve. It could be increasing subscriptions, improving engagement, reducing churn, or anything else you're focusing on. A clear goal helps you stay on track and measure success.

5. Now it’s time to set up the test content using the Variants table. Variants go in rows, placements go in columns, and paywalls are added where they intersect. By default, there are 2 variants and 1 placement, but you can add more of each. Once you add a second placement, the test becomes a cross-placement A/B test.

   

   Key:

1	Rename the variant to make it more descriptive for you.
2	Change the weight of the variant. Keep in mind that in total, the variants must build 100%.
3	Add more variants if needed.
4	Add more placements if needed.
5	Add paywalls to display in the placements for every variant.

7. Don’t forget to save your test. You have two options:
   1. Save as draft: This means the test won’t go live right away. You can launch it later from either the placement or A/B test list. This option is great if you’re not quite ready to start the test and want more time to review or tweak it.
   2. Run A/B test: Choose this if you’re ready to launch the test immediately. The test will go live as soon as you click this button.

To learn more about launching A/B tests, check out our guide on running A/B tests. You’ll also be able to track performance using a variety of metrics — see the metrics documentation for more details.

Editing A/B tests

You can only edit A/B tests that are saved as drafts. Once a test is live, it can’t be changed. However, you can use the Modify option to create a duplicate of the test with the same name and make your updates there. The original test will be stopped, and both the original and the new version will appear separately in your analytics.