![From corporate to indie: How and when to make the transition 17 From Corporate to Indie How and When to Make the Transition](https://adapty.io/wp-content/uploads/2024/09/From-Corporate-to-Indie_-How-and-When-to-Make-the-Transition.png)
Trends-insights
15 min read
September 5, 2024
December 21, 2023
24 min read
It’s not a secret that A/B testing is considered a powerful springboard for boosting revenue in mobile apps and the web. The statistics show that mobile app developers and publishers that run paywall experiments can make up to 100x more app revenue. Yet, this marketing tool has been lately covered with a plethora of myths concerning its effectiveness, safety issues, and being overrated in general.
In this article, we’ll take the role of myth-busters and try to disprove some of the most popular myths circulating around paywall A/B testing. We’ll provide more context to the myths to show that they are mostly based on the lack of information or someone’s bad experience. In addition to that, we’d also cover several common mistakes that prevent people from utilizing A/B testing to 100%. Let’s start with the myths first.
I’ve met people from the mobile app industry who avoid paywall A/B testing like the plague, claiming that it’s unsafe, complicated, and takes more time than provides final value. In this part, we’ll take a look at a few common myths and misconceptions regarding paywall A/B testing and see that they are either not true or simply rudimentary prejudice.
This is one of the most popular and common myths circulating in the mobile app community. Everyone in the industry has heard stories of this or that app getting banned and deleted by the App Store for guidelines violations. Of course, Apple is rather strict when it comes to rules and regulations, but they’re also pretty consistent in this matter, meaning they actually go by the book and do not go beyond what’s included in their guidelines.
If you’re new to A/B testing: remote config is a feature that enables you to remotely change an interface element with the help of placeholder variables inserted in the code beforehand which can later be tweaked via JSON file. Such manipulations let you change elements in your app or software without having to release an update or a new version.
The requirement that is feared the most in such cases is 2.5.2 of the Software Requirements which states: “Apps should be self-contained in their bundles, and may not read or write data outside the designated container area, nor may they download, install, or execute code which introduces or changes features or functionality of the app, including other apps”. Well, it’s a pretty reasonable point, but it doesn’t relate to the use of 3rd party frameworks and remote config for paywalls. Here are a few points to bust this myth:
To sum it up, the remote config or any other A/B testing platform that is based on it is safe to use in your app for testing or changing your paywall. Moreover, it may even be viewed as a test platform for providing the best user experience in your app, which is beneficial for the App Store itself.
When you finally decide to try running A/B tests for your paywalls, you start searching for a tool that should be preferably free and well-known. The first framework that pops up is usually Firebase. And it’s pretty clear why: Knowing that it’s powered by Google and has tons of tutorials can make you think that it must be a pretty reliable framework, working with which is going to be a walk in the park. But it’s not that simple.
Even though Firebase enables app developers to run A/B tests in their apps, it was not designed to experiment with paywalls. We have already prolifically investigated this matter, but let’s go through the main takeaways once again:
While Firebase may excel in general mobile app experience testing, it falls short in meeting the specific needs of paywall A/B testing, requiring additional steps, custom coding, and integration with third-party tools to achieve comprehensive and actionable results. Dedicated solutions like Adapty are more suitable options for streamlined and effective paywall experimentation.
Another thing that scares people from even trying A/B testing is complexity. In big companies such experiments are run and analyzed by growth managers or analysts, which makes many app developers think that launching a test and “deciphering” the results is nothing else but secret lore.
In reality, all you need is a basic understanding of mobile app metrics and a convenient platform. For example, launching a paywall A/B test in Adapty takes no more than 5 minutes:
For the more statistically significant results, wait at least 2 weeks before stopping the test (if there’s no definite decline in core metrics, which you can check live in the analytics dashboard). When the experiment is over, the system will show you a detailed report on every paywall’s metrics, including Revenue, ARPPU, Purchases, Trials, Views, Refunds, and many more. But most importantly, you will clearly see which one has the higher P2BB percentage (Probability to Be the Best) along with a neat visualization of the experiment timeline.
Following the default development scheme that’s been used for several years, we have the idea in our heads that every screen in your app must be created by the app designer, or at least made with their approval. Such an approach is important for the app’s consistency but it doesn’t mean that for every paywall A/B test you need to have a new paywall created by your team’s designer – not anymore.
With Adapty’s Paywall Builder, marketing or growth specialists, who usually run monetization experiments, can create beautiful paywalls with a native layout in a matter of minutes – with no developer involved. Paywall Builder is a visual no-code editor that enables you to design a paywall based on one of the numerous best-selling templates. Every element on the paywall can then be adjusted to your liking, including images, text font and size, button shape, color scheme, and more.
A high-quality paywall can actually be created in a few minutes and immediately launched in an A/B test. The same goes for recurring tests: if you’re not satisfied with the results of the previous test, you can apply new changes to the existing paywall pair and relaunch the test, which will also take no more than 5-10 minutes. The future of paywall testing is already here.
2024 subscription benchmarks and insights
Get your free copy of our latest subscription report to stay ahead in 2024.
Now that we’re through with the popular myths, let’s talk about equally common mistakes people tend to make to this day. Along with it, you’ll find out how innovations in the sphere of in-app purchases and monetization can improve the process of paywall A/B testing and solve older problems in a more convenient way.
This may be one of the most common and painful mistakes an app developer or publisher can make. Broken updates are always a loss – both financial and reputational – that’s why we always test new builds before publishing them. For some reason, it’s not always the case with paywalls. A hypothesis may seem so promising that one may not want to waste their time on running a test for a couple of weeks and apply changes to the paywall right away instead.
It may work out, but very often it leads to churn, higher bounce rate, lower conversion rates, and, as a result, a decrease in revenue. Why? Because one could miscalculate things, didn’t pay enough attention to dependencies in the funnel, or still made a technical error that led to troubles with purchases or user flow. Such situations require a quick reaction and swift actions to be taken to restore the previous version of the paywall. But Murphy’s law is still in the game, so you may get into a situation where the follow-up update gets stuck at the store review stage, leaving you to sit and watch your money going down the drain.
Should we say this can never happen when you use Adapty? Primarily for two reasons:
On the one hand, it’s a basic mistake you learn about right before diving into the world of A/B testing, but on the other, it’s a mistake that even experienced growth managers tend to make from time to time.
Important note: It’s completely normal to test two drastically different paywalls when you intend to test a new paywall template or structure and your only goal is to see that based on your hypothesis the new variant performs at least as well as the older one. In our case, we’re talking about testing several elements within the same template/structure with the hypothesis of increasing this or that conversion rate or some other metric.
To see the most transparent results for any split experiment, you must have the minimum amount of changes in the testing pair. For example, testing only a background picture will give you a more or less accurate result to see which one is more appealing to the users. But testing a background image along with a different product layout and one more new product option will be a mess.
I mean, you will get some kind of a result, maybe even a statistically significant one, and perhaps even the test version may win, but you’ll never know what exactly made the users turn to the paywall “B”. Was it a new product, or a new layout? Or maybe just a new picture was enough? It’s fine when your test paywall actually won, but usually, it’s the other way around, so in the end, you’re left with a failed test and don’t even know what happened.
I mentioned earlier that even experienced managers tend to abuse this approach, as they feel they already know their product and their audience, but most importantly, they do so because, at some point, time becomes money. Product managers of apps with more than $500k MRR may not want to wait for ~2 weeks to test only one element. So sometimes it feels easier and cheaper for them to come up with a more complex hypothesis and test it right away. It may seem rather bold, but the road to success is fraught with risk.
Once you’ve learned the beauty of paywall A/B testing, you may find yourself in a situation where you test one and the same paywall over and over for all the users who installed your app. On the one hand, it solves the problem of insufficient traffic, which we’ll talk about a bit later, but on the other hand, it may lead to you receiving less profit than you could from acquiring more high-quality subscribers.
Targeting helps set up A/B tests for particular audiences based on a number of factors, like age, region, language, or other custom attributes. Having not only one general paywall but also a few paywalls targeted to certain audiences, may help you find the most profitable subscriber flow that you could run paywall A/B tests on and find the best product-price relationship. Sadly, not every platform that features paywall A/B testing allows for such targeting, but Adapty does. Adapty enables you to show different paywalls to different audiences, distinguished by one or several attributes, and as a result, run separate A/B tests.
Continuing the idea of segmentation, it’s worth saying that paywall localization is another vital element that very often gets neglected. In most cases, when creating an app for the Western world, app developers simply leave the interface, along with the paywall, in English to make it more universal for users and easier for themselves. In other cases, they take 5-10 most popular languages and translate the app along with the paywall, either by Google translate or with the help of real translators. And in theory, it should be enough, but only in theory.
There may be a couple of problems here that could actually be depriving you of getting more revenue:
Why are we talking so much about the localization all of a sudden? Well, because all these problems can be easily solved by A/B testing, especially if you use Adapty.
Translations for your paywall may be added with the help of the aforementioned remote config and adjusted on the fly, which reduces the risk of showing a visually broken paywall. Or if you decide to use the Paywall Builder to create a paywall, you can add new locales right in the editor. Every locale you add is broken down into strings of text for a corresponding paywall element, which makes the localization process much easier. Once the translations are added, you can preview the localized text on the mockup of your paywall before publishing it.
The same goes for the cultural aspect problem – there’s no faster way to find out which paywall variant works best for a certain culture than A/B testing. Launch a test with two different background images and see if it works. But make sure to aim at the right audience, which is also super convenient with Adapty’s audience targeting. Thanks to its paywall management system, you can set up precise targeting for A/B testing based on many criteria, including region, age, channel, language, etc. So you get both localization and targeted A/B testing in one place – problems solved.
Another childish mistake comes from actually getting into A/B testing with too much enthusiasm. It’s great to get into experimenting at the early stages of your app growth, but A/B testing of any kind requires as much traffic as possible, which can be a problem at that time.
To get statistically significant results, you need a significant amount of users – it’s that simple. If your user flow is less than 3000-5000 people a month, you risk getting unclear and unreliable results, which may lead you in the wrong direction. There are two ways out of this: EIther postpone A/B testing for a while and wait till your app gets bigger organically or purchase ad traffic. Getting paid ads is one of the most popular User Acquisition strategies, so if your funds allow it – go for it.
Paywall A/B testing is a safe and efficient way to grow your mobile app, no matter what rumors you may hear. With current technologies, the process of split testing has become much more convenient, affordable, and easy to dive into. If you’ve been putting off tasting app monetization experiments, it’s the best time to start and Adapty is a great choice to use in this journey. Find out more about how it can help you boost your MRR.
Recommended posts
Trends-insights
15 min read
September 5, 2024