Incrementality
Attribution tells you which ad was there when a user converted. Incrementality asks the harder question: would that user have converted anyway?
Updated
Every attribution model hands out credit, but credit is not causation. A user who already searched for your app by name, or who was about to renew, can be attributed to an ad that changed nothing. Incrementality measures the difference the ad actually made, which is the number that should drive budget decisions.
How incrementality is calculated
You compare a test group exposed to the ads with a control group that is similar in every way except exposure. The difference in outcomes is the incremental effect.
Ways to measure it
- Holdout tests. A random share of the audience is kept from seeing ads. This is the cleanest design, but it needs platform support or user-level control that is often unavailable on iOS.
- Geo tests. Ads are paused or changed in some countries or regions and kept running in comparable ones. Geo tests work without user-level data, which makes them practical for app marketers, but regions must be matched carefully and run long enough to smooth out noise.
- On-off tests. Spend is switched off for a period and the drop, if any, is measured against a forecast. Cheap to run, but seasonality and competitor activity can easily distort the result.
- Model-based estimates. Marketing mix modeling estimates incremental contribution statistically from historical spend and outcomes.
The Apple Ads brand keyword example
Brand keywords are the classic incrementality debate in Apple Ads. Users who search for your app by name have high intent, so brand campaigns show excellent conversion rates and low cost per acquisition. But many of those users would have found the organic listing directly below the ad, which is the problem known as organic cannibalization.
The counterargument is defense. If competitors bid on your name and you stop, some of those users will tap a competitor’s ad instead. The honest answer depends on your market: test it by pausing brand campaigns in a few comparable countries and comparing total installs, trials and subscribers, paid plus organic, against countries where brand campaigns keep running.
Incrementality for subscription apps
- Measure on subscribers and revenue, not installs. An ad can add installs that never pay, or bring fewer but more valuable users.
- Allow for delay. Trials convert days later and renewals months later, so read results on a cohort basis after enough time has passed.
- Use attributed data as the starting point. Cohort ROAS by campaign and keyword shows where to test; incrementality tests tell you how much of that ROAS is real.
Related terms: ROAS and cohort analysis are covered in the adapty.io glossary; marketing mix modeling and retargeting have their own entries here.