A/B testing in Adapty Mail

An A/B test in Adapty Mail compares two or more full email campaigns against each other. Each variation is a complete, independent campaign. When a user matches the test’s segment in a flow, Adapty Mail routes them to one of the variations using the configured weights and tracks delivery, engagement, and revenue per variation.

What a variation is

Each variation is a full campaign. Variations can differ in anything a campaign can differ in — copy, hero images, tone, sequence length, or delay timing between emails. The A/B test itself doesn’t expose those as dials; you create the campaigns separately and add them as variations.

Create an A/B test

  1. Create the campaigns first in Campaigns. Each variation needs its own campaign.
  2. In Adapty Mail, go to A/B Tests and click Create.
  3. Add each campaign as a variation and set its weight. Weights must total 100%.
  4. Assign a segment to control which users the test applies to.
  5. Save.

The test is saved as a draft and doesn’t send anything yet. To go live, it has to be attached to a flow.

A/B Test create form with variations, weights, and segment selection

Launch from a flow

A/B tests can’t be launched from the A/B Tests page — both launch and finish happen inside a flow row.

  1. In Adapty Mail, go to Flows and open the trigger where you want to run the test.
  2. Click Create on a new row. In the dialog, set Content type to A/B Test, pick the test you saved, and click Save.
  3. On the row, click Launch A/B test.

The test’s state moves from draft to live and incoming users who match the segment start being routed to variations. See Create a flow for more on flow rows.

A/B Test create form with variations, weights, and segment selection

How routing works

When a user hits the flow’s trigger and matches the A/B test’s segment, Adapty Mail picks a variation using weighted random selection — each variation’s weight determines its share of the draw. Routing is not deterministic per user.

Read results

On the A/B Tests page, each variation shows its raw counters and derived rates:

  • Delivery: Sends, Deliveries, Bounces.
  • Engagement: Opens, Clicks, Unsubs.
  • Revenue: Purchases, Revenue.

See Campaign analytics for what each metric counts and how revenue is attributed.

A/B Test results with per-variation metrics side by side

Finish the test

Like launch, finishing happens from the flow row, not the A/B Tests page.

  1. Open the flow row where the test is running.
  2. Click Finish A/B test.
  3. In the Finish A/B test dialog, pick the winning campaign from the Replace with campaign dropdown — or leave it empty to remove the segment from the flow entirely.
  4. Confirm.

Users who are already mid-sequence in any variation — winning or losing — keep receiving their scheduled emails. They aren’t switched to the winner.

A/B Test results with per-variation metrics side by side

Lifecycle

An A/B test moves through four states:

  • Draft: Created, not yet attached to a live flow row.
  • Live: Attached and launched; routing incoming users.
  • Finished: Stopped via Finish A/B test.
  • Archived: Soft-deleted from the list.