A/B testing lets you compare two versions of the same sendout to understand what performs best before scaling your recipients. By splitting recipients evenly between two variants, you can test changes such as subject lines, content, or layout and base your decisions on real performance data.

Results of A/B testing are shown directly in the performance view, making it easy to evaluate outcomes per variant and per market. A/B testing is especially useful when you want to optimize engagement using data rather than assumptions.

How it works

Recipient distribution and variants

Recipients are always split evenly between variant A and variant B. If you send to multiple markets, each market gets its own A and B variant, for example: Default: A and B, Denmark: A and B, Finland: A and B.

Evaluation method: Bayesian probability

A/B test results are calculated using a Bayesian probability method. This method calculates how likely it is that variant B performs better than variant A. Variant B is always evaluated in comparison to variant A.

The result is shown as a probability, for example:

“There is a 92% probability that variant B performs better than variant A.”

What can I test?

You decide what differs between the variants based on the changes you make. Examples of elements you can test include:

• Subject line
• Pre-header
• Sender name
• Message
  • Content changes e.g., images, tone of voice, promotions
  • Styling and layout
  • Moving modules
  • Add or hide modules

Tracking parameters: Tracking parameters can be changed per variant if you want to follow contact behavior on site after a click.

All changes in variant B are always compared against variant A in the results.

Setting it up

Once the sendout is created, enable A/B test. A dialog explains that recipients will be distributed evenly between two variants. Click Continue to create the variants. 

Each market will get an A and B variant. 

In the design studio you can now work with your design of the email. Up topp you can switch between the A and B variant. and in the left side overview you will alway see which variant module you are working on. 

Recommended procedure:

1. Start by creating variant A
2. Make the changes you want to test to variant B
3. Localize content if needed

Variant A is the base version. All changes made in variant A are automatically inherited by variant B. To make unique changes in variant B, you must unlink the module you want to change. Once unlinked, you can edit the unlinked module in variant B without affecting variant A.

This applies to both message content and subject line. The subject line is edited directly in the canvas area. To use different subject lines, unlink before making changes in variant B.

When you are done designing both variants, publish and schedule your sendout as usual. 

Performance view and results

Once the emails are sent, data collection starts automatically and continues as long as more data is received.

A/B test results are shown in the performance view.

The "All view" shows overall sendout performance, including sent, delivered, bounces, opens, and clicks, as well as data per market. 

When you switch from "All" to a specific market or group, you see a direct comparison between variant A and variant B.

Choosing the winning metric

You can choose which metric should determine the A/B test result, such as:

Open rate or Click rate

When you change the metric, the probability result updates accordingly.

Interpreting the A/B test result

Read the result as a probability. For example: If the probability that variant B beats variant A is 17.7%, it is unlikely that B is the better option. Variant A is therefore more likely to perform better. 

Make sure you are evaluating the right metric. Open rate and click rate can lead to different winners depending on how the variants perform. 

Each market will get an A and B variant. Work with variants in the Design Studio. The sendout includes two variants: variant A and variant B.

Article last reviewed 09 February 2026 15:12