A/B testing is creating two slightly different versions of something to test which one performs best. It's effective in determining the best way to communicate with your customers.
Here are some examples of what A/B testing can be used for:
- Comparing subject lines
- Promotions or informative content—which do your customers prefer?
- Discounts—monetary vs. percentage
- Embedded links in buttons or in text
- Product images vs. scenic images
The main purpose of A/B testing is to determine the impact of a small difference. If the differences are too significant it'll be difficult to draw any clear conclusions from the results.
Therefore, a good place to start is by testing different subject lines. Follow these steps:
-
Create a new email and tick the box A/B testing.
An A and a B version are created and you're able to enter different subject lines.
-
When you've entered your subject lines and preheaders you'll see a preview of each version.
A/B testing cannot be used in combination with conditional subject lines and preheaders.
A and B versions
You may change the percentages allocated to the A and B versions of the message by dragging the black separator. Note that the percentage allocated to version B will always be equal to version A, and whichever versions wins will be sent to the remaining percentage (60% in the image below).
Best version
There are several options to choose from when selecting how the final, or best, version should be evaluated. There are a few things worth considering before making this selection:
- It takes time for all the recipients to open the message.
- The recipient's email server might lag before sending a response to Engage. Keep in mind that it is desirable to have as much data as possible when determining the final version.
- If, for instance, your Engage setup lacks transactional data, there will be no point in using the options based on purchase sum.
- If the option used is based on purchase sum, please consider the fact that it may take time before the related transactions have been imported to Engage.
Filtering content
When you're in the design step of the message creation, you can choose that each respective module is only presented in one version, or both. You'll find this setting in the module menu on the right-hand side of the editor.
Send
In the 'Send' step you're able to see a preview of the subject line and pre-header for each version.
Both test versions will be sent at the scheduled time to the defined percentage of recipients. The final and best version will then be sent to the remaining audience at the time delay you have specified.
If you choose to send batch-wise, only the winning version that is sent to the remaining audience will be batched up, not the test versions.
To review the results, locate the statistics overview for the specific email. Then choose which version you want to review in the top right corner.
Article last reviewed
Comments
0 comments
Please sign in to leave a comment.