Split test your subject line
Subject line split testing is a way of working out which of two, or three, subject lines is the most effective at encouraging opens. It's also a great way to learn what works and what doesn't, so that you're continually working toward improving conversion rates.
For example, you could test to see if a generic subject line gets more opens than a longer subject line that's more specific.
Alternatively, you could:
- Test two completely different topics as the subject line, to see what content is of most interest to your contacts.
- Add personalization to identical subject lines to see which gets a better response.
- See what kind of promotion works best, for example, 20% off versus free shipping.
You can also A/B test content within an email. To do this, see our guide to content split testing.
Test overview
In a subject line split test you'll write two or three versions of a subject line for one mailing. When the test starts, the mailing will be sent to 20% of the recipients selected. For example, if the total number of recipients is 5,000 contacts, the randomly selected test group will be 1,000 contacts.
Please note that a subject line split test must be sent to at least 30 recipients to function.
A test period runs for four hours, during which time opens are tracked to determine the winning subject line. When setting up a test, you can choose to have the winning version automatically sent to the remaining 80% of recipients after the test concludes.
Alternatively, you can manually select the winner after, or during, the test period.
Set up a test
The option to set up a subject line test is provided after you've created a new mailing. If you haven't already done that:
- Open the Campaigns section in your account.
- On the Emails page, click Create a new mailing, then follow the on-screen instructions to get started.
To add the subject lines you want to test:
- At the top right of the mailing editor, click Review & send.
- Below the subject line field, click Switch to split test.
- Enter two or three subject line variations.
- Optional: Select the "Autosend the winning variation" checkbox to automatically send the most popular subject line when the test period ends.
- Click Start split test now to begin testing immediately, or click Schedule split test to start later and follow the on-screen instructions.
-
While the test is running, you can check to see which subject line is winning, and optionally select the winner yourself.
Manually select a winner
To check on progress while a test is running, open the Response section of your account. The "Mailings" page will open on the "In progress" tab. Click on the name of the mailing to view the test results page.
From here you have the option to end the test early, for example, if there's a clear front runner, or a need to get the email sent earlier than planned. Click End test & send this variation next to the subject line you want to send to the remaining recipients.
FAQs
Below are some answers to questions we're commonly asked about split testing. If you have a question we haven't addressed please contact us.
Why does my winning subject line have less opens than the other subject lines?
After the winning version has been sent, opens continue to be recorded for the 20% of recipients in the test group. Sometimes this can result in another variation of the subject line outperforming the version that was in the lead when the test period ended.
Can you cancel a split test after the test has started?
Split tests can only be canceled by contacting support.
Can you edit a split test if it's scheduled to send later?
Yes, see our instructions for editing a scheduled mailing. Please note that canceling a scheduled mailing reverts it to a saved draft, and this will deactivate your split test settings.
On the "Review & Send" page you'll need to click Switch to split test again. Variation A will be preserved, but you'll need to re-enter the other variations of your subject line.
Will new contacts, added after the test has started, be sent the mailing?
No. Any contacts added to your recipient groups or segments while the test is in progress will have to be targeted separately.