What are Multivariant Test Emails?

When you are exploring the most effective ways to garner a Prospect's interest in your business, email performance is key. Knowing what techniques your customers will respond to can help you decide how to structure future content. 

Multivariant Test Emails allow users to send different versions of an Email to different groups of Prospects with the goal of understanding what draws a Prospect's attention most effectively.

Plans

Multivariant Test Emails are available to users on Professional and Enterprise plans.

Definitions

Variable

The element of your campaign you want to test. You can test by Subject Line, From Name, Content, and Send Time. Each version of the variable is called a variation.

Once the winning combination has been determined, the winning Email will be sent to any remaining Prospects on the recipient Lists.

Combination

A combination is a version of your Email that is created from your chosen variables. A Multivariant Test Email allows you to test up to three variables, with up to eight variations, in a single email test campaign.

We recommend at least 5,000 recipients per combination to have a statistically significant sample size.

When you click on the plus or minus below each of the variables, the summary is updated to show the number of tests and recipients for each combination.

If you were to start building a combination with two Subject Lines and three Send Times, the other variable options would become disabled. This combination produces six variations - three different Send Times for two Emails with different Subject Lines.

This means that if you were to add a third variable, it would go over the combination limit. For example, adding two different From Names would mean Insightly would be testing three different Send Times for two Emails with different Subject Lines, plus a different From Name on each of those Emails. Two Subject Lines x two From Names x three Send Times would create 12 test combinations.

Test

The period of time after the Email combinations are sent out. During the testing period, Insightly will measure which variations were the most successful and determine the winning combination of variables automatically.

If no winning criteria is met within the test period, the first combination will be selected as the winner.

If you test Send Times, the percentage of recipients to receive the test combinations will be automatically incremented to 100% and the wait time for the winning combination will be removed. This is because all Email recipients are being used in the test combination, and there is no winning portion.

Winner

The combination of variables that perform the most successfully. Users can choose whether the success of the Email is determined by email click rate or email open rate.

The winner of the test Emails will appear in the details tab of the record. If you want to review the results of the test emails, go to the Statistics tab of the Email record and use the Combination dropdown to see how each set of variations performed.

How Multivariant campaigns work

Set up

A Multivariant Test Email allows you to test up to three variables, with up to eight variations, in a single Email test campaign. For example, you may want to test two subject lines and two variations of the Email’s content.

Users can choose Lists to send these Emails too, but they cannot select which Email each Prospect will receive.

Email status

Draft - If the campaign hasn’t been started, the status will be set as Draft.

Scheduled - If the campaign is scheduled but no Emails have been sent yet, the status will be set as Scheduled.

Sending - Once the first test Emails have started sending or when the winning Email is being sent, the status will be set as Sending.

Determining Winner - Once the last test Email has been sent, the status will be set as Determining Winner.

Sent - Once all test Emails have been sent out (including the winner Emails), the status will be set as Sent.

Choosing a winner

The winning combination is selected automatically by Insightly. The winner can be determined automatically by either the highest open or click rate.

If you’re testing using Send Times, all of the Email recipients will be scheduled to receive a test combination. If not, Insightly will allow users to send their test Emails to a fraction of their Prospects and when a winner is determined, the winning combination will be sent to the remaining percentage of Prospects.

Variables

Subject line

Experiment with different wording, limited time offers, or emojis to test what Prospects respond to best.

From name

Check if your Prospects are more receptive to Emails coming from your company or from the name of an employee. Add the From name and From email address you want to use for each combination.

Content

Test different Email templates or other variations of the same template to see which gets more engagement.

Send time

Test sending Emails on specific days and times to understand when Prospects are most likely to open them.

Use cases

Use Test Emails to decide what techniques are the most effective, for example:

  • Using emojis in a subject line
  • Testing different templates
  • Altering the format and content
  • Using a personal From Name rather than a company name
  • Sending one variation in the morning and one in the evening

How to create

Please read the following articles to learn how to create these Emails:

How to create an A/B Test Email

How to create a Multivariant Email

 

Was this article helpful?