Video A/B Testing [BETA]

Increase click-through metrics and post-click conversion

For video email marketers who want to get more clicks and sell more stuff
Signup For Performance Beta

Can I use my email platform for A/B Testing my videos?

Yes; most email platforms include an “A/B Testing” feature. An advantage of using your email platform for A/B Testing your video content, is that the results are reported in a familiar format, along with your A/B Testing of subject lines and send times, if you do that too.Simply create two Playable videos, one for each variant on your email platform, and then configure your A/B test as shown in the examples below:
Mailchimp
Salesforce

Limitations of using for A/B Testing

Experience video email with our showcase of campaigns below

Complex multi-stage configuration

An email sent from your email platform is fixed at the time of send, and cannot be changed while it sits in someone’s inbox waiting to be opened. This means that A/B Testing from your email platform needs to be split into multiple stages; an initial send and testing stage during which engagement data from multiple variants is collected, then a subsequent send of the winning variant. This requires extra setup and guesswork to define the parameters of the multiple stages.

Extra campaign duration

The initial send and testing period adds extra duration to your campaign. MailChimp suggests an initial testing period of 4 hours, and Salesforce Marketing Cloud suggests 24 hours. How do you decide how much time is needed? Whatever you decide, this extra campaign duration may disrupt your sending schedules and increase your suppression lists.

Leaving clicks on the table

The initial send and testing period is also costing you clicks. If you have set the duration to 24 hours, as Salesforce Marketing Cloud suggests, but the engagement data reveals a statistically significant winner to the A/B Test within 60 minutes, the losing variants of your email will continue to be tested unnecessarily for 23 hours, meaning you are losing clicks and revenue during that time.

Lack of ongoing learning

Most email platforms consider each A/B Test as independent, and cannot use the results in any way to automatically improve your subsequent campaigns.

Introducing… Performance by Playable

To overcome the above limitations of A/B Testing your video content using your email platform, Playable is excited to be launching Performance, for video email marketers who want to get more clicks and sell more stuff.

Simple campaign configuration

No guesswork is needed on how long to conduct an initial testing stage before declaring a winner. This is possible because the Playable video content remains dynamic until the moment of open, rather than being fixed at the moment of send. Only at the time of open does Playable choose which variant of your video to deliver to each recipient, based on the up-to-the-minute engagement data.

Normal campaign duration

Your video is optimized in real-time during the normal duration of your campaign. No initial testing phase of 4 to 24 hours is required.

Mathematically proven to maximize clicks

Early beta customers have seen click through rates increase by up to 88% when sending to a list of at least 50,000 recipients.

Continual learning and improvement

Playable generates useful reports from your A/B Testing to help your subsequent campaigns.

Use Cases

Works with broadcast and segmented marketing campaigns. Playable quickly detects which variant of your video is best, and quickly starts to show that video to an increasing proportion of recipients who open your email.

Works with long-running automated and transactional emails. Playable detects when variants become old, and when other variants become newly engaging.
Works best for campaigns over 50k sends / 10k opens. Engagement uplift increases with more sends.

Simply add your video variants. No complex multi-stage configuration is required.

Simple reports show how many extra clicks have been generated by A/B Testing

Features

Test 2 to 5 video variants. More variants require more sends.

For the data scientists; a multi-armed bandit algorithm is used to learn which of your variants are best, and Thompson Sampling is used to choose which variant is delivered to each recipient when they open your email.

Signup for Performance BETA

No credit card required.

Get started For Free No Credit Card Required

You can get started with Playable for free, our 30 day trial account offers you:
l

Full access to Performance features for 30 days

50,000 videos Plays for use in A/B Testing. Videos include Playable branding

Point and Click Integration

Upgrade to the paid Performance plan to remove Playable branding
Sign Up With My Mailchimp Account

“E-commerce brands that A/B test their emails generate 20% more revenue on average.”

– Litmus

What feedback is helpful during the beta?

We’ll reach out during the beta to ask these questions:
u

What click-through results did you achieve?

u

Would you prefer to use the A/B Testing features of your ESP?

u

Would you prefer to see the A/B results displayed on Playable or within your ESP?

u

What additional reports would be useful?

u

Do you want to optimize videos based on downstream conversion behaviors such as basket and checkout events, rather than click-through rates?

u

How can the user experience be improved?

We love to hear any and all feedback

By signing up you agree to our Terms of Service and Privacy Policy.

Already have an account? Login.