A/B Testing for Video: What Variations to Test

We recently shared the first post in our A/B Testing for Video series, on what A/B testing is. Now that you’re in the know, and understand why you should be A/B testing your marketing videos, it’s time to start thinking about what elements of your videos you should test.

As we mentioned in our last installment, we think it’s worthwhile to A/B test as often as you can — sometimes your original video will perform best, but sometimes one simple tweak can make a major difference.

For example, when adventure company WV Skydivers posted a Facebook video ad, they A/B tested two versions – one that started with a photo and one that started with a video clip. The variation that started with a video clip reached over 10K people, while the variation that started with a photo reached only 1.2K. Had they only run the variation with the photo they may have missed out on more than $10K in new bookings that resulted from the campaign. Check out both the winning and losing variations in the video below.

Variations to Test

Ready to start testing your marketing videos? We’ve put together some lists of different variations you can test. And yes, there are loads of things that you can test when it comes to video — the possibilities are endless. But don’t get overwhelmed. Remember that with each test you do, you’ll learn something that you can take with you going forward. Testing helps you learn and improve your video strategy all the time. For instance, after the A/B test above, WV Skydivers knows to start with video clips instead of photos going forward. That leaves the opportunity, with their next campaign, to test something different to make their videos even more effective.

Media Variations to Test

The first set of test variations that we’ll take a look at are media variations. Like the WV Skydivers example above, these tests involve making changes to the photos and/or video clips in your video. Here are some ideas:

• Type of media: Swap out a photo for a video clip or vice versa, or try using text on a plain background versus text on top of a photo. This test can be especially important when it comes to capturing viewers’ attention in the first 3 seconds of your video.
• Content of clips or photos: Instead of testing a photo against a video clip, try testing two different photos or two different video clips against each other. In our previous A/B testing post, we shared our holiday video as an example. We tested the same video with three different opening clips. One clip (Variation 2) outperformed the other variations, so that was the video we chose to share with our customers for the holidays.

• Order of clips or photos: Rather than swapping out photos or video clips entirely, try simply switching up the order of your video. Perhaps an eye-catching title, photo, or video clip that you have at the end of your video would work better if it was at the beginning.

Text Variations to Test

The text in your videos is also a great thing to test — especially when you’re posting videos on Facebook, where 85% of videos are viewed with the sound off. There are a lot of different ways you can vary the text in your videos for testing:

• Text background: Try testing text on video footage, compared to text on photos, or text on a plain colored background.
• Text design: Change up your text color, text size, or font. Once you find a combination that performs well, consider using it to brand all of your videos. For example, here are two title style variations we could test for the video we posted above.

• Copy variations: Sometimes the way that you say something makes a big difference. For instance, “Merry Christmas” and “Happy Holidays” can be used to share the same message, but one may resonate more with your audience than another. You’ll never know which if you don’t test!
• Call to action: The call to action in your video can be the difference between a click through to your website or a scroll by in the news feed. A subtle change from “Learn More” to “Visit Us” could make a big difference. Again, you’ll never know if you don’t test.
• Descriptions: If you don’t have time to make changes to your video, you can still A/B test your campaigns using different descriptions and calls-to-action within Facebook’s interface (or wherever else you may be setting up your campaign).
• Captions vs. no captions: You can also test videos with or without subtitles (that you create in your video, or that you use Facebook’s captioning feature to add). Captions are generally a good idea in any case when you’ve got a voice-over or on-camera talent that can’t be understood with the sound off.

Testing Video Length

You can also create different lengths of the same video to see which performs better. In this example, vegan chocolate purveyor Rescue Chocolate ran two version of the same ad – one 0:15 seconds and one 0:30 seconds. The 0:15 second ad reached nearly 15,000 people and received nearly 500 reactions, comments, and shares. The 0:30 second ad reached 13,000 people and received just over 400 reactions, comments, and shares. Learn more about their story on our success stories page.

Audio Variations to Test

If your video has audio – whether it’s a voice-over, someone talking on camera, or simply a music track, you can test different versions. Here are some ideas:

• Music: See if one song performs better than another.
• Script: Do a test to see whether saying things in different words changes the results of your campaign.
• Actor: Does your video perform better if your voice or on-screen actor is male or female? Does an accent make a difference? You can change things up to see how it affects the success of your video. If you find a specific person or voice that outperforms the others, stick with this person for your videos going forward. You can also test different on-camera actors before settling on a single on-camera personality for your company videos.

Getting Started

Before we move on we just want to say, don’t get overwhelmed! We realize that we’ve listed a lot of potential variations to test. Don’t worry. You don’t need to test every single one of these things all at once. With each video, make a hypothesis about a single change that might make a difference in your results and test that one thing. The results of each test will help you improve your video marketing strategy, a little bit at a time.

Homework Assignment:

Last time we asked you to think about how and where you’re currently using video and what your intended goals for your videos are. In anticipation for the next post in our series, start thinking about what variations you can test for each of these videos. Try to come up with variations that you really think will lead to more engagement or capture viewers attention more effectively. In the next installment of our A/B Testing for Video series, we’ll show you how to create video variations for testing, using Animoto.

Explore the other posts in this series:

What is A/B Testing? Creating Video Variations with Animoto Setting Up a Test in Facebook Measuring Results in Facebook