In business, we constantly test ideas, products and hypotheses. So why not apply such tests to your digital content?

A/B testing a video can be very beneficial. You can test for conversions, engagement or other important metrics to optimize a desired outcome.

As a marketer, you will run two campaigns using two different videos and measure the metrics for which you want to optimize. If one of the videos has a greater click-thru rate, one of your primary metrics, that video is your winner. Over time, A/B testing can help make your content more efficient and tell you more about your audience. Below are tips to remember when A/B testing videos.


Before starting, make sure you have a clear plan. Understand the types of data that you’ll track and know how to attain statistical relevance. To determine how much traffic you must generate to reach significance, use this calculator. For more information on developing a plan, check out the video below.


When your plan is in place, start implementing the tests. First, find baseline metrics. How does the video perform as it currently exists? What metrics would you like to improve? When you understand a video’s current performance, you can develop hypotheses to test. When considering potential optimizations, write down your ideas as if/then statements. The image below offers more detail about this.

ab testing chart


When you have written your hypotheses, start by testing just one variable. A variable is any element that can be modified to produce a user’s desired response. If you change more than one thing in any test, you won’t know which variable produced the result. Here are variables you may want to employ when A/B testing videos.

  • Video length
  • Video copy
  • Thumbnails/preview screens
  • Calls to action

Of potentially hundreds of variables to consider when trying to optimize digital content, start with those that are easily modified and that you believe will give you the biggest lift in key metrics.


If you hurry tests or rush data, you could hurt key metrics and spend more money in the long run. If your statistical relevance number is 1,000 users to produce 95 percent certainty, don’t stop testing after only 100 users and think that you have the correct answer.


No single tool works for everyone’s tests because so many tests are necessary. However, powerful products will help if you evolve into multivariate tests or decide to A/B test many videos simultaneously. We like to use these:

Depending on the complexity of your testing, one of these should suffice. While Google Analytics can seem complicated, it’s the least expensive and fastest starter.

A/B testing can have a direct impact on the bottom line of your business. Often, you’ll find that just a couple of simple changes have a great impact on conversions.

The only way to know with certainty is to develop an A/B testing plan, form intelligent hypotheses and carefully test them until you’ve attained desired results. If you have questions about A/B testing or need help with video needs, reach out to us at