The Creative A/B testing tool automatically analyzes performance to identify any creatives over/under performing relative to the ad group average, based on user settings. The success criteria and settings for creative AB tests (testing the performance of different ad copies) and landing page AB tests are determined at the level of the Marin account.
Note: Creative testing settings do not currently exist in MarinOne, so your tests should still be set up in Marin Search. However, you can still view the results of your creative tests in the grids in MarinOne by following the steps listed below.
Creative testing is a built-in Platform feature that gives you the confidence and control you need to make smarter decisions about your creatives. It allows you to easily compare the performance of one creative to another, giving you valuable real-world data you can use to improve your campaign strategy.
You can use the A/B testing tool to choose from a number of criteria by which to test your creatives. For example, you can compare them by conversions & impressions, clicks & impressions, Return On Investment (ROI), and many more. Once you've chosen your settings, you'll be able to use the A/B Testing filter in the main Creatives grid to check the status of your creatives at any time. You can filter for Winner, Loser, or a Draw. You can also use the View Builder to bring the A/B Test column into the grid and see the per-creative results at-a-glance.
Note that to get the best results from creative testing, we recommend that you have your ads set to Do Not Optimize: Rotate ads indefinitely in your group-level settings. To learn more about ad rotation settings, check out our article Ad Rotation Options with Google.
When determining Winners and Losers within an ad group, the Platform will check the selected criteria (e.g. Conversions / Impressions) for each creative, then compare the result against the average for the other creatives in the group. If there is a statistical difference between a creative and the group average, that creative will be marked as a Winner or Loser. If there's no statistical difference, it'll be marked as a Draw.
Note: A draw can also mean that there's not yet enough data for the Platform to declare a winner; for example, if the thresholds or confidence level haven't been met.
When setting up your creative testing, you'll notice the Confidence Level section of the settings page.
It works like this: When comparing a creative against the average of all other creatives in the group, the Platform will use your chosen Confidence Level to determine if there is a significant difference or not. The Platform uses an algorithm known as the Student's T-test to calculate the statistical confidence level between two values (for example, the conversion rate of two creatives). If you're not sure about this setting, we recommend leaving it at the default value of 90%.
Before we dive into the details of setting up your Creative Test, let's take a look at a few best practices.
Note: Any differences between ads will cause Marin to see these ads as distinct for testing purposes. This includes capitalization in headlines.
There are two parts to using creative testing with the Platform: A/B testing setup and analyzing the results. We'll go through each of these below.
The first thing to do in order to use creative testing with your campaigns is to set up the testing criteria. To do that, just follow these simple steps:
Once your creative test has been running for your chosen period of time, it's time to check the results. You can analyze your results in a variety of different ways in Marin Search or MarinOne.
As a starting point, sort your active groups by highest spenders for the month. Bring in a Bubble chart and identify the high spend groups with the lowest conversion rate (revenue return) and CTR. These are prime starting points for A/B testing.
Use the tool to identify best performing ad copy for these groups. Pause the Losers. Clone and create message variations of Winners.
Additionally, utilize dimensions by tagging new creatives with time stamps to easily track progress. If you’re testing the same copy across multiple groups or campaigns, give them one tag within a dimension to determine aggregated best performers.
Note: At this time, Marin only offers creative testing at the creative level. Testing cannot be performed at the group or campaign level. These settings may, however, be available on the channel themselves (on Google or Microsoft's ad site) and we do offer Dimensions to aggregate your creative-level data in any group you would like.
You can also use the View Builder to see the results of your creative testing live in the grid. Just follow these steps:
Creative testing settings do not currently exist in MarinOne, so your tests should still be set up in Marin Search. However, you can still view the results of your creative tests in the grids in MarinOne by following the steps listed below.
In your AB Testing settings, we recommend using Profit/impression for clients with revenue data and Conversions / Impressions for those without. Of course, you should choose the metric that best suits your account needs, but these should serve as a best practice. The recommended confidence level setting is between 85-95% to determine a clear winner. Advanced settings are available for click, impression and time threshold minimums.
