Introduction

Users can change the criteria the application will use to determine Winners and Losers. To access the A/B Testing Settings in Marin Search, navigate to Admin, select Optimization, then click A/B Testing in the left-hand menu. The A/B Testing Settings will refer both to Landing Page and Creative tests.

Choosing an A/B Test Metric

You can choose the metric that the application will use to calculate Winners and Losers. The options are:

• Conversions / Impressions
• Click-through Rate (Clicks / Impressions)
• Conversion Rate (Conversions / Clicks)

• Profit / Impression

• ROI

• Margin
• ROAS (Revenue / Ad Spend)

When determining Winners and Losers within a group, Marin Search will use the selected metric for each creative and compare it against the average for the other creatives in the group. If there is a statistical difference between a creative and the group average, that creative will be marked as a Winner or Loser.

Understanding Confidence Level

You can select the level of confidence the application will use when determining Winners and Losers. When comparing a creative against the average of all other creatives in the group, the application will use the selected confidence level to determine if there is a statistical difference.

For example, a creative may have a conversion rate of 4% compared to the average of the other group creatives of 3%. The confidence that the difference in conversion rate is statistically different is 88%*. If the application is set to use a 90% confidence level, the creative will NOT be marked as a Winner. If the application is set to use an 85% confidence level, the creative will be marked as a Winner.

Note: The application uses the Student's T-Test to calculate the statistical confidence level.

Advanced Settings

• Days of Data to use
  You can select the amount of data to use when calculating metrics. Marin Search will use data for the selected time period when calculating metrics. The options are:
  • 7 days
  • 14 days
  • 30 days
  • 60 days
  • 90 days
     
• Minimum Impressions
  Marin Search will not use Creatives that have received less than the selected number of impressions when calculating Winners and Losers.
   

• Minimum Clicks
  Marin Search will not use Creatives that have received less than the selected number of clicks when calculating Winners and Losers.
   
• Minimum Conversions
  Marin Search will not use Creatives that have received less than the selected number of conversions when calculating Winners and Losers.

Important notes

In your Group-level settings, Ad Serving should be set to Do Not Optimize: Rotate ads indefinitely. To learn more about ad rotation settings, check out our article Ad Rotation Options with Google.

The A/B Testing tool does not distinguish between Desktop and Mobile creatives. If you're using Desktop and Mobile-Preferred creatives in the same ad group, the testing tool will not be able to test desktop against desktop or mobile against mobile. All creatives are tested up against one another.  

Related articles

Please refer to the following related Support Center articles for more information:

• A Guide To A/B Test Setup Strategy
   

• How to Set Up Landing Page Tests
   

• Creative Testing [Marin Search]