Introduction

Users can change the criteria the application will use to determine Winners and Losers. To access the A/B Testing Settings page, navigate to Admin > Optimization, then click A/B Testing in the left-hand menu. The A/B Testing Settings will refer both to Landing Page and Creative tests.

Choosing an A/B Test Metric

Users can choose the metric that the application will use to calculate Winners and Losers. Options are:

• Conversions / Impressions
• Click-through Rate (Clicks / Impressions)
• Conversion Rate (Conversions / Clicks)

• Profit / Impression

• ROI

• Margin
• ROAS (Revenue / Ad Spend)

When determining Winners and Losers within a group, the application will use the selected metric for each creative and compare against the average for the other creatives in the group. If there is a statistical difference between a creative and the group average, that creative will be marked as a Winner or Loser.

Understanding Confidence Level

Users can select the level of confidence the application will use when determining Winners and Losers. When comparing a creative against the average of all other creatives in the group, the application will use the selected confidence level to determine if there is a statistical difference.

For example, a creative may have a conversion rate of 4% compared to the average of the other group creatives of 3%. The confidence that difference in conversion rate is statistically different is 88%*. If the application is set to use a 90% confidence level, the creative will NOT be marked as a Winner. If the application is set to use a 85% confidence level, the creative will be marked as a Winner.

Note: The application uses the Student's T-Test to calculate statistical confidence level.

Advanced Settings

• Days of Data to use
  Users can select the amount of data to use when calculating metrics. The application will use data for the selected time period when calculating metrics. Options are:
  • 7 days
  • 14 days
  • 30 days
  • 60 days
  • 90 days
     
• Minimum Impressions
  The application will not use Creatives that have received less than the selected number of impressions when calculating Winners and Losers.
   

• Minimum Clicks
  The application will not use Creatives that have received less than the selected number of clicks when calculating Winners and Losers.
   
• Minimum Conversions
  The application will not use Creatives that have received less than the selected number of conversions when calculating Winners and Losers.

Important notes

In your Group-level settings, Ad Serving should be set to Do Not Optimize: Rotate ads indefinitely. To learn more about ad rotation settings, check out our article Ad Rotation Options with Google.

The A/B Testing tool does not distinguish between Desktop and Mobile creatives. If you're using Desktop and Mobile-Preferred creatives in the same ad group, the testing tool will not be able to test desktop against destop or mobile against mobile. All creatives are tested up against one another.  

Related articles

Please refer to the following related Support Center articles for more information:

• What is Creative Testing?
   

• A Guide To A/B Test Setup Strategy
   

• How to Set Up Landing Page Tests
   

• How to Test Your Landing Page
   

• A Guide to Best Practices for Creative Testing