Users can change the criteria the application will use to determine Winners and Losers. To access the A/B Testing Settings in Marin Search, navigate to Admin, select Optimization, then click A/B Testing in the left-hand menu. The A/B Testing Settings will refer both to Landing Page and Creative tests.
You can choose the metric that the application will use to calculate Winners and Losers. The options are:
Profit / Impression
ROI
When determining Winners and Losers within a group, Marin Search will use the selected metric for each creative and compare it against the average for the other creatives in the group. If there is a statistical difference between a creative and the group average, that creative will be marked as a Winner or Loser.
You can select the level of confidence the application will use when determining Winners and Losers. When comparing a creative against the average of all other creatives in the group, the application will use the selected confidence level to determine if there is a statistical difference.
For example, a creative may have a conversion rate of 4% compared to the average of the other group creatives of 3%. The confidence that the difference in conversion rate is statistically different is 88%*. If the application is set to use a 90% confidence level, the creative will NOT be marked as a Winner. If the application is set to use an 85% confidence level, the creative will be marked as a Winner.
Note: The application uses the Student's T-Test to calculate the statistical confidence level.
In your Group-level settings, Ad Serving should be set to Do Not Optimize: Rotate ads indefinitely. To learn more about ad rotation settings, check out our article Ad Rotation Options with Google.
The A/B Testing tool does not distinguish between Desktop and Mobile creatives. If you're using Desktop and Mobile-Preferred creatives in the same ad group, the testing tool will not be able to test desktop against desktop or mobile against mobile. All creatives are tested up against one another.
Please refer to the following related Support Center articles for more information: