What is an Automation A/B test?

When a subscriber triggers an automation, A/B testing is a way to test versions of your outgoing message to identify which version performs better.

When subscribers trigger your A/B test automation, they will receive one of the message versions you are testing. Strive will attempt to distribute each version round-robin across your audience as evenly as possible.
‍

Can A/B tests run on any automation?

Yes! Any of the automations types (keywords, integrations, API, and default) can run A/B tests.

A/B tests can be run on integrated form, including ActionKit, EveryAction, ActBlue, Salesforce, Action Network, and Facebook Lead Ads.
‍

Where can I see my A/B test results?

Review and compare A/B tests by navigating to the Automations tab of the Reports page. You can track and compare metrics like delivery rate, response rate and click rate in A/B Automation reports.

Note: The automation must be triggered at least once for the report to generate.
‍

Create an Automation A/B Test

1. Go to Automations, and navigate into one of the automations tabs, based on the type of automation you’d like to test:  keywords, integrations, API,
2.  Click Add Automation
3. Set up your automation audience type and trigger
4. Click A/B test
5. Select the message versions you would like to test
6. To add additional variants, click Add Version
   ‍
   

Pick a Winner

Picking a winner will convert your A/B test into a regular automation with the winning version you select.

1. From the Automations page, click the action menu (...) next to the A/B test
2. Click End A/B test
3. Select the winner from the dropdown
   ‍
   

Can I delete my automation A/B test once I’ve ended the test?‍

Yes, deleted A/B automation tests will disappear from the Automations table and appear as “archived” in the Reports tab.

Ending an A/B test automation will stop the test from running, and allow you to pick a winner. The automation will appear with the selected winner in the Automations table. Ended A/B tests will also appear as archived on the Reports page.

Note: Automations Reports do not contain results that took place previous to the launch date of this feature, September 16th, 2025. You can filter for previous automations interactions in the associated flow reports.