A/B testing in Custobar
There are three fields in Custobar that are used for testing campaigns: split_ab allows you to do ab testing with two groups of equal size. split_abc with 3 groups of equal size, and split_abcd with 4 groups.
When a customer is created in Custobar, they are placed in one of the groups A or B so that the groups are always roughly the same size. They are also placed in one of the A B or C groups and A B C or D groups.
Now the complicated part: Although they use the same letters, the group a from split_ab is not the same as the group a from split_abc or split_abcd.
That is because group A in split_ab represents 50% of all customers at any given time. Group A in split_abcd is 25% of all customers at any given time. So to keep the percentage correct, some customers from group A in split_ab will also belong to group B, C or D in split_abcd.
That is the technical part. In practice, if you want to do some ab testing, just create two drafts with the content of your choosing, and for the target audience, do your normal segmentation with the queries and filters of your choosing. When you are done, just add to the segmentation the field split_ab and choose group “a” for one campaign, and do the same and choose group “b” for the other. You can now activate both campaigns and compare the statistics depending on the criteria you are testing.
Repeat those steps with more drafts for abc or abcd testing.
A/B testing in Flow
It is also possible to use Flow to automate A/B testing in a given Customer journey.
The easiest way to do that is create two parallel branches and have an A/B Split node dispatch the customers down one of the branches depending on probability.
You can have two or more branches, and the ratio of customers sent to each can be adjusted in percentages using a sliding scale.
If some campaigns are used as part of this journey, it will be necessary to use one trigger for each group, properly labeled so that one can easily identify which branch the customer went down and trigger the matching message.
Regardless of which method is used, we strongly suggest testing only one factor (difference in content, headlines or scheduling) at a time to make sure the results are accurate.
If you have any questions, please contact Custobar support.
