AB testing in Custobar
There are three fields in Custobar that are used for testing campaigns: split_ab allows you to do ab testing with two groups of equal size. split_abc with 3 groups of equal size, and split_abcd with 4 groups.
When a customer is created in Custobar, they are placed in one of the groups A or B so that the groups are always roughly the same size. They are also placed in one of the A B or C groups and A B C or D groups.
Now the complicated part: Although they use the same letters, the group a from split_ab is not the same as the group a from split_abc or split_abcd.
That is because group A in split_ab represents 50% of all customers at any given time. Group A in split_abcd is 25% of all customers at any given time. So to keep the percentage correct, some customers from group A in split_ab will also belong to group B, C or D in split_abcd.
That is the technical part. In practice, if you want to do some ab testing, just create two drafts with the content of your choosing, and for the target audience, do your normal segmentation with the queries and filters of your choosing. When you are done, just add to the segmentation the field split_ab and choose group “a” for one campaign, and do the same and choose group “b” for the other. You can now activate both campaigns and compare the statistics depending on the criteria you are testing.
Repeat those steps with more drafts for abc or abcd testing.
We strongly suggest testing only one factor (difference in content, headlines or scheduling) at a time to make sure the results are accurate.
If you have any questions, please contact [email protected]