Split Test node¶

A split test node is a two-part node comprised of two required parts: a Producer and a Consumer. A Producer will assign a randomly-selected variant to the transaction it is processing at that time, and save two outputs: Selected Variant and Consumer ID.

The split test node can accomplish several tasks, including the following:

Randomly select a variant for A/B or multivariate testing
Choose the winning variant that has been getting the most engagement
Reassign the probability of selecting a variant to favor the winner

For example, you can use a split test node to do the following actions:

Select Twitter responses
Select different subject lines and images for outgoing emails
Select different texts and images to display on a personalized website

For example, the following split test node has four variants: red, blue, green, and yellow. The yellow variant is displayed more frequently, and is the favored outcome:

Split Test node example

Selected Variant - The variant that was randomly selected by the node
Consumer ID - A hashed string generated by the Producer node and passed to the Consumer node to determine the winning variant

When implemented, a split test node performs the following actions:

The Producer node selects a variant and consumer ID
The Producer node pushes the variant (email, web page, twitter response) to the user
The user engages with the variant
The Consumer node tracks the user’s engagement with the variant

Clicking the Producer node in the graph displays the Producer results pane to the right of the page, which updates every 60 seconds.

Producer results

Notes

The number of iterations can be greater than the number of responses; for example, an email can be opened multiple times by the same recipient.
Counts of iterations and responses value are only available in the results panel; they are not available in the schema or public variables.

Split test options¶

You can enable one of the following options based on your use case:

Adaptive Learning
New Test on Deploy

Adaptive learning¶

By selecting the Enable Adaptive Learning check box, you can change the distribution based on the winning variant, and reassign the probability of selecting a variant to favor the winner.

Adaptive Learning enabled

Minimum Iterations Produced - The number of transactions that must be produced before variant distribution is adjusted based on adaptive learning. Default value: 1000
Minimum Responses Consumed - The number of transactions that must be consumed from the Minimum Iterations Produced before variant distribution is adjusted based on adaptive learning. Default value: 100
Selection Certainty - The confidence level required to select the winner. A lower number means the winner will be decided more quickly, but with less confidence. The winner is chosen using a Chi-squared significance test. Default value: 95

After the Minimum Responses Consumed threshold is met, the node selects variants based on the winning rank and the selection distribution. In the previous example screenshot, after 10 responses are received, a winner is chosen. The winner will be recommended 90% of the time, then second place variant will be chosen 10% of the time.

Create new test on deploy¶

If you select the Create New Test On Deploy check box, the split test node will be reset and all information (including the winning variant) learned up to that point will be reset.

Creating a split test node¶

To create a split test node as part of an existing graph, perform the following actions:

Sign in to Acquia Journey.
From the projects list, identify the project you want to create a split test node for, and click its title. Acquia Journey will display the Overview tab for your project.
Click the Graphs tab, and then click Create.
Enter the name of your new split test node in the Name field.
Scroll through the list of node types to find the Learning section, and click Split Test.
Click Create Item.

Configuring a Split Test node¶

After creating a blank split test node, perform the following actions to configure it:

Sign in to Acquia Journey.
From the projects list, identify the project you want to modify, and click its title. Acquia Journey will display the Overview tab for your project.
Click the Graphs tab.
Scroll to identify the split test node you want to modify, and click its name.
In the upper-left of the Split Test Editor section, click Create Variant.
Enter a name for the test, and then click Create Variant.

Note

After you save a variant’s name, you cannot edit the name.
To set the likelihood of this variant being selected, in the gray field to the right of the variant’s name, enter a number between 1 and 100.
Create additional variants as needed. You must have at least two variants defined for the split test node to be valid.

For the Selected Variant and Consumer ID fields, set the field value by selecting the value you want to use from the Data pane (either Schema or Public variable) and clicking its name, and then clicking the left arrow icon.

Acquia Journey saves your changes as you configure the split test node.

Adding a split test node to a graph¶

To use your split test node, you must add a producer variant and a consumer variant to your graph. The consumer variant can be added to a different graph than the producer depending upon your requirements. Complete the following steps to add the node to your graph:

Sign in to Acquia Journey.
From the projects list, identify the project you want to modify, and click its title. Acquia Journey will display the Overview tab for your project.
Click the Graphs tab.
Under the Graphs section, identify the graph you want to modify, and click its name.
Right click on the canvas, and then click Add Node to Graph.
In the Add Node To Graph dialog, scroll to find the split test node you want to add to your graph, and then click Add Node. Acquia Journey will add the consumer variant of the node to your graph.
If you want a producer variant, in the lower-right corner of the node, click the exchange icon to change to a producer variant.

Note

You should create the Producer node in the graph that determines the A/B group variant, while placing the Consumer node in the graph that tracks engagement data.

Validation warnings¶

Warning	Solution
Split test Meta Data must be set to a valid data reference	Ensure that the ‘Selected Variant’ and the ‘Consumer ID’ are saved to a location in the schema
All split tests must have at least 2 variants to be valid	Ensure that the split test node has at least two variants to randomly select from for the A/B testing
A project must have at least one consumer node per split test	Ensure that a process in the project has a Consumer node version of the split test node to identify a winning variant
A project can only have one producer node per split test	Ensure that the same split test node does not have two Producers in the same project

Looking for something?