Information for: DEVELOPERS   PARTNERS

Explainable Machine Learning

Unlocking explainability improves campaigns and transparency. It also helps customers to build trust in machine learning.

In today’s complex marketing channels, marketers need deep, nuanced insights into micro-patterns and multi-faceted customer behavior. These insights are beyond the realm of traditional analytics tools.

Machine learning models make predictions by capturing these insights as part of their operating parameters during the training process. Unlocking these parameters gives marketers direct visibility and explanations about the micro-patterns and customer behavior, while also building trust in machine learning predictions. It also enables them to build highly optimized and performant campaigns.

The following are the new available machine learning Model Metrics and their use cases:

Clustering heatmaps

Heatmaps display average values per input feature of the machine learning model that defines the composition of each cluster. It is designed to help marketers understand customers’ significant behavior identifiers in a particular cluster.

The following example helps you understand a heatmap and the insights it enables:

Heatmap Analysis

This heatmap displays the average values for customers in four clusters across all the model input features. These four clusters are:

  • VIPs
  • Regular Buyers
  • Recent Low Value Buyers
  • One-time Buyers

In this example, model input features are arranged by rows, and clusters are arranged by columns. The average value of each input feature by cluster is represented as a numerical value.

Learning from this heatmap chart

VIP customers have bought many more products compared to regular buyers and low value buyers, and have a higher Average Order Value (AOV). However, they have received relatively low overall discounts, despite their loyalty.

You can use this insight to create a discount campaign that focuses on VIP customers, and bundle it with a deal to increase AOV and drive volumes.

Similarly, another insight is that VIP customers buy fewer items on their first purchase, compared to regular buyers and one-time buyers. However, the percentage margin on these goods is higher, which indicates that they are buying fresh/new products.

Feature importance

Feature importance charts are used to identify the weights of the model input features towards the model prediction. In other words, these charts are used to identify the impact and contribution of the input features towards the model predictions.

The charts surface positive, negative, and low contributing input features.

By focusing on what features positively and negatively affect a customer’s prediction score, marketers can build optimized campaigns by tapping into these features.

The following example helps you understand a feature importance chart and the insights it enables:

Feature Importance Analysis

This chart displays the feature importance value or weights for all the input features for a Likelihood to Buy model.

In this example, model input features are arranged on the left and the feature importance values are arranged on the right. Positive values (shades of Blue and Violet) have a positive correlation to the prediction, and negative values (shades of Red) have a negative correlation.

Learning from this feature importance chart

Likelihood to Buy scores are positively correlated with the transactions count in the last 30 days and email open ratios.

At the same time, they are negatively correlated with email click ratio For example, if people are not clicking in the email, likelihood to buy will be low.

Therefore, to maximize conversion, a marketer should design campaigns targeting customers who made a purchase in the last 30 days and exceed a certain email click threshold. Additionally, the total revenue contributed by an individual has a very low bearing on their likelihood to buy and should not be used by the marketer as a basis for any campaign.