Recommendation systems work best when you can provide them with as many relevant examples as possible. On the other hand, increasing the number of products or offers leads to recommendation blind spots, especially early on in training the system. This cold-start problem is a challenge for most recommendation systems. SAS Customer Intelligence 360 has developed a hybrid approach that alleviates some aspects of the cold-start problem for visitor-centric recommendations.

Available methods

Recommendations in SAS Customer Intelligence 360 use product or offer views as a proxy for explicit ratings used by traditional recommendation systems. The more that products are viewed together by similar users, the stronger the association is between these products (and the users) and the greater the interest. There are three basic methods used for calculating recommendations, and one that combines the other three:

• Visitor centric – currently uses a collaborative filtering approach based on Singular Value Decomposition matrix factorization. This is pretty standard and is not discussed in detail here.
• Product centric – currently uses graph theory-based link analysis and I describe this approach in this post.
• Popularity based – used as a control group for comparison, products are selected based on overall popularity.
• Hybrid recommendations – a combination of the above three (explained in more detail below).

The user can select between the first two methods when setting up a recommendation task. The two methods typically support slightly different use cases. For example, the product-centric recommendations might be used on a generic product page, but visitor-centric recommendations might be used on a basket view page.

Hybrid recommendations

Sometimes, when a marketer has set up a task to provide visitor-centric recommendations, little is known about the visitor. This is true of a new visitor or visitors who have only viewed a small number of products in the past. In this case, you will find that the recommendations generated by the visitor-centric system are of limited value. SAS Customer Intelligence 360 uses a hybrid system to determine recommendations when not enough is known about a visitor to be confident in the output of collaborative (visitor-/product-centric) filtering.

The idea is simple ‒ if you don’t know enough about the visitor, then you can revert to showing product-centric recommendations.

Research shows that about seven product views provides enough information to make collaborative filtering-based offers. Customers with six or less receive product-centric offers, and those with seven or more get visitor-centric recommendations.

Popularity-based recommendations – the control group

An important question to answer about any personalization or recommendation system is whether it works and provides value. It is hard to answer this question based on goodness of fit or similar measures (though precision and recall values on a validation set can provide an indication). The only way to really measure the benefit is to put it in practice and compare the results of recommender-based offers against an alternative business strategy.

A common, but basic, strategy for making recommendations is to show the most popular products. SAS Customer Intelligence 360 also provides an easy way to compute the most popular products and rank them. This is mostly done to provide a control group to compare visitor- and product-centric recommendations against. But sometimes, when not enough products can be found using these techniques, you can also add some popular products to the generated list.

Calculating popular products from the collected data is simple. For each product, count the total number of times it has been viewed, purchased, etc. and sort the list and assign each product a rank. Then offer the top n products by rank.

Execution

Training and execution are independent activities. Training involves creating product communities and the collection of associated metrics. Operationalization requires retrieving, for a given item or visitor-item combination, a list of close, or similar, products. It also means generating an appropriate control group product list based on popularity and applying post-processing logic such as white and black lists.

The following diagram shown the logic flow when deciding what sort of product recommendations to make to a visitor.

Backfill process

Sometimes the final number of recommendations is fewer than the marketer requested. Reasons for this include (for product-based recommendations):

• The community or category the product belongs to is very small.
• A white list black list application has removed too many products from a community or category.
• The product has weak links with other products and is removed from a link analysis or got assigned to solitary community.

In these cases, a post-processing backfill technique is used to fill the list to the required minimum. This is true for both the link-based recommendations (LBR) and the popularity-based recommendation (PBR) lists. The following table shows all the different possibilities (assuming the marketer has requested 10 products to recommend).

Using this strategy, you can ensure that you always have the correct number of products to recommend to a visitor.

Summary

SAS Customer Intelligence 360 uses a hybrid system to deal with the situation when not enough is known about a visitor to be confident in the output of collaborative filtering. This helps avoid the cold-start problem associated with visitor-centric recommendations. The system uses product views as a proxy for explicit ratings and uses network theory to form communities of similar products to draw recommendations from. It also provides a mechanism for producing control groups of products based on simple popularity measures. The marketer can always be confident that they will get a full complement of high-quality products to recommend through our intelligent backfill process. The recommendations also respect white and black list post-processing logic.

In a related follow-up post, I’ll explain in more detail how the link-based product recommendations work, how they differ from approaches based on association analysis and why we think they are an exciting step forward in recommender technology.