Recommendations across digital channels

The problem

The client wanted personalized product recommendations on its main website and webstore, across several product lines, run as a product that keeps improving rather than a one-off project.

What I did

I owned the product vision and roadmap and led a team of seven across design, data science, machine learning and software engineering, from the design phase to continuous improvement in production. The recommendations appeared where they could affect a purchase: on product listing pages and in the cart and side cart.

Different places got different kinds of recommenders. Some updated on the fly from what the visitor was doing, in the current session and in earlier ones when we had them. Others were more static: the cart recommenders looked at what was already in the basket and suggested the items most often bought together with it.

How we knew it worked

We measured the impact with an A/B test against a holdout. One group of visitors saw no recommendations; the other saw them on listing pages and in the cart. We compared add-to-cart rate, revenue per session and items purchased per session between the two. The test ran for about six weeks in an off-peak season with no other promotions running, so nothing else muddied the comparison. Projected over a full year, it comes to about €800,000 in new sales.

What I learned

Even a strong value hypothesis doesn’t guarantee results in the real world. So we always ran several experiments at once. That turned “admit failure” into “choose which one to continue”, and nobody lost credibility when an idea didn’t pan out.