Recommendation Engines: A Combination of Art & Science
Shopping experiences go beyond a rational analysis of available alternatives, yet they are guided and emotional activities. Relevant product recommendations help retailers build a deep relationship with their customers as they give them the sense of being understood and properly served. Today retailers are sold on the high value of product recommendations – otherwise would be dumb, considering one of the primary factors behind the story of Amazon becoming the multi-billion dollar behemoth it is, is its integrated recommendations. Also, it is worth reminding you of Steve Jobs’ immortal words: “a lot of times, people don’t know what they want until you show it to them.” However, many businesses are still grappling with the best way to capitalize on it and measure the return on investment. Not to worry though, as you will not be one of them after reading this piece.
When you need a recommendation system, you typically have two options: building your own or buying an existing product. Developing your own recommendation system can liberate you from relying too heavily on another company to run a core part of your business but it is an expensive initiative, and the tech talent shortage of data scientists can get in your way. In fact, a study by McKinsey projects that “by 2018, the US alone may face a 50 percent to 60 percent gap between supply and requisite demand of deep analytical talent.” After all, Netflix has spent years building and improving its recommendation engine and even sponsored a $1 million contest to improve its algorithm.
For e-commerce websites, homepage, product page, and shopping cart are the pages where a recommendations engine works most efficiently. However, the technology shouldn’t be limited to online as doling out recommendations to consumers via email also delivers a significant conversion rate and efficiency. There is a strong correlation between the number of different places showing recommendations and it’s effectiveness. One of the best ways to find out the most proper placement is running A/B testing so you could know which locations perform well. When it comes to testing, there are a number of approaches depending on your goals, and before diving into return on investment of recommendation engines, I would like to briefly explain the most common three among those approaches:
It is also known as cognitive filtering which matches up the attributes of a user profile in which preferences and interests are stored, with the attributes of a content object (item), in order to recommend to the user’s other items. At a simplistic level, the algorithm compares the content of the items such as author, price, color, location, etc with a user profile, then makes a recommendation accordingly. The content of each item, here, is entitled to a set of terms, typically the words that occur in a document. The user profile is entitled with the same terms and developed by analyzing the content of items which the user has displayed.
It is a practical approach as it can run as soon as the data about items is in the system without waiting for any user input to make recommendations. In other words, content-based recommendations are unsupervised and do not suffer from the cold-start problem. On the flip side, though, since the recommendations made based on the items’ attributes, tags, and stuff like that, rather than shoppers’ experiences personality assessment is often left out by the system. To tackle this issue, content-based systems are preferred to be used with the collaborative filtering systems which leads me to the next approach.
Item-to-item Collaborative Filtering
The wisdom of crowds would be the best term to describe the item-to-item collaborative filtering system. In the real word, we often encounter the face of the algorithm in the form of: "people who viewed/liked/bought this item also viewed/liked/bought that other item." Sounds familiar? You bet! In fact, this approach has been embraced by Amazon. Unlike traditional collaborative filtering, Amazon’s algorithm’s online computation scales independently of the number of customers and number of items in the product catalog, according to the company.
Hybrid Recommender Systems
To produce high-quality recommendations in real time and scale to massive datasets, today many recommendation engines typically tap into both of these approaches together, in some cases, even more approaches are combined into the mix. To put this into context, let’s take an e-commerce website as an example. When a product page is viewed, the recommendation engine uses content-based filtering to confine the list of possible recommendations as it provides alternatives that are contextualized to the item viewed, and then, selects the most popular options based on collaborative filtering. As a result, the odds of reaching an accurate and valid prediction is much higher with the hybrid recommender systems.
The ROI of Recommendation Engines
According to a Forrester study, over 15% of visitors admit to buying recommended products.
A personalized experience sells: As explained above, the bottom line of utilizing recommendation engines is to create an experience that makes the visitor feel understood and served well as individuals. Given that personalization is a sophisticated and complex area, there is an exchange of profitable benefit to the organizations for a little bit of time and effort being given away as it leads to greater conversions. In fact, the study, which surveyed 2,000 U.S. and UK consumers, found that 73 percent of customers prefer personalized shopping experiences. To learn more about how personalization can help and its impediments, check out the CMS-Connected article entitled “What Has Taken Personalization So Long to Become Mainstream?”
Cross-selling complimentary products generate revenue stream: Are you familiar with recommendation widgets statements like “Frequently bought together” or “People who bought this also bought”? Of course! It is not a secret that offering complimentary products to visitors boosts conversion and the volume in orders. According to Mckinsey, cross-selling techniques increase sales by 20% and profits by 30%. It also creates a value for the customers by facilitating such serendipitous discovery about products available. They might not purchase at that time but those offers definitely remain in the back of their mind for a while. At that point, if brands follow up with them on social media or any other channels by displaying those offers through online ads, there is a good chances consumers will return to the site and make a purchase. After all, Amazon stated that 35% of its revenues come from its cross-sales and upselling efforts.
Social proof prompts online sales: Have you ever heard of the idiom “keeping up with the Joneses?” It refers to the comparison to one's neighbor as a benchmark for social class or the accumulation of material goods and clearly demonstrates the profound impact of social proof on making purchases and decisions. Before the widespread adoption of social media, one of the most obvious examples to come to mind is TV shows playing canned laughter to accelerate the comical perception of scenes. Now that social media is dominating, this concept has gained even more momentum. Of course, marketers and merchandisers are not wasting any time harnessing this psychological element, and it comes to consumers in the form of a statement like “People who viewed this also viewed” The motivation behind taking action is based on this kind of statement is the fear of missing out which is, scientifically, a form of social anxiety and also known as FOMO in pop culture. The study conducted by Sharethis and Paley Centre for Media suggests that content sharing can influence consumers more than price and brand, and motivate people to spend 9.5% more.
To me, making recommendation engines work is a combination of art and science. In the era of rising online shopping, cognitive recommendation engines are becoming mainstream. However, they have a huge value not only for e-commerce but also for a wide array of industries as they simply boost customer engagement. For instance, 35 percent of what consumers purchase on Amazon and 75 percent of what they watch on Netflix come from product recommendations based on such algorithms. The technology is already sophisticated but in the near future, I believe that we will see a significant increase not only in the adoption of the technology but also in accuracy and relevancy of recommendations. On that note, I would like to end my piece with the good news that in May 2016, Amazon opened up its sophisticated AI technology as a cloud platform by unveiling DSSTNE, an open source artificial intelligence framework that Amazon developed to power its own product recommendation system, in an attempt to extend the promise of deep learning beyond speech and object recognition to other areas such as search and recommendations.