NEWS RECOMMENDATION ENGINE

THE CHALLENGE

Readers today consume news content digitally in smaller and smaller bytes, with limited attention spans. Frequently, they only peruse the article that is of interest to them, and exit the mobile/web site. A leading magazine in India wanted to personalize the user experience. The challenge , therefore, was to engage readers beyond a single article, thereby increasing their stickiness to the site.

OUR APPROACH

There are different techniques used for information filtering in recommendation systems; one is content filtering and other is collaborative filtering. Our approach was to use a hybrid of the two. We not only computed a user profile based on the content consumed and defined a neighbourhood for the user but also computed a profile for the “Average Joe” – effectively a profile that reflects composite consumption across the user base. To recommend items, we balance the users own signal with that from his neighbourhood and that from Average Joe, with weights that reflect the current state of consumption signal of the User – effectively a personalized approach to personalization (Personalization 2.0).

THE METHODOLOGY

  • Our data scientists processed articles into structured form (extract named entities, topic mix, score words).
  • We built a customer profile based on article consumption to recommend items based on similarity computing Article to Profile similarity and Profile to Profile similarity.
  • We mapped the features of articles with the features of users to obtain user- article similarity.
  • We created an average user profile taking the average of all vectors for each user.
  • A neighborhood was defined for each user based on similarities of interest.
  • Weights were assigned to each of the profile and the neighborhood after taking into account the consumption pattern of the user.
  • Article Recommendations are generated.

TECHNIQUES, TECHNOLOGIES, TOOLS

Named Entity Extraction, Part of Speech Tagging, Topic Modeling, content based filtering, collaborative filtering, implicit ratings, Elastic Search

RESULT/IMPACT

The recommendation system continues to learn and adapt automatically as it sees what the users are viewing. This ongoing improvement through machine learning enables us to understand what the user wants and ensures that the recommendations always reflect what the readers are interested in leading to a significant increase user stickiness because of a more personalized customer experience.