Use case: optimizing content recommendation with artificial intelligence

Linkedin logo

Published on 28 January 2025

This article was written by Constance FROMONOT, Senior Data Consultant at SQORUS, with contributions from Mahmoud, IT & Data Consultant, who both led this innovative recommendation system project.

Explore the possibilities of artificial intelligence (AI) in the field of recommendations. Our latest project aims to transform an internal technology watch application through the integration of a recommendation system.

Find out how our expertise gave rise to a successful POC (proof of concept), exploiting the MovieLens dataset to shape a robust recommendation system based on employee preferences.

The application of artificial intelligence to content recommendation continues to grow in 2024. Among the best-known are Netflix’s to recommend series/movies, and Amazon’s to recommend products you’re likely to buy.

But applications are not limited to commercial uses. In our POC, we want to develop a recommendation algorithm for an internal tool, enabling employees to monitor their various fields effectively and proactively.

The project was carried out in 5 distinct phases.

1. Data selection, definition and extraction

Lacking a pre-existing database, we carried out a POC of a recommendation system using the MovieLensdatasetavailable online. We chose this dataset because of its similarities to our future database (item genre, user-item interactions, date, several tables, etc.).

This first step required a thorough understanding of the variables, creating a solid foundation for the rest of the project.

2. Expression of hypotheses

Our approach is based on 3 types of recommendations, each with its own assumptions:

Content Based Filtering hypothesis: We assume that users are more likely to enjoy films with similar characteristics to those they have rated positively in the past.

Collaborative Filtering hypothesis: We assume that a user hasn’t watched a movie if he hasn’t rated it.

Levenshtein’s Distance Hypothesis: It is assumed that films with similar titles, even if there are slight differences or spelling mistakes, share similarities in content.

3. Exploratory analysis

The real data, although rich in information, required processing of missing values and modeling to obtain the dataset required for our study. The latter is based on the 6 tables contained in the MovieLens dataset.

We obtain the following final dataset: 20,000,263 notes given by a user to a film, 1342 film genres.

To delve deeper into the subject of Data Science and its impact on corporate functions, particularly in the HR and training fields, we invite you to read our dedicated article on the Data Science revolution in HR functions.

4. Modeling and evaluation

In this section, we have taken into account the 3 types of recommendations defined above:

Content based filtering

The principle of this algorithm is based on recommending films to users according to their characteristics (genre) and preferences.

We explored genres, converting them into a numerical representation using TF-IDF (Term Frequency-Inverse Document Frequency). The latter works by assigning weights to terms according to their frequency in a document (term frequency) and their rarity in the document set (inverse document frequency).

In our case, TF-IDF transforms film genres into a matrix, capturing the importance of each genre in describing a film’s content.

The recommendation system then uses this matrix to find similar films based on genre and content.

In addition, the code efficiently filters and sorts the list of similar films, ensuring that the recommendations provided are relevant and ranked according to their similarity scores.

Collaborative filtering

 

Collaborative filtering is a method of anticipating which films a user is likely to like, based on the reactions of other users sharing similar preferences. These types of models are highly effective in delivering personalized content while being able to adapt to changing user preferences.

To implement our collaborative filtering, we’ll opt for the k-nearest neighbor method. The process will begin with the creation of a matrix based on our data, encompassing the reactions provided by a group of users to specific elements of a set of films.

Then, to assess the similarity between two users, we’ll measure the distance between the ratings they assigned to shared movies.

These ratings for each user are considered as vectors in a multi-dimensional space, with each dimension representing a movie. Once similarities have been calculated for a target user (the one for whom a recommendation is desired) with the set of other users, we select a group of similar users constituting a neighborhood.

This choice is based on a predefined number of nearest neighbors, a parameter that can be adjusted according to functional requirements.

Distance from Levenshtein

 

We’ll use Levenshtein’s distance to identify closely related items when an application user doesn’t search for the exact name of the item in the search bar.

Here, it’s just a matter of retrieving all the individual film titles and measuring the distance between them.

Conclusion of a recommendation system

This POC combines different types of recommendations, enabling it to achieve a balanced and effective approach, responding to diverse scenarios and user preferences.

Using the same approach, you can recommend not only content to customers, but also training courses to employees, or even jobs to employees in the context of internal or external mobility.

If you’d also like to take advantage of the benefits of artificial intelligence for recommendations or any other subject, we can help you do just that.

Contact us today to discuss how SQORUS can help you integrate these innovative solutions into your strategy.

    HR Data strategy: what if we accelerated?

    Imagine a world where the HR function is propelled into a new dimension thanks to the power of data. What if this world were within our reach? Discover how to harness the full potential of HR Data to revolutionize your organization.

    Contact

    A project? A request?A question?

    Contact us today and find out how we can work together to make your company’s digital future a reality.

    Additional articles

    SQORUS logo

    To make sure you don’t miss out, sign up for our newsletter!

    Our mission

    Discover the strengths of the SQORUS strategy

    We have been able to adapt to new digital challenges, the arrival of the Cloud and changes in working methods. We have succeeded in forging strong partnerships with the main publishers in the market and in attracting business and technical experts.

    Our strength: over 300 talented people dedicated to the success of your projects and sharing strong values: diversity, commitment and solidarity, which represent real value for the company and its customers.

    Great Place to Work for 10 consecutive years, SQORUS is sensitive to the personal development of its Sqorusien.ne.s, their career development and their training in future-oriented solutions.

    SQORUS specializes in digital and business transformation for HR, Finance and IT functions. For over 30 years, our consultants have been working with major corporations on strategic, international information systems projects: development strategy, selection assistance, integration, Business Intelligence, Data Management, support and change management, as well as on Cloud and Artificial Intelligence issues.