An Introduction to Recommender Systems (+9 Easy Examples)

Personalization.

The Holy Grail of the Internet.

Gone are the days when anonymous shoppers browsed generic stock for an elusive item. Today, anyone can serve customers unlimited, personalized offers tailored to their interests.

All you need is data, right?

Not quite.

With all the options online, you need a system that narrows down the possibilities. Something that learns what people like.

That’s where recommender systems come into play.

Implementing a recommender system allows you to turn raw data into personalized offers. And personalized offers result in higher customer satisfaction, engagement, and sales.

Sounds good, right?

That’s why the following article will tell you:

• What recommender systems are and how they work.
• Different strategies for implementing recommender systems.
• How to check if a recommender system is effective.

Already know that you need a recommender system for your project? We can help! At Iterators, we design, build, and maintain custom software and apps for startups and enterprise businesses.

Schedule a free consultation with Iterators. We’re happy to help you find the right solution.

What is a Recommender System?

So, let’s start with the basics:

What is a recommender system?

A recommender system is a type of information filtering system. By drawing from huge data sets, the system’s algorithm can pinpoint accurate user preferences. Once you know what your users like, you can recommend them new, relevant content. And that’s true for everything from movies and music, to romantic partners.

Netflix, YouTube, Tinder, and Amazon are all examples of recommender systems in use. The systems entice users with relevant suggestions based on the choices they make.

Recommender systems can also enhance experiences for:

• News Websites
• Computer Games
• Knowledge Bases
• Social Media Platforms
• Stock Trading Support Systems

And the list is not exhaustive. Bottom line? If you want to provide users with targeted choices, recommender systems are the answer.

Here’s an example of a recommender system in e-commerce. H&M served the following recommendations to users who clicked on “pleated skirt” as a potential buy:

Why Adding a Recommender System to Your Website is Beneficial

So, what are the advantages of adding a recommender system to your website or software?

Here’s a list of just a few:

• Increase in sales thanks to personalized offers.
• Enhanced customer experience.
• More time spent on the platform.
• Customer retention thanks to users feeling understood.

A recent study by Epsilon found that 90% of consumers find personalization appealing. Plus, a further 80% claim they are more likely to do business with a company when offered personalized experiences.

The study also found that these consumers are 10x more likely to become VIP customers, who make more than 15 purchases per year.

The moral of the story? If you’re interested in cross-selling or serving personalized offers, a recommendation system is right for you.

How to Solve the Long Tail Problem with Recommender Systems

Another positive benefit of using a recommender system is that you solve the long tail problem of online shopping.

When you go into a brick-and-mortar store, you only see a limited number of items to buy. There’s only so much space, right?

So, recommending customers things to buy is easy. At the front of the store, you put your newest, most popular goods on display. Walk into a bookstore and there are three tables:

• New York Times Best Sellers
• Hot New Vampire Books
• Popular Autobiographies

Take your pick.

Now, go online. Boom! you’re presented with millions of options. And if you don’t know what you’re looking for, you might find the sheer number of options overwhelming.

The problem is known as the long tail problem.

So, how does an online retailer help customers suffering from information overload?

Recommender systems.

So, let’s say you want to buy a book. You go online to Amazon and the first thing you see:

It’s the same as the display tables in the brick-and-mortar stores. But once you start making choices on the platform, Amazon’s recommender system takes over. Let’s say you search for The Great Gatsby. Amazon recommends:

Here the system served you Fahrenheit 451. That’s because past Fitzgerald customers must have also bought Bradbury. As an alternative, your recommender system could offer other Fitzgerald books.

How Recommender Systems Provide Users with Suggestions

Using machine learning, recommender systems provide you with suggestions in a few ways:

• Collaborative Filtering
• Content-based Filtering
• Hybrid (Combination of Both)

“Hybrid recommender systems are the future, blending personalization with robust guardrails. As AI ethics, alignment, and regulation gain importance, these approaches ensure that recommendations are not only accurate but also responsible, balancing user preferences with ethical considerations.”Jacek GłodekFounder @ Iterators

Jacek Głodek

Founder @ Iterators

Collaborative Filtering Recommender Systems

For starters, popular examples of collaborative filtering systems include Spotify, Netflix, and YouTube. But what does a collaborative filtering recommender system do?

A collaborative filtering recommender system analyzes similarities between users and/or item interactions. Once the system identifies similarities, it serves users recommendations. In general, users see items that similar users liked.

There are different types of collaborative filtering systems including:

• Item-item Collaborative Filtering
• User-user Collaborative Filtering

Item-item Collaborative Filtering

An item-item filtering algorithm analyzes product associations taken from user ratings. Users then see recommendations based on how they rate individual products.

For example, you rate a book or movie as a 10/10. Now, you will see the top-rated books or movies with similar attributes. Below is an example from Goodreads.

I created a special list for books that I gave five-star ratings. Goodreads then recommends me the highest-ranked books from similar readers’ lists.

It’s not always easy to get users to give items ratings. That’s why item-item filtering can be as simple as clicking on a dress and seeing more dresses.

Ever come across the “people who viewed this item also bought” copy under a product?

That’s right. That’s also an item-item filtering system.

Amazon invented item-item filtering for their recommender system. Item filtering works best when you have more users on your platform than items.

User-user Collaborative Filtering

The other kind of collaborative filtering takes the similarity of user tastes into consideration.

So, user-user collaborative filtering doesn’t serve you items with the best ratings. Instead, you join a cluster of other people with similar tastes and you see content based on historic choices.

Let’s say you use YouTube for the first time. You play a Beyonce song. The system clusters you with other users who also like Beyonce. Then the YouTube recommendation system shows you other videos chosen by users in your cluster. The more choices you make, the more relevant the results.

Ever wonder why your YouTube playlist gets messed up after a party?

It’s because Brenda decided to karaoke Disney songs for two hours. Brenda chose Disney songs, now you’re clustered with the Disney kids.

Thanks, Brenda.

Content Based Recommender Systems

Content based filtering uses the characteristics or properties of an item to serve recommendations. Characteristic information includes:

• Characteristics of Items (Keywords and Attributes)
• Characteristics of Users (Profile Information)

Let’s use a movie recommendation system as an example. Characteristics for the item Harry Potter and the Sorcerer’s Stone might include:

• Director Name – Chris Columbus
• Genres – Adventure, Fantasy, Family (IMDB)
• Stars – Daniel Radcliffe, Rupert Grint, Emma Watson

A content based recommender system can now serve the user:

• More Harry Potter Movies
• More Adventure, Family, or Fantasy Movies
• More Chris Columbus Movies
• More Daniel Radcliffe Movies

Of course, the list is not exhaustive. Once the user makes choices, the recommender system can serve more targeted results.

Let’s say they choose Swiss Army Man next. It’s safe to assume the user likes movies starring Daniel Radcliffe. The system tracks these choices and begins to recommend films starring Daniel Radcliffe.

The system may also show the user more Harry Potter movies. The hypothesis is that if a user liked an item in the past, they might like similar items in the future.

Content based filtering systems can also serve users items based on users’ profiles. You can create user profiles based on historical actions. You can also ask users upfront about their interests and preferences.

Pro Tip: Using a hybrid recommender system allows you to combine elements of both systems. In general, that means elements of one system can remedy the pitfalls of the other.

Pitfalls of Different Types of Recommender Systems

And now for the bad news. Each type of recommender system has its own set of problems. Let’s take a look.

Types of Recommender Systems Problems – The Collaborative Filtering Problem

Collaborative filtering needs a lot of data to create relevant suggestions. So, when you start using a platform with a collaborative filtering system, you start cold.

The cold start problem in recommender systems is common for collaborative filtering systems.

For example, when John visits YouTube for the first time, the system has to wait for him to watch several videos. Only then can it serve him relevant recommendations for other videos.

Types of Recommender Systems Solutions – The Collaborative Filtering Solution

A solution to the cold start problem in recommender systems is clustering data with attribute similarities. Let’s go back to our YouTube example.

John visits YouTube for the first time. The first video he selects is a Beyonce video. As mentioned before, the platform will cluster John with other users who watched the same video.

It could also add him to other clusters. Let’s say the video belongs to the “pop song” cluster. Needless to say, the pop song cluster is populated with pop songs, such as “Hit Me Baby One More Time” by Britney Spears.

Now, the system can recommend other songs based on the following criteria:

• Other Beyonce Listeners’ Choices
• Other Songs in the Pop Cluster

Another solution is to start by recommending users popular items.

YouTube and Amazon’s homepages both show users trending or popular items. IMDb has a similar strategy, showing new visitors the top-rated 250 film titles.

A final strategy is to request new users to provide the platform with information.

Types of Recommender Systems Problems – The Content Based Filtering Problem

The problem with content-based recommender systems is that they are restrictive. You click on a dress and you see more dresses. The system is incapable of knowing that your interests go beyond liking dresses.

Types of Recommender Systems Solutions – The Content Based Filtering Solution

Again, a common solution is to ask users upfront about what kind of things they like. And as users interact with your site, you can use historical data to recommend them more tailored choices.

The customer buys a dress and some shoes. Now, you know that she likes both.

How to Implement a Recommender System

There are a couple of ways to go about adding a recommender system to your software or website. But first, you need to find the right people.

Usually, implementing a recommender system comprises three activities done by different people. These activities include:

• Designing and evaluating a recommendation model. 
• Scheduling system updates and piping data into the model and out to the user.
• Integrating the recommender system with the company’s “business system.”

Here’s a basic flowchart of a recommender system:

To oversee these activities, you would need to hire (respectively):

• Data Scientists (Design and Evaluation)
• Data Engineers (Updates and Data Pipeline)
• Web and Front-end Programmers (Integration with Website)

Let’s say you want to hire in-house people who will implement and manage your recommender system. Expensive? Yes. Here are the average salaries for each according to Glassdoor:

• Data Scientist – $139,840 per year
• Data Engineer – $151, 307 per year
• Web Developer – $93,402 per year

Not to worry. You don’t have to create a recommender system from scratch. There are ready-made SaaS solutions for recommender systems in e-commerce.

And for movie or music recommendation systems, there are off-the-shelf solutions. For example, it is possible to get an algorithm similar to the one that runs Netflix’s recommendation system.

Depending on your needs, you could also consider outsourcing. Outsourcing is beneficial because it enables flexibility and can be cheaper. Plus, you’re sure that you’ve hired people who know what they’re doing, and you don’t have to worry about employee turnover or recruitment.

Pro Tip: You don’t want to hire a data scientist before you have data. They won’t have any work and they will leave or get stuck doing the work of a data engineer. So, it’s best if you hire a data engineer first.

Need to hire a programmer but don’t know the first thing about programming? Not sure how to get started when you’re looking for a data specialist? Then read our article: How to Hire a Programmer for a Startup in 6 Easy Steps

Getting Started – Gathering Data for Your Recommender System

First, you need to have your data engineer gather data for your recommender system.

As you can see in the chart, there are different datasets you might want to consider like:

• User Behavior or User Ratings
• Item Attributes

User data is necessary for serving user-item recommendations via a collaborative filtering recommender system. It does require you to have access to a large number of user interactions.

To get user data, you can either ask for ratings or draw conclusions from user behavior. Asking for ratings is problematic because most users don’t bother to give items ratings. Drawing conclusions from user behavior is problematic because of the cold start problem.

As mentioned before, some solutions include:

• Recommending Popular Items from the Start
• Onboarding Users through Profile Creation
• Clustering Users with Similar Users

Second, you can design an attribute ontology to gather item attributes for item-item recommendations. If you have limited metadata from users, you can use item attributes to create a content-based recommender system.

Either way you’ll need your web developer to integrate both front and back-end features into a supported system. Once that’s in place, the process runs continuously.

Remember, the more data you have and the fresher it is, the better your recommender engine.

Designing and Building a Recommender System for Your Business

Once you have data, your data scientists design a way to use it to build recommendations.

Building a utility matrix using matrix factorization techniques for recommender systems is a popular way to get started.

Do keep in mind that there are many ways to build recommendation systems. The following example is a highly simplified version of what a system looks like in reality.

The technique allows you to complete unknowns on a matrix based on user-item interactions.

Here’s a very basic example of a utility matrix for recommender systems:

When using a utility matrix, you should assume that the data will be sparse with more blank spaces than rankings. The goal of the matrix is to predict how John and Jane will rank the remaining films. Utility matrices organize rankings users assign to different items. In the example above, John gave Monty Python and the Holy Grail the highest score, while assigning The Meaning of Life a two. He didn’t rank Life of Brian.

The example above doesn’t give much insight into whether Jane will like either of the two films she did not rank. But in a real-life scenario, you would have much more data.

A real recommender system would compare her score with hundreds of similar scores. Patterns begin to emerge. For example, let’s say people who gave Life of Brian a 3 gave Holy Grail a 5 while giving The Meaning of Life a lower score.

The logical conclusion? Jane will like Holy Grail more than The Meaning of Life as the others did. So, let’s recommend her Monty Python and the Holy Grail.

Regardless, building a utility matrix requires large amounts of data. And that data is always going to be sparse, so your recommendation system algorithms will need to account for that.

Once the system is in place, data engineers flood the system with vast amounts of data. The model then runs in real time, preparing batches of recommendations for users.

Pro Tip: To populate a utility matrix with behavioral data, you use “1” for “like” and “0” for no action taken. Using 0 doesn’t mean the user dislikes that thing, only that she took no action.

Serving Users Recommendations and Evaluating Performance

Finally, you’ll serve users recommendations through a user interface or email campaigns. Your web developer integrates your recommender system with your website for that purpose.

Once users interact with those recommendations, guess what?

Your data engineers have even more data to feed your recommender system. Your data scientists can start collecting performance data to test your recommender system.

You’ll need to test for effectiveness so you can continue to improve recommendations.

How to Evaluate if a Recommender System is Effective

How do you know if your new recommender system is up to snuff? Well, there are a few ways to measure the effectiveness of your recommender engine:

• User Studies / Personas
• Offline Recommender System Survey
• Online Recommender System Survey (A/B Testing)

But first, you might want to take a look at the different features of recommender systems. You can use features as metrics for benchmarking.

For example, coverage is the degree to which you cover all available items and actions with your system. It’s a useful metric for making sure your system is thorough. Keep in mind that not every feature is suitable for every type of recommender system.

• User Preference: When systems make recommendations based on user interests, habits, and goals.
• Prediction Accuracy: When systems make accurate predictions about the results of serving a recommendation to a user.
• Coverage: The degree to which recommendations cover all available items and actions.
• Confidence: The system can tell the user how confident it is about its recommendations.
• Trust: How much users trust the system. Often, if a system explains it’s recommendations and they are reasonable, the user trusts the system.
• Novelty: When the system recommends items that users did not know about prior.
• Serendipity: How surprising the user finds the recommendations.
• Diversity: The more diverse the recommendations, the broader the offer for users.
• Utility: Measure of how useful the recommendation is for the user.
• Risk: When the recommender system can tell the user the risk of following a recommendation.
• Robustness: The stability of the system in the face of abuse or fake information.
• Privacy: Despite users willingly giving information to recommender systems, they still want that information to be private.
• Adaptivity: When the recommender system can adapt and serve users relevant recommendations even when the content and environment is dynamic.
• Scalability: The system can handle a growing amount of data.

Once you’ve decided which features you need, you’ll want to make sure they’re working. You can do that by testing your system.

User Studies as a Way to Evaluate Recommender Systems

In the beginning, you will have no real knowledge about your users or their interests. So, how do you create personalized offers for people you don’t know?

The easiest way to test what will be effective is to create user personas. There are various assumptions you can make about your future users before they start using your platform.

Let’s say you sell books.

You can create personas based on choice prediction. Michael selects a book. With some statistical accuracy, your recommender system can predict whether Michael will like another book. You could base it on the genre, author, and other content attributes.

You can create personas along these interest paths to serve users initial recommendations.

A good way to assign user personas is to have new users answer a few brief onboarding questions. And that’s especially true for mobile applications.

Here’s an example of Foursquare’s onboarding process:

Good onboarding questions might also include requests for demographic data about your users. Knowing simple information like age and location targets your recommendations. If your reader is 15, she is going to be much more tempted by a Twilight recommendation than if she’s 80.

One free and easy way to get some basic insight into user demographics is to use Google Analytics. Most of you already have Google Analytics hooked up to your website. So, why not use it to give you insight into user demographics?

Google Analytics shows you the age, gender, and interests of the people visiting your website. You could also use social media analytics to get ideas for user personas based.

To use Google, you click on “Audience” and then “Demographics” to view the age, gender, and interests tabs. Enabling the advertising features can give you a deeper look into user demographics.

Offline Recommender System Survey

It is important to note that offline recommender system evaluations are often treated with skepticism. That’s because you use offline methods before the launch of a recommender engine. So, instead of involving users, you run simulations.

And the results of these simulations may not correlate with real user satisfaction. The responsibility of running a simulation will fall to those who created your recommender system.

Online Recommender System Survey (A/B Testing)

Once you launch your recommender system, you run online evaluations. These evaluations comprise various A/B tests that you serve to users live.

If you’re unfamiliar with A/B testing, it involves serving different options to users who arrive at the same point. One user sees option “A” while the other sees option “B.” Whichever option does better, that’s the one you continue to serve all users.

By running A/B tests, you can see which recommendation inspires more clicks and conversions. While ideal, in the long run, this form of testing can have a negative impact on revenue and user experience if users don’t like either option.

Conclusion

Internet users and online shoppers have come to expect personalized experiences. At the very least, they want websites to make recommendations so they don’t waste time sifting through things they don’t like.

Recommender systems are a great way for any business to personalize their offers. While implementing a system may be costly, you’re sure to benefit from targeting your content.

Whether your goal is to keep people on your platform (YouTube) or show users exciting, new offers (Amazon) – a recommender system is the answer.