Algorithm for suggesting products

Question

What's a good algorithm for suggesting things that someone might like based on their previous choices? (e.g. as popularised by Amazon to suggest books, and used in services like iRate Radio or YAPE where you get suggestions by rating items)

Answer 1

Simple and straightforward (order cart):

Keep a list of transactions in terms of what items were ordered together. For instance when someone buys a camcorder on Amazon, they also buy media for recording at the same time.

When deciding what is "suggested" on a given product page, look at all the orders where that product was ordered, count all the other items purchased at the same time, and then display the top 5 items that were most frequently purchased at the same time.

You can expand it from there based not only on orders, but what people searched for in sequence on the website, etc.

In terms of a rating system (ie, movie ratings):

It becomes more difficult when you throw in ratings. Rather than a discrete basket of items one has purchased, you have a customer history of item ratings.

At that point you're looking at data mining, and the complexity is tremendous.

A simple algorithm, though, isn't far from the above, but it takes a different form. Take the customer's highest rated items, and the lowest rated items, and find other customers with similar highest rated and lowest rated lists. You want to match them with others that have similar extreme likes and dislikes - if you focus on likes only, then when you suggest something they hate, you'll have given them a bad experience. In suggestions systems you always want to err on the side of "lukewarm" experience rather than "hate" because one bad experience will sour them from using the suggestions.

Suggest items in other's highest lists to the customer.

Answer 2

Recommended products algorithms are huge business now a days. NetFlix for one is offering 100,000 for only minor increases in the accuracy of their algorithm.

Answer 3

Consider looking at "What is a Good Recommendation Algorithm?" and its discussion on Hacker News.

Answer 4

Market basket analysis is the field of study you're looking for:

Microsoft offers two suitable algorithms with their Analysis server: Microsoft Association Algorithm Microsoft Decision Trees Algorithm

Check out this msdn article for suggestions on how best to use Analysis Services to solve this problem.

link text

Answer 5

There isn't a definitive answer and it's highly unlikely there is a standard algorithm for that.

How you do that heavily depends on the kind of data you want to relate and how it is organized. It depends on how you define "related" in the scope of your application.

Often the simplest thought produces good results. In the case of books, if you have a database with several attributes per book entry (say author, date, genre etc.) you can simply chose to suggest a random set of books from the same author, the same genre, similar titles and others like that.

However, you can always try more complicated stuff. Keeping a record of other users that required this "product" and suggest other "products" those users required in the past (product can be anything from a book, to a song to anything you can imagine). Something that most major sites that have a suggest function do (although they probably take in a lot of information, from product attributes to demographics, to best serve the client).

Or you can even resort to so called AI; neural networks can be constructed that take in all those are attributes of the product and try (based on previous observations) to relate it to others and update themselves.

A mix of any of those cases might work for you.

I would personally recommend thinking about how you want the algorithm to work and how to suggest related "products". Then, you can explore all the options: from simple to complicated and balance your needs.

Answer 6

I think doing a Google on Least Mean Square Regression (or something like that) might give you something to chew on.

Answer 7

just thinking out-loud:

you need to calculate correlation between everyone and everyone (O^2?) - if your pattern of good and bad ratings matches those by someone else, then it can suggest their high-rated things to you

but how does this work if you have only a few data points?

ratings need to be normalised - a rating of 2* by someone who rates everything else as 1* is clearly a yes vote, while a rating of 2* by someone who rates everything else as 4*-5* is more like a down-vote

how to stop spammers rating everything of theirs highly and competitors badly? maybe the correlation system does that anyway - if the spammers don't correlate with your ratings then their suggestions are devalued

Answer 8

CPAN module Math::Preference::SVD is apparently a "Preference/Recommendation Engine based on Single Value Decomposition"

Answer 9

I think most of the useful advice has already been suggested but I thought I'll just put in how I would go about it, just thinking though, since I haven't done anything like this.

First I Would find where in the application I will sample the data to be used, so If I have a store it will probably in the check out. Then I would save a relation between each item in the checkout cart.

now if a user goes to an items page I can count the number of relations from other items and pick for example the 5 items with the highest number of relation to the selected item.

I know its simple, and there are probably better ways.

But I hope it helps

Answer 10

As you have deduced by the answers so far, and indeed as you suggest, this is a large and complex topic. I can't give you an answer, at least nothing that hasn't already been said, but I an point you to a couple of excellent books on the topic:

• Programming CI: http://oreilly.com/catalog/9780596529321/ is a fairly gentle introduction with samples in Python.

• CI In Action: http://www.manning.com/alag looks a bit more in depth (but I've only just read the first chapter or 2) and has examples in Java.