coollector.com
https://www.coollector.com/

Coollector compared with other movie recommendation systems
https://www.coollector.com/viewtopic.php?f=1&t=1700
Page 1 of 1

Author:  (cool) Hector [ Sat Nov 04, 2017 3:49 am ]
Post subject:  Coollector compared with other movie recommendation systems

In this lengthy post (sorry) I will explain in details how Coollector is different from the other movie recommendation systems. Quite a lot of persons are interested in receiving movie recommendations. It's quite a large market, and as a result there's quite a large offer, with various approaches...


Movie ADN

Let's begin with the approach that I like the least. Several years ago, there was a website called jinni.com. They claimed to have decrypted what they called the "entertainment genome". In other words, they had put a lot of effort tagging each movie with as many keywords as they could to describe the movie's ADN. The mood of the movie could be bleak, offbeat, cringy, clever, etc... The plot could have aliens, heist, superheroes, zombies, etc... In the end, there was a ton of criterias that you could choose from, and you picked up the ones that you were in the mood to watch.

But realistically, who in the world chooses a movie this way? Personally I've never been in the mood to watch a cringy movie with aliens. It's not the ingredients that make a great recipe. An awesome movie and an awful movie can have the exact same ingredients. What I want before all is to watch a movie that I'll like. The ingredients may count a little, but they're secondary. This service could certainly help you to discover some interesting movies, but in my opinion it was the most clumsy way to hunt them.


Movie Connections

"People who liked this also liked this..." You can see this approach on IMDb, Amazon, and many other websites. It's indeed interesting to know what other movies you might like if you liked this one. The problem is that only a handful of movies are suggested, generally famous movies which you've probably already seen. Even if they could remove the movies that you've seen, they could not suggest movies indefinitely, because it doesn't make much sense to connect a movie with hundreds of other movies. Even if the underlying idea was good, this kind of recommendation ends up being highly uneffective, and you'll be lucky if it helps you to discover more than a few movies. Personally I've never found even one movie with this method.


Machine Learning

Machine Learning is the approach that was chosen by Netflix, the undisputed leader in movie recommendations. It's a fact that Netflix recommendations have greatly contributed and keep contributing to their success as a company. To describe it simply, machine learning is a complex mathematical model which you feed with a large volume of data, which crunches that data and then extracts some meaning out of it. When they've started, Netflix was basically feeding their system with all the ratings made by their millions of users, and compared them with your own ratings in order to predict how much you would like the movies that you haven't seen yet. Nowadays, they feed their system with much more data, they dissect your behaviour, the time of the day, etc... as you can read in this article:

http://www.wired.co.uk/article/how-do-netflixs-algorithms-work-machine-learning-helps-to-predict-what-viewers-will-like

Over the years, Netflix has greatly changed the way they make recommendations, and the direction they've taken is somehow surprising. You've probably heard about the recent uproar that occured when they've dumped their star rating system in favor of a thumbs up/thumbs down system. How can they make accurate recommendations if they can't distinguish between a movie that you loved from another one that you just mildly enjoyed? Personally, I don't want to give a thumbs up to an average movie, even if it doesn't deserve a thumbs down either, and in the end it's a majority of the movies that I can't rate with the new system. This change seems counter-productive as obviously a less accurate rating system will produce less accurate recommendations.

On the other hand, Netflix now takes into account the time of the day, the day of the week, if you watched the entire movie or just the first 10 minutes, etc... Seriously? Those details are more important than an accurate rating system? Unfortunately, the answer is yes, they are, at least from Netflix point of view. Netflix has more than 100 million customers worldwide, and only a small fraction of them can be qualified as "movie lovers", the rest are just movie consumers. Netflix has simplified their rating system to engage people that didn't rate with the old system, and as a result they now collect more data this way than before. Even with the thumbs up/tumbs down, many users still don't rate at all and that's why Netflix analyzes their behaviour instead.

Because the movie lovers can't give accurate ratings anymore, the recommendations they get must be less accurate too, it's mathematical. On the other hand, the rest of the customers are probably receiving slightly more accurate recommendations with the new system. Netflix has chosen the majority over the minority, it kind of makes sense. But why not create an option for the movie lovers to allow them to keep using the old rating system if they want to? My hypothesis is that Netflix is radically shifting the focus of their recommendations, and decided to completely ditch the old system to concentrate on the new one. One could even imagine that they had already ditched the star ratings some time before and were just converting them into thumbs up/thumbs down. I think Netflix is no longer trying to make the most accurate recommendations like it was the case during the 1 Million Netflix Prize (they didn't even bother implementing the improvements that were discovered). Instead they now rather focus on strategies to keep their customers subscribed to their service while optimizing the return on investment from their content acquisitions.

Netflix has an army of the best engineers and they use a very complex technology, but we must keep in mind that they also have a big handicap which is their limited catalog of movies. There are tons of movies missing from their streaming catalog, and their recommendation system is like a horse wearing blinkers, it can't see any of those movies, it has no way to know which ones you've seen and liked, and of course it can't recommend any of them. At the time of the Netflix Prize, Netflix was still a DVD rental service, and their catalog was much more exhaustive than their current streaming catalog. But now that streaming has becomed the core of their business, it's not possible anymore for Netflix recommendation system to be the best in the world, so I think they've kind of abandoned the idea. Instead, their recommendations are more and more biased for economical reasons. For example, it's a well known fact that they're forcing their in-house productions down your throat, while some other movies are burried and hard to find.

It's difficult to evaluate the quality of the Netflix recommendations, because there's no way to measure it. We can only get a feeling, and most customers do seem happy, even though it's easy to find some other customers who complain, as well as a bunch of websites who poke fun at the most ridiculous Netflix suggestions (simply google "netflix suggestion fail"). Netflix was a pioneer in movie recommendations. They're probably still very good at it, but considering how their recommendation system has evolved, I'm inclined to think that it is getting worse instead of better.


TCI & PSI

Those 2 acronyms were coined by the website criticker.com. The TCI (Taste Compatibility Index) is the measure of how closely your taste matches that of another user, while the PSI (Probable Score Index) is your predicted rating. In simple words, they firt look at your ratings to find the other users who seem to like the same movies as you, then they look at the ratings that those users gave to movies you haven't seen yet in order to predict what your own ratings would likely be.

In my opinion (shared by some users on their forum), the Criticker system suffers from two flaws:

1) because it takes a lot of computation power, the TCIs are rarely refreshed (even though they are the foundation of their recommendations).
2) to understand your ratings, Criticker uses a tier system which is easily biased if you don't rate bad movies (unfortunately in real life people avoid bad movies as much as they can and simply don't watch them).

In spite of those issues, I've found their predictions to be surprisingly good, much better than I had anticipated. I must admit that for years I have under-estimated this website and now I understand why so many people are raving about the accuracy of their recommendations. Because the Coollector ratings convert perfectly into Criticker ratings (and vice-versa), it's possible (though very tedious) to compare the 2 systems. To get an idea, I've conducted a test... until I got too bored. Only one test isn't enough, so more testing is definitively needed to confirm the conclusions. Anyway, here are the results of this quick test, for what they're worth:

https://www.coollector.com/CritickerTest.txt

I would have loved to say that Coollector gives much better recommendations than Criticker, but the difference happens to be pretty small. To begin with, the two systems both give very good results. Criticker made roughly 50% more errors (to be confirmed with more tests), but 50% of a little, that's still not much!


Coollector's Approach

Coollector takes an approach which is very similar to both Criticker and the Netflix from the Netflix Prize days. Like them, we analyze your ratings in their globality, as a whole, and we use the ratings from all the other users to predict how much you'll like the movies that you haven't seen yet. It's worth noting that Coollector doesn't suffer from the same problems as Criticker, and you'll get all the predictions in a fraction of second, without any bias even if you rate only the movies that you liked.

I'm afraid that's about all I can say about Coollector's recommendations system. I'd love to give more details, it burns my lips because I've worked 2 years to develop this system and I'm very proud of my discoveries, but as a small company I would shoot myself in the foot if I revealed the recipe of my secret sauce.

Page 1 of 1 All times are UTC + 1 hour [ DST ]
Powered by phpBB® Forum Software © phpBB Group
http://www.phpbb.com/