It is currently Thu Mar 28, 2024 9:29 pm

All times are UTC + 1 hour [ DST ]




Post new topic Reply to topic  [ 10 posts ] 
Author Message
 Post subject: Coollector compared with other movie recommendation systems
PostPosted: Sat Nov 04, 2017 3:49 am 
Offline
User avatar

Joined: Tue Sep 27, 2005 2:28 am
Posts: 7632
In this lengthy post (sorry) I will explain in details how Coollector is different from the other movie recommendation systems. Quite a lot of persons are interested in receiving movie recommendations. It's quite a large market, and as a result there's quite a large offer, with various approaches...


Movie ADN

Let's begin with the approach that I like the least. Several years ago, there was a website called jinni.com. They claimed to have decrypted what they called the "entertainment genome". In other words, they had put a lot of effort tagging each movie with as many keywords as they could to describe the movie's ADN. The mood of the movie could be bleak, offbeat, cringy, clever, etc... The plot could have aliens, heist, superheroes, zombies, etc... In the end, there was a ton of criterias that you could choose from, and you picked up the ones that you were in the mood to watch.

But realistically, who in the world chooses a movie this way? Personally I've never been in the mood to watch a cringy movie with aliens. It's not the ingredients that make a great recipe. An awesome movie and an awful movie can have the exact same ingredients. What I want before all is to watch a movie that I'll like. The ingredients may count a little, but they're secondary. This service could certainly help you to discover some interesting movies, but in my opinion it was the most clumsy way to hunt them.


Movie Connections

"People who liked this also liked this..." You can see this approach on IMDb, Amazon, and many other websites. It's indeed interesting to know what other movies you might like if you liked this one. The problem is that only a handful of movies are suggested, generally famous movies which you've probably already seen. Even if they could remove the movies that you've seen, they could not suggest movies indefinitely, because it doesn't make much sense to connect a movie with hundreds of other movies. Even if the underlying idea was good, this kind of recommendation ends up being highly uneffective, and you'll be lucky if it helps you to discover more than a few movies. Personally I've never found even one movie with this method.


Machine Learning

Machine Learning is the approach that was chosen by Netflix, the undisputed leader in movie recommendations. It's a fact that Netflix recommendations have greatly contributed and keep contributing to their success as a company. To describe it simply, machine learning is a complex mathematical model which you feed with a large volume of data, which crunches that data and then extracts some meaning out of it. When they've started, Netflix was basically feeding their system with all the ratings made by their millions of users, and compared them with your own ratings in order to predict how much you would like the movies that you haven't seen yet. Nowadays, they feed their system with much more data, they dissect your behaviour, the time of the day, etc... as you can read in this article:

http://www.wired.co.uk/article/how-do-netflixs-algorithms-work-machine-learning-helps-to-predict-what-viewers-will-like

Over the years, Netflix has greatly changed the way they make recommendations, and the direction they've taken is somehow surprising. You've probably heard about the recent uproar that occured when they've dumped their star rating system in favor of a thumbs up/thumbs down system. How can they make accurate recommendations if they can't distinguish between a movie that you loved from another one that you just mildly enjoyed? Personally, I don't want to give a thumbs up to an average movie, even if it doesn't deserve a thumbs down either, and in the end it's a majority of the movies that I can't rate with the new system. This change seems counter-productive as obviously a less accurate rating system will produce less accurate recommendations.

On the other hand, Netflix now takes into account the time of the day, the day of the week, if you watched the entire movie or just the first 10 minutes, etc... Seriously? Those details are more important than an accurate rating system? Unfortunately, the answer is yes, they are, at least from Netflix point of view. Netflix has more than 100 million customers worldwide, and only a small fraction of them can be qualified as "movie lovers", the rest are just movie consumers. Netflix has simplified their rating system to engage people that didn't rate with the old system, and as a result they now collect more data this way than before. Even with the thumbs up/tumbs down, many users still don't rate at all and that's why Netflix analyzes their behaviour instead.

Because the movie lovers can't give accurate ratings anymore, the recommendations they get must be less accurate too, it's mathematical. On the other hand, the rest of the customers are probably receiving slightly more accurate recommendations with the new system. Netflix has chosen the majority over the minority, it kind of makes sense. But why not create an option for the movie lovers to allow them to keep using the old rating system if they want to? My hypothesis is that Netflix is radically shifting the focus of their recommendations, and decided to completely ditch the old system to concentrate on the new one. One could even imagine that they had already ditched the star ratings some time before and were just converting them into thumbs up/thumbs down. I think Netflix is no longer trying to make the most accurate recommendations like it was the case during the 1 Million Netflix Prize (they didn't even bother implementing the improvements that were discovered). Instead they now rather focus on strategies to keep their customers subscribed to their service while optimizing the return on investment from their content acquisitions.

Netflix has an army of the best engineers and they use a very complex technology, but we must keep in mind that they also have a big handicap which is their limited catalog of movies. There are tons of movies missing from their streaming catalog, and their recommendation system is like a horse wearing blinkers, it can't see any of those movies, it has no way to know which ones you've seen and liked, and of course it can't recommend any of them. At the time of the Netflix Prize, Netflix was still a DVD rental service, and their catalog was much more exhaustive than their current streaming catalog. But now that streaming has becomed the core of their business, it's not possible anymore for Netflix recommendation system to be the best in the world, so I think they've kind of abandoned the idea. Instead, their recommendations are more and more biased for economical reasons. For example, it's a well known fact that they're forcing their in-house productions down your throat, while some other movies are burried and hard to find.

It's difficult to evaluate the quality of the Netflix recommendations, because there's no way to measure it. We can only get a feeling, and most customers do seem happy, even though it's easy to find some other customers who complain, as well as a bunch of websites who poke fun at the most ridiculous Netflix suggestions (simply google "netflix suggestion fail"). Netflix was a pioneer in movie recommendations. They're probably still very good at it, but considering how their recommendation system has evolved, I'm inclined to think that it is getting worse instead of better.


TCI & PSI

Those 2 acronyms were coined by the website criticker.com. The TCI (Taste Compatibility Index) is the measure of how closely your taste matches that of another user, while the PSI (Probable Score Index) is your predicted rating. In simple words, they first look at your ratings to find the other users who seem to like the same movies as you, then they look at the ratings that those users gave to movies you haven't seen yet in order to predict what your own ratings would likely be.

In my opinion (shared by some users on their forum), the Criticker system suffers from two flaws:

1) because it takes a lot of computation power, the TCIs are rarely refreshed (even though they are the foundation of their recommendations).
2) to understand your ratings, Criticker uses a tier system which is easily biased if you don't rate bad movies (unfortunately in real life people avoid bad movies as much as they can and simply don't watch them).

In spite of those issues, I've found their predictions to be surprisingly good, much better than I had anticipated. I must admit that for years I have under-estimated this website and now I understand why so many people are raving about the accuracy of their recommendations. Because the Coollector ratings convert perfectly into Criticker ratings (and vice-versa), it's possible (though very tedious) to compare the 2 systems. To get an idea, I've conducted a test... until I got too bored. Only one test isn't enough, so more testing is definitively needed to confirm the conclusions. Anyway, here are the results of this quick test, for what they're worth:

https://www.coollector.com/CritickerTest.txt

I would have loved to say that Coollector gives much better recommendations than Criticker, but the difference happens to be pretty small. To begin with, the two systems both give very good results. Criticker made roughly 50% more errors (to be confirmed with more tests), but 50% of a little, that's still not much!


Coollector's Approach

Coollector takes an approach which is very similar to both Criticker and the Netflix from the Netflix Prize days. Like them, we analyze your ratings in their globality, as a whole, and we use the ratings from all the other users to predict how much you'll like the movies that you haven't seen yet. It's worth noting that Coollector doesn't suffer from the same problems as Criticker, and you'll get all the predictions in a fraction of second, without any bias even if you rate only the movies that you liked.

I'm afraid that's about all I can say about Coollector's recommendations system. I'd love to give more details, it burns my lips because I've worked 2 years to develop this system and I'm very proud of my discoveries, but as a small company I would shoot myself in the foot if I revealed the recipe of my secret sauce.


Top
 Profile  
 
 Post subject: Re: Coollector compared with other movie recommendation syst
PostPosted: Thu Dec 21, 2017 4:50 pm 
Offline
User avatar

Joined: Thu Jan 06, 2011 2:29 am
Posts: 707
Location: Jamaica
Damn that was a good read...


Top
 Profile  
 
 Post subject: Re: Coollector compared with other movie recommendation syst
PostPosted: Thu Dec 21, 2017 5:01 pm 
Offline
User avatar

Joined: Tue Sep 27, 2005 2:28 am
Posts: 7632
Thank you :D


Top
 Profile  
 
 Post subject: Re: Coollector compared with other movie recommendation syst
PostPosted: Sun Apr 08, 2018 11:41 pm 
Offline

Joined: Sun Apr 08, 2018 11:36 pm
Posts: 3
Why not take the extra time for more accurate recommendations? Web sites must pay for every second of processing time while the user can just take a break. It could make the difference and have your program stand out from the many movie recommendation sites.


Top
 Profile  
 
 Post subject: Re: Coollector compared with other movie recommendation syst
PostPosted: Mon Apr 09, 2018 12:14 am 
Offline
User avatar

Joined: Tue Sep 27, 2005 2:28 am
Posts: 7632
ran88dom99 wrote:
Why not take the extra time for more accurate recommendations? Web sites must pay for every second of processing time while the user can just take a break.

The recommendations are calculated in just a few seconds because I've added a ton of optimizations, all the optimizations that I could think of. The first version was 50 times slower. Optimizations are an obsession of mine. I've spent months refining the algorithm, sometimes to gain only a few percents. Also the app is written in C++, the fastest language, and you'd be amazed at how many things today's computers can do in a second.

I assure you that the recommendations are as accurate as I could make them. More computing time won't make them any more accurate. They only get more accurate from one version to the next as I gather more and more data to crunch. They also get more accurate as you rate more movies, as it allows the system to understand your tastes better.


ran88dom99 wrote:
It could make the difference and have your program stand out from the many movie recommendation sites.

I think that my recommendation system already stands out. How long have you tried it? How was your experience with it? Do you know an alternative that gives better recommendations? You can speak frankly.


Top
 Profile  
 
 Post subject: Re: Coollector compared with other movie recommendation syst
PostPosted: Mon Apr 09, 2018 12:41 am 
Offline

Joined: Sun Apr 08, 2018 11:36 pm
Posts: 3
https://alternativeto.net/list/2554/rec ... end-user-/

MovieLens by U.Minnesota is only MOOC for recsys. Its good, at least it's 6 genre are great. Movix use LSTM recurrent neural network and is really good too. I will definitely try coollector.


Top
 Profile  
 
 Post subject: Re: Coollector compared with other movie recommendation syst
PostPosted: Mon Apr 09, 2018 12:44 am 
Offline
User avatar

Joined: Tue Sep 27, 2005 2:28 am
Posts: 7632
ran88dom99 wrote:
I will definitely try coollector.

Then we'll be able to talk. I've tried dozens of other systems, and I prefer mine by far, but I'm biased.

I'll gladly listen to your user experience.


Top
 Profile  
 
 Post subject: Re: Coollector compared with other movie recommendation syst
PostPosted: Mon Apr 09, 2018 1:08 am 
Offline
User avatar

Joined: Tue Sep 27, 2005 2:28 am
Posts: 7632
ran88dom99 wrote:
Movix use LSTM recurrent neural network and is really good too.

I didn't know about Movix. I gave it a try, but only after a couple of minutes I already have a problem with it... You can only like a movie or not like it! I've started by liking my favorite movies, but now I'm reluctant to give the same like to movies that are clearly less good. I'm now presented with a list of movies that could all deserve a like, but what would be the point? I won't even try to do it as it's a P.I.T.A. having to wait several seconds before I can like another movie. In my opinion, the shuffling animation is an annoying gimmick. I'm not even sure that this is a recommendation system, it could simply be listing the most popular movies. Sorry, this system isn't for me, and I don't understand for whom it is. Please help me understand and tell me why you like it and how it can be useful.


Top
 Profile  
 
 Post subject: Re: Coollector compared with other movie recommendation syst
PostPosted: Mon Apr 09, 2018 1:20 am 
Offline
User avatar

Joined: Tue Sep 27, 2005 2:28 am
Posts: 7632
(cool) Hector wrote:
In my opinion, the shuffling animation is an annoying gimmick.

OMG, the animation is in fact the least problem. The more you rate, the longer it takes, and very soon it's incredibly sloooooooow. More than 10 seconds after rating only a handful of movies! Sorry, I have a life to live.

I've tested a lot of systems. This one is ridiculous.


Top
 Profile  
 
 Post subject: Re: Coollector compared with other movie recommendation syst
PostPosted: Mon Apr 09, 2018 1:34 am 
Offline
User avatar

Joined: Tue Sep 27, 2005 2:28 am
Posts: 7632
OMG, I'm asked to rate movies that I've already rated. Is it a joke?


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 10 posts ] 

All times are UTC + 1 hour [ DST ]


Who is online

Users browsing this forum: Bing [Bot] and 39 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  


Powered by phpBB © 2024 phpBB Group