Netflix Prize Winners Announced – But What Should Their Next Prize Be Focusing On?

The Netflix prize, whose winner was announced yesterday, was either an innovative method of crowdsourcing previously specialised and disparate skills, or a way of palming off some nerds with $1m in exchange for priceless intellectual property, depending on your cynicism. The LoveFilm-esque company set up the challenge to developers to improve their recommendation software by 10%, and dangled a $1m prize as a carrot. 

The problem for Netflix was that the existing algorithm had plateaued in terms of insight into each of our viewing choices – if you’d rented Bride Wars and The Ugly Truth, then you’d get other morally redundant empty-calorie romcoms recommended to you, but the algorithm couldn’t get beyond that. When the various teams of programmers gunning for the prize set to work on the problem, they made some quick breakthroughs, but then hit a wall. The problem was films like Lost In Translation and Napoleon Dynamite, that proved to be Marmite for Netflix users – people loved it and hated them in equal measure, but there was no parity between the types of people who loved it and hated it. Accursed human caprice!

But the maddening complexity of free will was nevertheless somewhat ironed out, and two teams, BellKor and Ensemble, managed to crack to 10% barrier in July. BellKor managed to file their successful model ten minutes earlier though, so they get the money. Among BellKor’s innovations was the idea of looking at whether users rated movies at all rather than how highly, as an indicator of worth, and only at reviews that were made shortly after the film was watched, which are seen as more accurate than those made later.

BellKor go away with more than just the cash though, as their team originally included members from US telecoms giant AT&T and Yahoo; AT&T are planning to use what they learned during the race in their own video-recommendation engines. In the slightly unfortunate words of BellKor’s team manager: “You need to think outside the box, and the only way to do that is find someone else’s box” – collaboration with as many people as possible was key to pushing the results home, and past the many off-duty mathematicians working alone into the night in their garages. They combined with two other leading teams, from Graz University in Austria and Canadian software developers Broadsoft, to pool resources; the team only met in the flesh for the first time when picking up the prize yesterday. Ah, the real world – only useful for photo-ops with outsized cheques.

The prize-as-solution-stimulant model has grown in popularity recently, with the X Prize prompting the development of the spaceship that has become Virgin Galactic’s vehicle of choice, and currently poking people into developing gene sequencing, fuel efficient cars and a lunar lander. Netflix meanwhile is so chuffed with the idea of other people clubbing together and solving their problems that they’ve launched their next prize. Players will be furnished with 100m bits of data, with which they’ll have to model “taste profiles” of users.

The whole thing oscillates between fascinating and obnoxious. On the one hand, more accurate recommendations lock customers into a more trustworthy relationshop with Netflix, while providing a service that’s ever more sympathetic to one’s personal taste – everybody wins. On the other hand, I want to start giving simultaneous 5 stars to Step Up 2 The Streets and Werckmeister Harmonies just to freak out an algorithm whose sole purpose is to reduce and distil the mad, perverse chaos of human choice. Irony, the glory of genuine awfulness, the blindness to style that genuine good taste signifies – these are all things beyond the ability of maths today.

The data from the first contest also throws up some interesting ideas about the subjectivity of art, and what we like about movies. While there were the aforementioned Marmite movies, which highlight the infinitely variegated shades of human taste, there are some movies that are so universally derided that it’s tempting to see them as truly awful, that some art is inherently bad rather than an objective lump filtered through human taste. The Stepford Wives, Gigli, Sky Captain, Battlefield Earth – the world chimes as one like never before: “these are terrible movies”.

But you get the feeling that others on the hate list – Birth, Solaris, Full Frontal, In The Cut – are despised, and drive people to vent their hate in a feedback forum on Netflix, because they’re not what was expected. These are films where established, establishment stars (Kidman, Clooney, Soderbergh and Meg Ryan) do weird uncomfortable things; for viewers, there’s nothing worse than being unexpectedly driven out of your comfort zone. Surely this is the factor that algorithms need to search for – acknowledging a film not as good or bad, but as formulaic or abrasive.

Posted by Ben Beaumont-Thomas in Sci-tech | September 22, 2009 12:39PM |