« Why Long Tails need filters, Calvin and Hobbes version | Main | Highbrow *and* Despicable »

October 03, 2006

TrackBack

TrackBack URL for this entry:
http://www.typepad.com/services/trackback/6a00d8341bfb6353ef00d834641b6869e2

Listed below are links to weblogs that reference Why better DVD recommendations are worth a million bucks:

Comments

Hashim

When I saw this competition, I thought to myself - when will this be covered on the LT blog? :)

In general, are the economics better for most products down the Long Tail, or is Netflix unique?

Bob

Netflix has another advantage in squeezing customers 'down the tail': it is a particularly bad way to rent new releases. This effect should be accounted for somehow. I am guessing that the number of subscribers who rent new releases from 'guaranteed in stock' B&M places is not insignificant but I could be wrong.

Thus, while its stats do indicate the importance of niches the causes cited for this - namely variety and findability - may not tell the whole story. Perhaps the supply of new releases vs. demand should be included.

Netflix's queue functionality works to drive this, too, by automatically sending a different movie when #1 (etc) is not available. Since new releases seem more likely to be 'out of stock' it would be possible for one customer to receive many non-new releases before receiving one new release and thus squeezing more and more area into the tail.

Ultimately, if people are satisfied than it doesn't matter. Everybody wins and the emphasis on improving the recs is right on.

David

I'll bet the folks at Netflix feel that the million $ prize itself (and the publicity it generates) will generate enough new business that it doesn't matter if anyone improves their recommendations enough to win it. In fact, if nobody wins they'll have gotten an enormous boost in public attention without spending a penny. Pretty smart.

Matt C

"by 10% (measured by correspondence to customer ratings)"

Could someone explain this a little better?

Ian

Why is Netflix a bad way to rent new releases? They'll ship out a new release the day before it releases so it arrives at your door the day it hits the open market, and then I can keep it as long as I want. Seems pretty efficent to me.

Netflix does a good job of keeping valuable customers happy. I think people that see "out of stock" on new releases a lot are the ones that aren't actually profitable to Netflix due to the volume that they rent. In a very real sense, Netflix has no interest in making these customers happy. I rent 4-6 movies a month on the 4-out at a time plan and haven't seen "out of stock" in years, so I assume they want to keep me.

Jakob Nielsen

Matt: "10% improvement" presumably means that if users' average rating of recommended films was 4.1 with the old system, then it should be at least 4.5 with the new system.

Actually, there are many ways of calculating differences between average ratings, and it will be almost impossible to achieve a 4.5 rating if the scale goes from 1-5, because of the ceiling effect of the top rating.

A more fair way would be to say that out of the users who didn't give the top rating in the past, at least 10% should give at least one point more with the new system. In other words, throw out those users who already gave the top rating, because they can't give a higher rating, even if they are more happy.

Another point:

It makes sense that the rating for recommended DVDs is higher than the rating for "interest" DVDs. In the latter case, the system only knows that one data point about the user's interests, whereas in the former, it knows all his/her previous rentals as well as the satisfaction score they provided if they rated these previous rentals.

However, it's a bit of a puzzle that search should score lower than recommendations, because search is something the user is actively seeking out. One option is that the search engine is really bad and doesn't find the films that the user really wants. (I don't know if they sort the search results by recommendation score, but that would be an obvious thing to try if it's not done.)

Another option, of course, is that people simply don't know what they want. I will be willing to believe that, but I still think that the quality of the search engine should be the first suspect, given how poorly search performs in user testing of most other e-commerce sites.

Bob Carpenter

The contest rules clearly state the 10% criterion in terms of root mean square error (RMSE). According to their FAQ, guessing the global average scores 1.05 on this metric and their current system, Cinematch, based on linear regression, scores 0.95. I'm surprised the baseline's so good.

Rick Presley

I'm wondering if NetFlix will see Amazon do an end run with their Unboxed service. Now all you have to do is download videos. No messing with mailers and actual DVDs. Amazon will allow you to download to two PCs and two portable devices for $1.99 per episode or you can order a whole season.

Right now their selection isn't that great, but downloadable and portable video (that can play on your DVR through your home theater system) may sound the deathknell to not only Blockbuster, but NetFlix as well.

My guess is that NetFlix already has a plan to leapfrog Amazon. Or at least I hope so. It's also interesting to see my DVD player doesn't have a chance to gather dust before being replaced by a DVR device which may be replaced by a wireless network card from my PC straight into the TV. For that matter, the DVR, PC and TV may all morph into a single system all controlled from my cell phone.

Jean-Baptiste Rudelle

At the end of the day, 10% increase in their predictions is worth much more than $ 1 million.
But then, it depends on the bargaining power of each party.

Felix McGlade

Your premise that better recommendations are worth $1M to Netflix is wrong. It necessarily has to be worth only $50K per year to make sense to Netflix. See, Netflix is playing a trick on the contestants. The bar for $1M is so high that it's highly improbably that anyone will achieve it given the data. However, Netflix will get the best results that thousands of people around the world can achieve, fueled by dreams of $1M, for just $50K a year. Even a 9% improvement still only gets 50K. This is a brilliant twist on crowd-sourcing.

Greg Banville

Ian said:

I think people that see "out of stock" on new releases a lot are the ones that aren't actually profitable to Netflix due to the volume that they rent. In a very real sense, Netflix has no interest in making these customers happy.

This is what puts potential customers like me off Netflix. I'm accustomed to being treated as if I were a valuable customer wherever I shop and I really don't like the idea that using a service too much is going to get me treated like second class. Such as by being put on the slow list or whatever. It isn't the customers who decided to call it an "unlimited" number of dvds per month. The customers aren't lining up to trade in their ration tickets for food either, they are paying, and deserve to know up front exactly what they can get for what they pay instead of being underserved by Netflix at the company's discretion, as they reserve the right to do in their terms of service. At least today the customers who read the TOS know what they are getting into, but it took a lawsuit to even get Netflix to own up to their practices even in the fine print.

I look forward to competition between Netflix, Amazon and Blockbuster Online, maybe one of the services will distinguish itself by being a but more honest about what their customers are getting for what they are paying.

Margaret  Hedstrom

Chris, I enjoyed your talk yesterday at the I-School conference in Ann Arbor. I agreed with most of what you had to say, but I would like to pose a question. My sense from your talk was that if entrepreneurs can figure out how to exploit the "after" or "hidden"market, most of the niche content will be taken care of. My question is this: What is the role of public institutions (libraries, archives and museums) in serving the niche market? Haven't these institutions always been supporting niche interests because market solutions did not work?

Hunter McDaniel

Another reason Netflix wants to drive demand down the long tail is that this puts them in a better bargaining position with the content providers. Netflix has the least leverage on big new releases, which a significant number of subscribers want RIGHT NOW. For older or niche releases Netflix can afford the risk of not reaching agreement with owner and thus is in a better position to dictate terms.

Greg, I would suggest to you that the TOS conditions you refer to have no practical impact on the vast majority of subscribers, and that those few who are affected still get a very high level of service. Any business that offers 'unlimited' service has strategies to keep the very heaviest users from breaking the model; it just happens to be easier to see with Netflix.

Eddie Schneider

I was shocked, shocked to read about video store closings in Manhattan today...

san

I look forward to competition between Netflix, Amazon and Blockbuster Online, maybe one of the services will distinguish itself by being a but more honest about what their customers are getting for what they are paying条码条码机条码打印机 仓库管理条码枪

The comments to this entry are closed.

Tidbits

The Long Tail by Chris Anderson

Notes and sources for the book

FREE was available in all digital forms--ebook, web book, and audiobook--for free shortly after the hardcover was published on July 7th. The ebook and web book were free for a limited time and limited to certain geographic regions as determined by each national publisher; the unabridged MP3 audiobook (get zip file here) will remain free forever, available in all regions.

Order the hardcover now!