The Long Tail

Bad news: we're flooded. Good news: I won a Loeb

I'm staying a farmhouse in the middle of England for a family reunion and, as luck would have it, it's the wettest June in recorded British history, with widespread flooding and unseasonable cold. A nearby town is underwater and they're helicoptering people off roofs. Our tennis is really suffering.

A ray of sunshine was just what we needed, and fortunately that appeared this morning in my inbox. I won the Loeb Award for the Best Business Book of the Year!

If you know anything about the Loeb you'll know why I'm thrilled:

"Celebrating their 50th anniversary, the Loeb Awards are the highest honors in business journalism. They were established in 1957 by Gerald Loeb, a founding partner of E.F. Hutton, to encourage quality reporting in the areas of business, finance and the economy in order to inform and protect private investors and the general public. Presented by UCLA Anderson School of Management since 1973, the awards recognize writers, editors and producers of both print and broadcast media for the significant contributions they make in this field."

Bear Stearns Takes a Second Look at the Long Tail

Bear Stearns has released an interesting new report on the Long Tail effect in media, focusing especially on television. It's a follow-up on an earlier report on Long Tail effects, and as such it's an effort to address "pushback" and other objections from media industry heavyweights who make up a big part of Bear Stearn's client base.  

The main objections were these three:

1. How much demand is there for user-generated content (UGC)?
2. Can UGC actually be monetized?
3. Won’t “content always remain king”?

To the first question, Bear Stearns brings some real data to bear and finds that UGC is indeed a big deal:

Some investors remain skeptical that UGC is more than a passing fad. If we define UGC as page views only from sites such as Myspace.com, Facebook.com, Youtube.com, Wikipedia.org, Blogger.com, and Digg.com (which is quite conservative), we estimate that UGC now accounts for 13% of total U.S. Internet traffic, up from 0%-1% in 2004. Based on these statistics, we submit that UGC is here to stay.

On the second, the investment bank's answer, like everyone else's, is a fuzzy "probably", predicting that we'll get video advertising right someday. Of course text advertising on UGC content is already there.

Finally, on the third question, Bear Stearns believes (as do I; indeed a third of my book is focused on this) that in a world of infinite choice, content is only as valuable as your ability to find it. They call that "context and aggregation", and it's what both Google and your favorite blogger do when the filter the web according to a narrow lens, be it your expressed search term or their own sensibility.

And then, in one of the best parts of the report, they illustrate the problem with the notion that "content is king". In practice, only good content is king, and good content is impossible to consistently create.

One of the counterarguments to our “context and aggregation are king” thesis is that,
for as long as most can recall, the entertainment industry has lived by the axiom
“content is king.” However, no one company has proven consistently capable of
producing “great content,” as evidenced by volatility in TV ratings and box office per
film for movie studios, given the inherent fickleness of consumer demand for
entertainment goods.

This chaotic mess of a chart below says it all. Like the wise man said, in Hollywood, despite the bluster about track records and  taste, "nobody knows anything." It's all a crapshoot. Better to play the big-n statistical game of UGC, as YouTube has, than place big bets on a few horses like network TV.

How big would the movie industry be if there were more screens?

One of the examples I use in the book to illustrate the distorting effect of limited "shelf space" in traditional markets is Hollywood box office revenues. The American megaplex theater network has only has enough screens to show about 120 films per year. Meanwhile, there are about 13,000 films shown in film festivals each year. So only a tiny fraction of the movies made get enough theatrical distribution to register any sort of significant box office revenues.

How much bigger would the movie industry be if it didn't have this distribution bottleneck suppressing measured demand for niche film? I get that question all the time, for various markets, and usually I can only guess at the answer. But now Kalevi Kilkki, Principal Scientist at Nokia Siemens Networks, has actually done the math. Building on the work in his earlier paper on this subject, he finds that for movies, the "latent demand" for films that don't get adequate distribution is 60%-70% as big as the existing industry.

In other words, if we had a more efficient delivery system for theatrical films so the carrying capacity of the theatrical network were increased many-fold (using something like digital distribution and projection, which takes most of the cost out of distribution and adds ultimate flexibility over where the films run and how long), box office revenues could theoretically be 60-70% larger than they are today.

In practice, there are other economic barriers to giving all 13,000 films their day on the screen, such as the cost of marketing, the theaters and the screens themselves. But we do have another distribution system that does ahve unlimited capacity and the ability to play on an infinite number of screens. That's Netflix, which currently offers 80,000 DVD titles, and as I've shown before its rentals do follow the powerlaw model very neatly.

Here's how Kilkki comes up with his estimate: If you rank films by their box office revenues and plot the results on a log-log scale and you get the following. The dotted blue line is the real data from 2006; the solid blue line is what the powerlaw/long tail model would predict, and the red line represents curve fit of the real-world data.

Once you have equations to describe both the predicted demand and the measured demand, you can figure out how much one differs from the other. The answer varies from country to country, but the US gap is the largest. If you're into powerlaw math, you can read more in his research note. It's a clever model he's come up with and you'll find that it can be applied to markets of all sorts. This one's for you, math geeks!

Ad Update 2 (last one, I hope)

Our investigations into why wired.com was shut down for eight hours by a misbehaving ad continue. We've now pulled the server records and they show that what I was told earlier this week is incorrect. The agency did not switch the creative without telling us; instead, it appears that our own internal quality assurance process somehow didn't spot that the ad didn't have a close box (our policies don't allow those kind of ads) when they'd approved it weeks before. That's why the ad didn't have a timeout, and why it wouldn't go away.

As previously noted, we have taken all interstitial ads off the site until we can come up with a better process that will guarantee this never happens again. The silver lining may be we create a better process that improves standards across the industry.

And on that note, I hope this is the last you hear of this, and I can now happily return to Long Tail/FREE blogging, of which more later today.

Bad ad update

Several of you asked for an update when we figured out why the Wired.com site got blocked by a popover ad for about five hours on Monday night and Tues morning. I wanted to share with you the result of our investigations so far.

The way ads work on our site, like those of many others, is that we enter into a contractual relationship with an advertiser or their agency to run ads at some rate. In many cases the ads are actually hosted by a third party--in this case the agency that created the ads. Our site automatically pulls the "creative"--the ad itself--from the third party site and then runs it on our own site according to the terms of the contract.

In this case, an ad had been approved by our own quality assurance people, and then, around 10:00pm PST on Monday, the agency had switched the creative to one that had not been approved (possibly accidentally; I don't know yet). Our site started automatically displaying this new ad, and that was the one that took over our site with no close button and no way to make it go away.

Needless to say, it's a bit shocking to me that we let third parties run code (which is what that popover ad is) directly on our site. There's supposed to be an approval process in the case of creative changes, but that's a matter of policy (which was violated this time) rather than technical enforcement. We basically trust the agencies to do the right thing, which is obviously a policy hole that you can drive a bus through. That bus is what blocked our site for thousands of readers yesterday. 

We're revising our policies to prevent this in the future, and for now we've stopped accepting such invasive "interstitial" ads. This problem is not unique to us, so perhaps the silver lining in all of this is that our eventual solution may be adopted by others and will help prevent this sort of fiasco--which is as bad for the advertiser as it is the content site and the reader--elsewhere in the future.  

An Apology

I don't normally blog about my day job here, but I'll make an exception this time. If you're a regular at Wired.com, you may have seen the following monstrosity covering virtually every page of our site from about 10:00PM PST last night till sometime early this morning:

This ad, which doesn't have a close button, also didn't go away. It made the site unreadable, which is even worse than the whole site going down altogether, because it looked like we did it on purpose.

Suffice to say, we didn't. It was bug, combined with a quality assurance breakdown, combined with alert system breakdown, combined with a time zone issue (the site is administered from the East Coast). But the bottom line is that it was a major screw-up, and we're getting to the bottom of how it happened.

We've received hundreds of emails and comments complaining about this, and with good reason: it's about the worst thing you can do to a website, and we're just sick about it.

Anyway, please know that we're working hard on finding out what happened and why and ensuring that it never happens again. Many apologies.

The Long Tail of Google Maps

Everyone's very excited about the new Street View feature in Google Maps and the cool 3D view in Microsoft maps. And they both are indeed great...if you live in NYC or San Francisco or some other major metropolitan area where the Google van or the Microsoft aerial photography planes have cruised past. But the sad reality is that most of us aren't so lucky (okay, as it happens I am--that's my SF office building at right--but you get my point).

What about everyone else? I was reminded about this in reading the BoingBoing thread on this, where reader Andrew Iverson says:

I guess i'm in a small group, i wish they would cover more than just those cities, but i live in a very small town (River Falls, WI) that on google maps is in very bad resolution.

Good point. I suggest taking matters into your own hands.

Rather than wait for the Microsoft plane to darken your rooftop or the Google van to drive down your block, just build your own UAV and get high resolution aerial imagery of your own neighborhood (or anybody else's) as often as you want. It's the Long Tail of mapping!

And you thought my crazy summer project had nothing to do with the subject of this blog....

BTW, if you want to see the UAVs up close and working (well, at least on the ground), I'll be presenting them this evening at IDEO's Quickie fest in San Francisco.

Three things about "FREE"

Two weeks of constant travel are nearly at an end (I'm in Madison, WI, after having been to Singapore, Seoul, Hong Kong, New York, Denver and one day in San Francisco in between). That means that it's almost time to start writing the next book!

I'm going to be blogging a lot more about FREE this summer, as the research phase hits its stride. But in the meantime, here are three canapés to whet your appetite.

1)  My BookExpo speech on FREE got covered by the New York Times today. In an article titled Waxing Philosophical, Booksellers Face the Digital (hey, don't blame me if that's not grammatical), Motoko Rich wrote:

Chris Anderson, the editor of Wired magazine who made his own splash last year with his book “The Long Tail: Why the Future of Business is Selling Less of More,” returned to the convention to talk about the possibility of giving away online his next book — which he fittingly intends to title “Free” — to readers who were willing to read it with advertisements interspersed throughout its pages. (He still intends to sell the book traditionally to readers who’d rather get their text without the ads.)

Although that's correct, as far as it goes, I actually discussed a lot of more interesting possibilities in applying the free business model to books. (Important note: aside from a free downloadable audiobook, we haven't decided which if any we'll use on FREE. These were just examples of potential paths.)

2) AppScout had a more comprehensive report on the session, which is worth reading for the big picture. The description of the audiobook strategy is spot on:

Anderson also announced that he is going to offer the audio version of his next book for free to customers who buy the print copy. "The audio book as an MP3 costs me nothing to manufacture, copy, and distribute. In every physical book there's going to be a code, which you can then go to the Web site, type in the code and download the audio book for free. Why should you have to buy a book twice? Why punish your best customers?"

3) Speaking of free, I'm delighted to report that my UK publisher, Random House, has put huge chunks of The Long Tail online for free. Every microchunk extract is "standards-compliant and fully optimized with metadata to ensure that we are super-Google-and-other-search-engine-friendly," they tell me. Whatever that means, I'm sure it's good. Check it out and many thanks, Random House!