The Long Tail

Excellent HBR piece challenging the Long Tail

Anita Elberse, a Harvard Business School associate professor, has a really interesting article in the new Harvard Business Review that analyzes some Long Tail data and challenges some of the theory's predictions. Based on Rhapsody music data and DVD rental data from an Australian Netflix clone called Quickflix, she concludes that the blockbusters are not losing share to the long tail of niche products in those markets; indeed, they're gaining it. She writes:

Although no one disputes the lengthening of the tail (clearly more obscure products are being made available for purchase every day), the tail is likely to be extremely flat and populated by titles that are mostly a diversion for consumers whose appetite for true blockbusters continues to grow.

That's surprising (not least to me), and now that I've had a chance to give the paper a quick read, let me jot down some quick thoughts on why Elberse (who I collaborated with on some of my research and respect highly) would come to such different conclusions than I do.

Let me start by saying that the paper looks rock solid and I'm sure her analysis is accurate. But there is a subtle difference in the way we define the Long Tail, especially in the definitions of "head" and "tail", that leads to very different results.

The best example of this is in what she describes as a growing "concentration" of sales around a relatively small number of blockbuster titles. In the Rhapsody data, she finds, the top 10% of titles (out of more than a million in that data sample) accounted for 78% of all plays, and the top 1% account for 32% of all plays. That sounds pretty concentrated around the head, until you reflect, as she notes, that "one percent of a million is still 10,000--[...]equal to the entire music inventory of a typical Wal-Mart store."

This is a good moment to remind everyone of the normal definition of "head" and "tail" in entertainment markets such as music. "Head" is the selection available in the largest bricks-and-mortar retailer in the market (that would be Wal-Mart in this case). "Tail" is everything else, most of which is only available online, where there is unlimited shelf space.

So in the data she cites, the head of the online music market represents 32% of the all plays, and the tail represents 68%.  That's certainly no challenge to the Long Tail theory; indeed, it's even more tail-heavy than the data I cited in my book (probably because I used a more generous estimate of 50,000 tracks for Wal-Mart's inventory).

She then looks at Quickflix data. Here the top 10% of DVDs accounts for 48% of all rentals, and the top 1% accounts for 18%. "The concentration [of sales around the blockbusters] is not as strong as Rhapsody, but it's still substantial," she writes.

But here, too, the use of percentages misleads. Quickflix had 18,000 titles at the time of the research, compared to the average Blockbuster's 3,000 titles--there's only a factor of six between their inventories, as opposed to a factor of 100 in the Wal-Mart/Rhapsody comparison. If you look at her chart, you'll see that the top 3,000 titles (ie, the amount equal to Blockbuster's inventory, or the "head") accounts for 70% of rentals and the "tail" accounts for just 30%, making it more concentrated on the head than Rhapsody, not less. (BTW, I calculated almost exactly the same split for Netflix in the book.)

My point is not to suggest that Elberse is wrong and that I'm right, it's only to point out that different definitions of what the Long Tail is, from "head" to "tail", will generate wildly different results.

Anyway, it's getting late and I just wanted to highlight a few other interesting data points and conclusions from her article:

• Much of the paper is about consumer satisfaction in the head vs tail. In the Quickflix data, she says, "customers give lower ratings to obscure titles...it is a myth that obscure books, films and songs are treasured. What consumers buy in Internet channels is much the same as what they have always bought." That may be true for the specific example of the Australian DVD data, but it is not clear from the paper why she feels able to extrapolate that to all Internet commerce.
• The heaviest DVD renters were the most likely to venture into the tail; light consumers largely concentrated on the hits.
• In music, of the 2.4 million digital tracks sold in 2007 in the US (most of them through iTunes) 24% sold only one copy and 91% sold fewer than 100 copies.

And there are pages and pages of other nuggets like this. It's an excellent article, and although I don't agree with all the conclusions, I'm delighted to see research of this rigor on the topic. Recommended.

Where to run the One Machine?

In the new issue of Wired, Kevin Kelly has written another one of his patented mind grenades: the observation that the Internet has now hit approximate computational equivalence to a single human brain:

A hyperlink is much like a synapse in the brain. Both work by making associations between nodes. Each unit of thinking in the brain — an idea, for example — grows by gaining links to other thoughts. The greater the number of synapses connecting to an idea, the stronger it becomes. Similarly, the more heavily linked a Web node is, the greater its value to the Machine. Moreover, the number of hyperlinks in the World Wide Web is approaching that of synapses in the human brain. But the Machine contains a million times more transistors than you have neurons in your head. And, unlike your brain, it's growing at a rate that outpaces Moore's law. By 2040, the planetary computer will attain as much processing power as all 7 billion human brains on Earth.

The full piece is here, along with a nifty animated version of the fold-out graphic that's running in the print magazine. But it's just a hint of where Kevin's going as he contemplates the increasing ability to treat the world's computational resources as one, what's often called ubiquitous cloud computing. Among the most radical implications of this is how we consider the new landscape of globalization in an age where anything digital can theoretically be done anywhere. How do we decide where?

In a post today, he describes three strategies:

Follow the Sun:  As one time zone wakes up for another day of commerce and entertainment, the peak activities will migrate around the planet in a wave that follows the sun. While California crunches, India sleeps. And vice versa. Here the maximum computation and energy needs will be found nearest to the time zone in the sun. 

Follow the Moon:  If the costs and latencies of communication are smaller than computation, then the many huge data centers can be placed where energy costs are least. And no matter where they are, their loads will ordinarily be less at night. So India crunches to keep California awake. And vice versa. Therefore the least expensive computation will be a wave flowing around the globe at night, or following the moon. 

Follow the Law:  Perhaps neither energy nor communication costs will be the gating factor in the One Machine; rather it may be law. Differences in privacy laws, censorship, and national security fears may restrict places where data can flow freely. In that case computation will have to hopscotch around the world following the law. 

Most likely different industries adopt a different scenario. Maybe financial follows the moon, while commerce follows the sun, and entertainment follows the law.  A single computing environment (One Machine) should not suggest homogeneity. A meadow is not homogeneous, but its does act as a coherent ecological system.

To that legal point, I was recently chatting in Seattle with a guy who runs the largest collection of server farms in North America outside of Google--he actually owns many of the facilities that Amazon's EC2 service and Microsoft's cloud computing initiatives are running on. Like everyone in that business these days, he's all about finding cheaper electricity. But although his facilities are all in the Pacific Northwest, using clean and relatively cheap hydro power, he hasn't crossed the border into Canada, where the hydro power is even more plentiful.

Why not? Because of political instability. Canada's governments shift from right to left too often, he said, and the threat of regional secession was too real to risk putting multi-hundred-million-dollar data facilities there--between changes in the laws to even the slight risk of nationalization should the wrong person be elected, he thought Canada's political liabilities outweighed its energy assets. Surprised? I was. But right or wrong, that's the sort of calculus that's required in the new era of global data. Anything can be anywhere. Where do you want to go today? 

Free Dailies: The one bright light in the newspaper industry

While all is not well in the newspaper business,  the global picture is far better than the US one, according to stats released today by the World Association of Newspapers. Overall circulation is up nearly 3% globally, mostly thanks to Asia. Even in the US 62% of adults read a paper every day. Although that's declining, it's still a pretty big business. 

But the one unambiguous growth area in the industry? Free newspapers:

Free dailies are surging ahead, growing circulation 20% last year, mostly in Asia and their native Europe. Free dailies now make up 7% of worldwide print circulation (EU 23%, US 8%).

That's from a PaidContent report, ironically enough. Hey, Rafat--when are you going to change your great site's name?  ;-)

[UPDATE: Piet Bakker notes that the WAN's data collection methods appear to be pretty sloppy. Take these numbers with a grain of salt--probably directionally accurate, but no more than that.]