The Long Tail

What is the Long Tail?

I've been struggling to come up with a pithy definition of the Long Tail, something longer than a bumper sticker and shorter than an elevator pitch. So I'm going to give it a few goes here and perhaps you, the wise participants in this conversation, can tell me which resonates most (note: some of these are my own, others are stolen from other people's attempt to define it). Pithy construction and TypePad layout flexibility allowing, I may offer the winner a permanent place at the top of this blog.

A) The Long Tail is the infinite shelf-space effect--the new mass market of niches that rises when the existing bottlenecks in distribution that favor hits are removed.

B) The Long Tail is the myriad of niche products whose collective market share can rival the blockbusters.

C) New efficiencies in distribution, manufacturing and marketing are resetting the definition of what’s commercially viable across the board, turning sub-economic customers, products and markets into economic ones and creating a Long Tail of demand.

D) The Long Tail is about the economics of abundance—what happens when the bottlenecks that stand between supply and demand in our culture start to disappear and everything becomes available to everyone.

E) The Long Tail is the story of how formerly sub-economic products and customers are suddenly becoming the biggest market of all.

F) None of the above. Please try harder.

Variety is not enough

When you think about it, today's vibrant Long Tail actually arose in two steps. The first was the abundance boom of the 1990s that arrived with an explosion of variety on supermarket shelves (thanks to revolutions in supply chain efficiency and globalization) and the miraculous appearance of the wide-open Internet. Both were impressive in their own way, but neither were enough. In fact, many people found them oppressive. 

In the physical world, the experience of trying to make a wise purchase among 150 kinds of toothpaste or three dozen kinds of iced tea introduced the concept of the "tyranny of choice." Studies showed that people actually felt worse about their purchases when they had more to choose from (did I make the right decision?), and often even bought less because of it.

Online, in those pre-Google days, the flat hierarchy of hypertext and the trueism that 90% of everything is crap led to much hand-wringing over "information overload" and learned distinctions between mere information and true understanding. After wandering lost for days  in the unfamiliar wild of  sophisticated amateurs and clueless companies, the early online explorers could only agree that there sure was a lot there. Whether it was good remained to be seen.

What was missing in both these cases was the necessary second step: not just the presence of lots of things and information, but the arrival of  information about them--"data" about products, and "metadata" about information itself. 

For the physical goods, this can take the form of something as simple as Amazon's "rank by bestselling" lists to more complex background information such as reviews, price comparisons, version histories and manufacturing details.  Even a little such information can make a huge difference in helping a consumer make a decision.

As Amazon's Jeff Bezos explains it, for a product that a potential purchaser has a great deal of interest in, no amount of information is too much: from reader and trade  reviews to service records, the more they can learn about a product the more comfortable they are buying it. But for products that they just don't care much about, even something as simple as knowing what most other people bought can make the difference between being frozen by overwhelming choice and purchasing with confidence. 

On the supermarket shelf today the only information you have about all those toothpastes is whichever brands have been lodged in your brain, price, the promotional text on the boxes, and whatever display placement the manufacturer has paid for. None of that is very helpful, and it certainly isn't independent. But imagine if you could add just one simple bit of information: what sold most in the last hour? Suddenly you are able to tap the wisdom of the crowd.  This is what information about products allows, and why the online retailers, which can offer that most easily, have been more successful in enticing consumers to venture down the tail towards less familiar products.

By the same token, Google offers information about information, namely how many other people thought a given page was important enough to link to it. As a result, its ranked search results are like the Amazon category bestseller lists: when in doubt, start at the top. Collaborative filtering does the same for music and film, drawing connections between the uneasily categorized and revealing popularity. And several companies are working on ways to use cameraphones as barcode readers, bridging the divide between the bare goods in the physical world and the rich information online. 

In all these instances, data and metadata can structure a flat hierarchy and bring order out of chaos.  The variety boom may have created the Long Tail, but it takes information about that variety to entice people down it.

The "Pro-Am" Revolution

One of the big difference between the head and the tail of producers is that the further you are in the tail the more likely you are to have to keep your day job. And that's okay. The distinction between "professional" producers and "amateurs" is blurring and may in fact be ultimately irrelevant.  We make not just what we're paid to make, but also what we want to make. Both can have value. Once, with high barriers to marketplace entry, only the professional work found an audience. But now those barriers are dropping.

Bloggers are perhaps the most obvious example; is anyone really paying the rent with Adsense alone? And yet 8 million Americans have created their own blog, according to a new Pew Internet survey. There are loads of other examples, from open source software contributors to homebrew remix artists.  Demos, a UK think tank, has written a great report on the "Pro-Am" phenomenon, which they summarize as follows (if you have trouble loading the pdf from your browser, as I did, save it to your desktop and doubleclick that):

From astronomy to activism, from surfing to saving lives, Pro-Ams - people pursuing amateur activities to professional standards - are an increasingly important part of our society and economy. For Pro-Ams, leisure is not passive consumerism but active and participatory, it involves the deployment of publicly accredited knowledge and skills, often built up over a long career, which has involved sacrifices and frustrations. The 20th century witnessed the rise of professionals in medicine, science, education, and politics. In one field after another, amateurs and their ramshackle organisations were driven out by people who knew what they were doing and had certificates to prove it. The Pro-Am Revolution argues this historic shift is reversing. We're witnessing the flowering of Pro-Am, bottom-up self-organisation and the crude, all or nothing, categories of professional or amateur will need to be rethought. Based on in-depth interviews with a diverse range of Pro-Ams and containing new data about the extent of Pro-Am activity in the UK, this report proposes new policies to support and encourage valuable Pro-Am activity.

Nanoeconomics

I've been asked to do a Long Tail "high-order bit" presentation (a concept first seen at the Web 2.0 conference) at O'Reilly's Emerging Technology conference in San Diego in March. Chatting with one of the organizers today, I was given the choice to go deep or go broad. Rashly, I chose deep, offering this:

The Economics of the Long Tail: Forget macroeconomics. While you're at it, forget microeconomics, too. The really interesting new work is in nanoeconomics, or economic behavior in markets of a million niches far down the long tail of demand. From Pareto distributions to the rise of secondary markets, the ability to find big businesses in small sums is casting a new light on the dismal science.

Now a confession.  At the time I thought nanoeconomics was my own neologism (much like "The Long Tail" itself), which I would have a few months to develop before presenting. That shouldn't have been a problem. There are, as it happens, a lot of really interesting, even surprising, economic effects that crop up in markets with a large number of low-volume transactions. And the economics of abundance are always fascinating. I won't say that visions of Oslo were dancing in my head, but it is a rich seam.

But I was naive. Of course someone had coined the term before me. Indeed, the problem is that there are at least four radically different definitions for nanoeconomics. There is, naturally enough, the economics of nanotechnology. Also, the economics of auctions and other micro-markets where the rules are important to determining the outcome. The economics of massively multiplayer games and virtual worlds. And, quite randomly, the microeconomics of children.

Still, Google only shows 223 hits for the word, so I figure there's still hope for me to reclaim it.  I'll start with these topics:

--The economics of hits vs. niches.

--The economics of abundance.

--Zipf vs. Pareto vs. Powerlaw: why shape matters.

--The importance of transaction costs (Coase redux).

--The connection between long tails and secondary markets, from used goods to overstock items.

Proper economists (or even improper economists) are invited to offer thoughts on any and all of the above while I reteach myself statistics. As if.

The biggest market is in the smallest sales

Daniel Gross, writing in the New York Times today with a very Long Tail-ish celebration (reg req'd) of big markets in small sums, mentions a book I've just started reading. Like the Long Tail, The Fortune at the Bottom of The Pyramid, by CK Prahalad, is about finding ways to effeciently address sub-economic markets. In this case, he's talking about how to sell goods and services to the world's 4 billion poor, for mutual benefit.

Don't be put off by the do-gooder tone and dry textbook style; this an important guide to finding ways to offer products at lower overhead and in lower-cost form to profitably reach markets that previously looked hopeless. For instance, take this table, which when you substitute "niche consumers" for "the poor" and "hits" for "products sold in developed countries", could apply to Short Head thinkers everywhere:

("BOP" is bottom of the pyramid; "MNC" is multinational corporation)

You can get the thrust of the argument in his 2002 article (free reg req'd) by the same name in Strategy+Business. Prahalad is setting up a San Diego-based consultancy to help companies implement his advice, and recently hired my friend and former Economist colleague Azeem Azhar to help him. Welcome to California, Azeem!

More as I get further into the book.

Department of double-entendres

Although I'm delighted to have thelongtail.com as my domain, I wish I had longtail.com, too. I almost did, but it appears that Live365 (an Internet radio network) gazumped me in the auction with the Russian who held the domain at the time of my article (rather randomly so, since I'm not aware that the phrase "long tail" was in any significant use before my piece).

Recounting this story at a dinner with some high-spirited entrepreneur and advertising folks, it became clear by the subsequent half-hour of increasingly indelicate one-upsmanship that I should have fought harder for that domain: it's sure to be a sloganeer's dream, loaded as it is with cheeky bumper-snicker goodness. Indeed, I wouldn't be surprised if that's why the Russian had it in the first place. 

If a smart marketer can't do something subliminal (or not) with "head", "tail", and "long", they should consider another profession. I, however, will try to refrain.