Does the Long Tail create bigger hits or smaller ones?

Over the past few weeks there has been a flurry of reappraisals of the Long Tail, most of which center around the question of whether it creates bigger blockbusters or smaller ones (more concentrated markets or less concentrated ones). 

My predictions have always been that massive increase in variety plus massive improvements in "filters" (tools to make it easier to find new stuff that's right for you) would tend to reduce the blockbuster effect and redistribute attention over a wider range. And, indeed, that's what the data I cited in my book showed, where online markets of books, DVDs and music saw between 20% and 40% of the demand shift to products not available in traditional bricks and mortar stores. 

But there were clearly exceptions to this. One of the main ones was the irony that there was a very short Head of Long Tail aggregators: Amazon, iTunes, Google and their kin dominate their markets to a blockbuster-like degree. 

I blamed this on a still-young market and assumed that even aggregators would fall victim to the flight from one-size-fits-all someday. But new research from McKinsey (free registration req'd) suggests that this sort of radical inequality is increasingly the norm as markets get more networked. 

The past decade has seen the rise of many “mega-institutions”—companies of unprecedented scale and scope—that have steadily pulled away from their smaller competitors. What has received less attention is the striking degree of inequality in the size and performance of even the mega-institutions themselves. Plotting the distribution of net income among the global top 150 corporations in 2005, for example, doesn’t yield a common bell curve, which would imply a relatively even spread of values around a mean. The result instead is a “power curve,” which, unlike normal distributions, implies that most companies are below average.

Here's an example: the increasing gap between the top two or three financial services companies and everyone else since 1994 (the recent collapse of many financial firms and the roll-ups that have led to three "megabanks" will only accentuate this):

 

 

 

And it's not just companies. The Long Tail--the powerlaw created by network effects--may be creating super-celebrity, too. Here's what Google CEO Eric Schmidt (pictured above) says about the Long Tail and blockbusters (I've transcribed his interview with McKinsey):

I would like to tell you that the Internet has made such a level playing field that the Long Tail is absolutely the place to be, that there’s so much differentiation, so much diversity, so many new voices. I’d love to tell you that that’s in fact how it really works. Unfortunately, it’s not. What really happens is that we follow what’s called a powerlaw. A powerlaw has the property that a small number of things are very highly concentrated and most things have relatively little volume. And virtually all of these new network markets follow what’s called Zipfs’s law, or a powerlaw. 

It means unfortunately for our political point of view, although the tail is very interesting and we enable it, the vast majority of the revenue remains in the head. And this a lesson that businesses have to learn. While you can have a long tail strategy, you better have a head, because that’s where all the revenue is. 

And in fact it’s probable that the Internet will lead to larger blockbusters, and more more concentration of brands, which doesn’t make sense to most people, because it’s a larger distribution medium and when you get everybody together they still want to have one superstar. It’s a bigger superstar. It’s no longer a US superstar, it’s a global superstar. Global celebrities, global scandals, global politicians. And just to be clear, it’s a 90/10 model. We love the long tail, but we make most of our money in the head, just because of the math of the powerlaw. But you need both. You need the head and the tail to make the model work.

Since Google is the canonical Long Tail company, I should probably have a good response to this. But the truth is that he's no doubt right. Powerlaws do imply wildly unequal distributions of money, power, celebrity and everything else. But this is nothing new. The rich get richer, fame snowballs, and so on. Vilfredo Pareto spotted this in 1906 in Italian wealth distribution, which is why a powerlaw is more commonly known as a "Pareto distribution."

So how to square this with my own prediction of more widely distributed markets? Three ways:

1. There's a big difference between a highly concentrated market with a small number of players and one with a huge number of players. If there are ten stores in the world, having one of them with 70% market share smacks of a monopoly. If there are ten million stores, it doesn't. Amazon may be by far the largest e-tailer, but there are hundreds of thousand of other, smaller specialists. Google may be the largest online advertising company, but there are again many thousands in its wake. The Tail is Long. As Schmidt notes, these countless specialists are essential to a well-functioning market, but it's true that few of them are getting rich at it.
2. The consequence of the above is that it's very hard to measure the entire market. Schmidt rightly notes that network effects on a global scale mean that Paris Hilton is more famous today, in terms of the number of people who have heard of her, than she would have been a half-century ago. But at the same time, there are millions of microcelebrities (of which I am one, I guess) who are also more more famous than they would have been a half-century ago, for the same reasons. What is the total market of celebrity, and what is Paris Hilton's share? We have no idea--it's practically impossible to measure, to say nothing of how it has changed over the years. But I would bet that the aggregate market for microcelebrity (think: Facebook) is gaining share on traditional celebrity.
3. Finally, it's not all about money. As I've said many times, both in the book and elsewhere, most of the rewards in the Tail are non-monetary: a larger audience for producers, and more choice for consumers. Sometimes those can lead to economic benefits, but often they can't. Today, a decade into the arrival of unlimited variety online, the Long Tail is still more of a cultural force than an economic one.

I'll end by conceding a point: It's hard to make money in the Tail. As Schmidt notes, it's also hard to make money if you don't have a Tail (to satisfy minority taste, which improves the consumer experience), but the revenues are disproportionately in the Head. Perhaps that will never change, but what will change is our definition of Head. Once that was choice counted in tens or hundreds of items. Now, especially in Google's world, it's counted in tens or hundreds of thousands.  Powerlaws may indeed create bigger fish, but the Long Tail is all about the bigger pond.

Comments

[Adam Wexler]

Chris,

As always, it is great to hear your point of view, and I believe your analysis is correct. If you can satisfy the head AND the tail, you stand a greater chance of sustainability because the consumers will remain on YOUR site to peruse. If you fail to include either one, their attention span will shift to a more niche-based site that can satisfy their needs.

Adam Wexler

[Alessandro]

Hi Chris,

a few days ago I've sent you an email about this interview, so in someway I think this post could be also your answer to the doubts I've raised you after having read it.
One of the message I got from your book (see ex. chapter "The rise and the fall of the hit" )is that Internet should change as well the "hit-driven mindset". This has significant impacts not just from an economic point of view but especially from a cultural perspective.
Now if the Internet is mainly amplifying this hit-driven paradigm, we'll have more niches, but we won't probably see the turn "(...) from a mass market back into a niche nation".

Alessandro

[ronald van den hoff]

Chris, being a serial enterpreneur from Europe, I know that "making money", weather that is serious money or not, is always difficult. Untill you make it. Than it suddenly is relatively simple. Head or tail makes no difference.
E.g. I have a meeting-conference formula that is making money in the physical world in the tail, although through the social network belonging to that formula, we have such a large "virtual" marketshare, that we are in the head....
Does it really matter or are we still trying to fit new knowledge into old paradigms?

[Tom Worth]

Great reconciliation of Long Tail and Fat Head. You can throw Netflix in there, too. These are the companies that offer all of the hits, all of the long tail, AND give up nothing in terms of price competitiveness or ease of use. Then, they pour all of their winnings back into making their offerings even better. Why would any consumer need to go anyplace else, and on what basis could any new competitor hope to break into books, music, movies, or search? The consumer would have nothing to gain by taking a risk and trying out someone new.

[Kevin Kelly]

Chris,

You say:
"the rewards in the Tail are non-monetary: a larger audience for producers, and more choice for consumers."

I think you continue to confuse who the producers are. The Long Tail primarily rewards aggregate distributors and customers. If you can bundle the long tail you have an expanding audience. But most of the producers in the long tail of production produce only few items. While they have access to a potentially larger audience, they also face actual larger competition for that audience.

My own efforts to show evidence that the average long tail producer (not aggregator) can increase their audience is only half successful. My verdict would be that the long tail rises the audience for a long tail producer, but not by very much.

On the other hand the long tail of production is good news for the "producers" at the head of production: that is the Amazons, Googles, and Netflix of the world.

I suggest this: that you have TWO long tail curves. One is consumption and one is production.

[christopher]

"Amazon may be by far the largest e-tailer, but there are hundreds of thousand of other, smaller specialists."

but doesn't amazon include thousands of small retailers using amazon as a storefront? isn't amazon itself part of the long-tail? it seems the distinction between tail and head is being blurred.

[Steve Hopkins]

I agree with Christopher above. It doesn seem that, as more and more net-organisations catch on to idea of long tails more will attempt to blur that business model into their own, previously 'head based' business.

I also think that we are begining to see real long tails being introduced in a non-economic sense with the growing niche communities being formed online (and, thus, offline.)I think this will continue, in line with the observation that people are more famous than they would have been 50 years ago. Once people know more people, around their areas of expertise, will equal more global niche communities.

[Panayotis Vryonis]

I'm tempted to do a comparison between FatHead/LongTail markets and oscillators. If "hits" are maximum vibrations, what is the resonance of each system and how do they compare?

I would compare the Fat Head to a monolithic homogeneous system and the Long Tail to a heterogeneous system with many loosely coupled parts. How do degrees of freedom compare between systems? Is it equally possible to have a big hit? A small one? How much "energy" does it take in each case? What are the limitations in each case?

I'd love to hear from someone with a more solid background in physics who could take the idea a bit further. (Or is it totally irrelevant?)

[Eben Carlson]

Finally some long tail discussion that I can agree with!

If the old economy was a "regular body" that grew a "long tail" then this is the "beautiful face".

We think of celebrity as a negative because our current economic "body" has very average faces, but a super celebrity could (and I'd argue will be) be the world's best artist, a super philanthropist, most interesting businessperson, an unbelievable singer, etc... That is, if superlative content made money.

The long tail deals with increased quantity, the beautiful face deals with increased quality! I've been writing about this for a long time at whiteg.com. The ramifications--economic, social, cultural, and political--are incredible.

The future is more of everything AND better of everything. And the money will be where it's always been--in the increases in quality, not in the proliferation of quantity. Put it this way--those interested in higher quality will be artists, entrepreneurs, designers, poets, muses, etc. Those interested in greater quantity will be spreadsheet jockeys, warehouse managers, marketing guys and shipping greats. Long tail or beautiful face--take your pick.

To the guy who asked about physics--there are many strong parallels.

Essentially, the "regular body" is Newtonian physics, slow and inefficient, the long tail is quantum physics, much faster and more efficient but not without problems. The beautiful face, then, is the new physics surrounding dark energy, which replaced quantum physics as our most complete description of reality when it was discovered in 1998. The universe is estimated by scientists to be at least 96% dark energy. (Which gives you a good indication of how profitable its economic systems will be in relation to energy, which is what our current economy is based on).

In physical terms, the "body" is our tangible body, the "tail" is our intangible brain (which makes distribution and production--quantity--easy), and the face is our untangible soul (the exploration of which makes quality easy). There are many parallels and conclusions to be gained from this--it's investigated in a much more comprehensive fashion at my web site, mentioned above.

The basics go like this: before 1900 the world, our economy and our physics, were tangible, inefficient and slow. During this time intangible reason was not valued at market rates--in fact it was often violently suppressed. About 1% of the population before 1900 was "free" and war and slavery were commonplace. The world GDP grew at less than .3% a year.

Around 1900 reason began to be valued by the market and the intangible era was born. Our economy, politics and social systems were ordered around reason and became much more efficient. Just 100 years after the introduction of these systems, half the planet is free and self-sufficient. The world GDP grows at an average rate of 3% a year (a remarkable 10 fold increase).

(It's also interesting to note that slavery, colonialism, raw sewage, rampant disease and high infant mortality rates--the most intractable problems of the pre-1900 intangible era--were only solved by the introduction of new more efficient intangible systems. These problems after festering unaltered for tens of thousands of years were obliterated in under 100--an historical blink of the eye--with the introduction of a new paradigm.)

The intangible systems we currently employ are better than tangible systems but are still inherently inefficient. A nuclear reactor is more efficient than an internal combustion engine but it produces significant amounts of dangerous waste. Intangible systems, as the name implies, are less tangible than tangible systems but are still somewhat tangible. Untangible systems, on the other hand are entirely non-tangible and thus offer the only true relief from arbitrary waste and discomfort.

The emerging knowledge based on the study of dark energy will yield untangible systems that produce no "waste" whatsoever. Fusion would be one example (I know it's not viable yet). Untangible systems are hyper efficient.

Luckily, knowing how the intangible age was born, we don't have to wait for intangible scientists to "discover" us into this new untangible age. The current market mechanism which retards the proliferation of untangibles is as clearly apparent as the social mechanisms which suppressed the growth of intangibles was 150 years ago.

Just as in 1800 the market refused to value intangible reason, the market today refuses to value untangible creativity. It prices all songs at $.99, all movies at $10, all DVDs at $24, all books at $14--regardless of quality or demand. The same is true for television programming, magazines and any place that creativity is offered in the mass market. This effectively prevents the generation of a critical mass of high quality creativity.

This flat pricing structure is eroding fast. It is facing immense pressure from al sides as it destroys valued industries and creates mediocre creative content (and runs counter to all other sectors, which are increasingly creating value with premium offerings). If cars were priced the same way as songs, we'd still be driving Pintos. There'd be no reason to invent catalytic converters, hybrids, better or sexier rims, bigger sound systems--or just about any of the style or quality we enjoy in even cheap cars today. All this quality is driven from the top down. Without a robust and profitable top--a beautiful face--quality suffers everywhere. This is what every experiment with fixed price economies has shown us.

Once this dam breaks, historical analysis suggests that we could see world GDP growth averaging close to 25% a year, the creation of an entirely new and more valuable mass culture, and the proliferation of prosperity and freedom to almost all remaining parts of the world.

This could all be done while decreasing our environmental impact, offering increased job satisfaction, and providing more relaxed and enjoyable lives to people around the world. All we need to do is switch out our current paradigm for the new model. Greater efficiencies will do the rest.

And it will do them almost overnight: if the "regular body" took at least 1000 years and the "long tail" took 100, then the "beautiful face" should only take 10.

Whether you're interested in making money, saving the world, making a better culture or improving politics--untangible systems are your best bet. They'll offer returns that make current ROIs look stingy. If the tangible age made millionaires and the intangible age made billionaires, it's easy to imagine that the emerging untangible age will make trillionaires. The economy is ultimately about meaning and the only way to get high quality meanings is to value untangibles at market rates.

The tangible age was about what's predictable, the intangible age was about what's probable, the untangible age, then is about what's possible.

More about what's possible at www.whiteg.com

[RC]

The big problem I see with your theory is that you assume that 100, 1000, or 10,000 search sites besides Google have an equal chance of being accessed/discovered. Once a dominant player or group of players emerges it naturally lays claim to a majority of the potential customers. Once this segregation has begun the influence of popularity leads to the creation of credibility. At this point the others beyond the first few leaders cease to be a factor.

I just don't see the long tail as an enduring trend. It only has a finite existence until a significant differentiator creates a group that becomes popular which leads to the creation of credibility.

[Morten Blaabjerg]

Great analysis, Chris! Have not yet followed the discussion, but wanted to rush in and say, that I think your theory is rock solid. In fact, I believe Schmidt's analysis is flawed.

Check out the "long tail" of search as recently examined : http://weblogs.hitwise.com/bill-tancer/2008/11/sizing_up_the_long_tail_of_sea.html

Now after reading that article I can't see that Schmidts claim to "earn most of the money in the head" has any credibility. Google, as one of the first truly global aggregators of information, has, if any company, earned their billions in the tail.

[guerilla ontologist]

Google wags the dog.

[Mukund]

Also, we need to keep in mind that technology plays a critical role, especially for the leaders monetizing on the head-end of the power curve: the sheer economies of scale, and the lessons learned in building the web scale technology platform (whether it is Google's search platform with their home-brew GFS, BigTable, Google Apps etc or Amazon's eCommerce infrastructure for smaller merchants, EC2/S3 service) puts them in a sweet spot: not only do they enjoy the revenue side of the power curve, they are also enabling hundreds if not thousands of developers and smaller players to ride on THEIR platform, but on the Long-tail...making money on tons of smaller players riding the long-tail.

[Arun D]

I agree with Morten 100%. I find it quite surprising that Schmidt did not at least concede how much Google make from the tail. Am I wrong to think that MOST of their money would be in the tail? If Google was to only book ads for Fortune 1000 companies, that would be a big change in the number of sponsored links I see per week!

[NVL]

This is the first analysis I've read that really explains what's been going on in Hollywood over the last decade.

In this era of unprecedented choice, Hollywood films have arguably become less diverse than at any time in history. If you look at the films that were released as recently as ten years ago, it's clear there's been a giant contraction of diversity and originality. Sequels have always been popular in Hollywood, but it's become almost impossible for a film that's not a sequel (or otherwise built on a well-known brand) to do much business. Even the rare indie hits ("Little Miss Sunshine" and "Juno") are so few and far between that the specialty film business has become all but unsustainable. Artistically ambitious films are now basically a form of corporate welfare solely justified by the chance of winning an Oscar.

Schimdt's comments offer an explanation for why this is happening. Ironically, the diversity of the internet has turned Hollywood into an increasingly efficient blockbuster factory.

It's true that it's not all about money -- but film can only be created with at least some money. Even though digital technology is making film cheaper to produce, and short online videos are increasingly popular, there's a big chasm between Hollywood blockbusters and YouTube. I fear that many of the most original and creative films of our era simply aren't being made because they fall into this increasingly vast chasm.

[Alessandro]

You shouldn't limit yourself to Hollywood.

This is the top ten of movie production for 2003 by country, from Film Digest (July 2003). I reckon Hollywood's lead has further diminshed since then, but I couldn't find the data.

1. India- 1200
2. US- 543
3. Japan- 293
4. France- 200
5. Spain- 137
6. Italy- 130
7. Germany- 116
8. China- 100
9. Philippines-97
10. Hong Kong- 92

This means Hollywood represents roughly 22% of the top ten world movies producers - and there are many more. Now, I remember from the discussion on the Canadian movie Chris cited in his book that US theaters are pretty "nationalistic" in their choices, projecting almost only Howllywood movies.

Hollywood movies seem to be, or are perceived to be (in snobby Europe at least ;)), pretty self-similar. Lots of special effects, paper thin plot, explosions, firearms, explosions, explosions.

Which is good by me, I like that kind of movie, it's very relaxing. European movie studios stay away from that niche tho, probably because Americans or so good at it, or maybe because they don't have enough money and stuff to blow up.

For example, there's no European movie I know of that was taken from a comic.

So I see much more variety in movies around me as the internet gives me even more access to international productions(even tho American action movies still top the box office).

[Greg Link]

Hi Chris, Well said. Two thoughts. 1. I think the attraction to the head or Star is a function of efficiency. In a world of increasing content overload our limited attention is grabbed by those we trust. The Wisdom of Crowds tends to cause us to trust the well known reliable sources or best selling authors for example. This is why endorsements by other best selling authors helps a book rise to the top. 2. That said the LongTail is a valuable breeding ground for soon to be stars. If not for the long tail many great authors would not survive long enough to be discovered or as Gladwell says to reach a tipping point. Many books that are ultimately in the head start out in obscurity. Before the internet long tail many great books went out of print before they gained momentum. To your second point of microcelebrities this is a valuable opportunity for the best idea to survive long enough to win. Pre internet for example it took Chairman Peter (Peter Drucker) 50 years to become a star. It took Stephen Covey 20 years pre internet. Malcolm Gladwell did it in less than 5. The long tail is a very valuable asset.

[Mark Nunney]

Already referenced here, this piece from Hitwise:

http://weblogs.hitwise.com/bill-tancer/2008/11/sizing_up_the_long_tail_of_sea.html

... gives us a small taster of how big the long tail is when defined by 'search terms' (aka keywords) on search engines. It's so big the head is irrelevant.

Looking at the head, tail and body in terms of actual searches made on the internet is a different perspective to that of looking at individual companies (or celebrities or films, as done here).

So what is Eric Schmidt on about because he didn't just make it up. Correct me but he only talks in abstract so we must guess. He refers to a 90/10 version of Pareto's 80/20. Here are two possibilities:

* 90% of Google's revenue comes from 10% of advertisers.

* 90% of Google's revenue comes from 10% of search terms.

Anyone know which he meant?