The Long Tail

The effect of P2P file-sharing depends on popularity

A fascinating paper from David Blackburn, a Harvard PhD student, on the economics of P2P file-sharing concludes that it does indeed depress music sales overall. But the effect is not felt evenly. The hits at the top of the charts lose sales, but the niche artists further down the popularity curve actually benefit from file-trading.

Blackburn does a little mathematical magic to simulate what would happen if file-trading were reduced by 30%. [As you read, visualize the powerlaw curve that I use as my blog's logo: "leftward" means towards the hits and greater sales.].

[From page 32] In this counterfactual world with 30% less file sharing, the lower 75% of the distribution of sales is shifted further to the left, while the top of the distribution increases its sales. This is what should be expected given the estimates from above. Artists who are unknown, and thus most helped by file sharing, are those artists who sell relatively few albums, whereas artists who are harmed by file sharing and thus gain from its removal, the popular ones, are the artists whose sales are relatively high.

This conclusion leads to further questions regarding the impacts that file sharing has had and will have on the recorded music industry. In particular, if file sharing essentially shifts sales away from established acts toward unknown acts, this has potentially very important implications for how talent is developed and distributed in the industry. As with the simple short-run effects of file sharing on sales, the direction of the impact is not clear. While one might guess that increasing the sales of new acts would lead to more investment in developing new talent, it is also possible that the investment in new acts is done as a fishing expedition to find artists who will sell millions of records. File sharing is reducing the probability that any act is able to sell millions of records, and if the success of the mega-star artists is what drives the investment in new acts, it might reduce the incentive to invest in new talent. This is, at its heart, an empirical question which is left to future work.

The data follows, but first a little explanation. The 1% line refers to the sales of the least popular albums; the 99% line is the sales of the most popular. The third column is the calculated sales if file-sharing were reduced by 30%. As you can see from the below, that would help the sales in the top 25% of popularity, and hurt those below. Which is another way of saying that file-trading seems to help those in the bottom three-quarters of popularity, probably for the reasons stated above.

Percentile | Actual Sales | Sales with 30% less file-sharing 
1%                73             70
5%                170            166
10%               281            277
25%               757            745
50%               2,852          2,851
75%               10,110         9,831
90%               26,531         26,934
95%               45,255         47,357
99%               133,983        165,054

The Long Tail implications of this are pretty clear. For the majority of artists further down the tail, free distribution is good marketing, with a net positive effect on sales. Which is yet another reminder that the rules are all too often made to protect the minority of artists at the top of the curve, not most artists overall.

Thanks to Miscellaneous Factz for the pointer.

Could the labels actually be right?

Although it's tempting to assume that the evil record labels are once again trying to gouge us, there's some sense in their latest efforts to get Apple to abandon it's one-size-fits-all pricing model. A New York Times article over the weekend reported on the ongoing struggle between the labels and Apple over its fixed $0.99 price point. The labels would like to sell most new music for more--$1.49/track?-- while older or more obscure tracks could go for less.

There's plenty to like about variable pricing. For starters, it's almost always the most efficient way to maximize markets of disparate goods and customers. As Barry Ritholtz puts it:

It's a basic rule of economics: goods that have elastic demand  (i..e, non essential) are highly price sensitive. Further, any item easily available for free (albeit illegally) will have an even bigger response to price increases.

Apple has argued that single-price simplicity was necessary in the early days of the service, when people were just getting used to paying to download music. But now, after 500m tracks have been sold, we're clearly past the early adopter phase. So what's the right pricing model going forward?

Most accounts of the dispute between Apple and the labels have focused on the industry's efforts to raise prices, which are undeniably a big part of their plan. No surprise there. The research we've been doing for the book shows that within the bulk of the online music business--the top 100,000 downloads--only 3.5 tracks on the average CD sell. So the record labels are getting less than $3 in revenue (wholesale) from albums when the music is sold by the track. That's less than half the wholesale price of a CD (although with none of the physical costs of making and distributing a CD). The shift from an album model to a track model is indeed an alarming thing for the labels, and it's easy to see why they'd want to raise retail prices online as a result.

But there's more to the story that that. The labels may be evil, but they're not (all) stupid. They--to say nothing of many of their artists--also see the virtues of dropping the price for lots of their music, too. For decades they've been playing with CD pricing models that range from cut-price classics to top-dollar boxed sets, and when freed of the overheads of traditional retail, they're likely to experiment more, not less. Although some of the more vocal commentators have encouraged Apple to hold the line at $0.99, there's a strong argument that introducing variable pricing might ultimately lead to a more consumer-friendly outcome.

The reason is simple Long Tail math: there's a lot more music in the Tail than there is in the Head, and labels are generally more willing to experiment with discount pricing outside of the top 1,000 than they are with their hits. Those niches represents most of the music available today, measured by number of titles, and because they're only modest sellers individually they're less likely to create channel conflict with CD retailers, who tend to only stock the hits.

Imagine, for starters, that Apple introduces a three-tiered band of pricing: $1.49, $.99 and $.79 (that would no doubt soon expand to include $.49, but below that the transaction costs of credit card processing and the like start to loom large). Tiered pricing--gold, silver, bronze--is still pretty simple for consumers to understand, yet it introduces a valuable new dimension of demand creation.

Rhapsody, for instance, saw demand triple last year when it cut prices in half, to $0.49. And the average usage per customer in the all-you-can-eat subscription services is typically 5-10 times that of the pay-per-download music stores, such as iTunes. Add to that the potential of free music, which Apple has already paved the way for with its podcast service and could easily be extended to music licensed under Creative Commons, and you could grow grow the user base of the commercial music services manyfold.

Once iTunes allowed variable pricing, I imagine labels would do pretty much what they already do with CDs: set a range of wholesale prices depending on age, popularity and target market. Some labels will unwisely try to jack up all their prices and will eventually suffer lower overall revenues as a result. But smarter labels will experiment with all sorts of pricing variations, from lower prices for bands they're trying to break to volume discounts for packages of older music, and even higher prices for special reissues and repackaging for hard-core fans. That's already what they do in the CD world, where even standard retail prices range from around $9 to $18.

Apple may think it's protecting us from record label avarice with $0.99, which may be true in the short term. But long term, one-size-fits-all pricing is just constraining the economics of the industry and holding back the market. If Apple introduced variable pricing, it's not hard to see how the average price might actually fall in a year or two, thanks to the number of titles in the discounted niche/backcatalog categories vastly outnumbering the more expensive hits.

As long as prices can go down as well as up, I'm confident that market forces will eventually reveal the right set of models. And I'm even more sure they will confirm that no one model is right for everyone and every song.

"Just enough piracy"

It's not news that the main reason the movie and television industries are wary of BitTorrent is that they're freaked out by the music industry's experience with piracy. Although they see the economic advantages of P2P distribution, they're concerned that once they put their stuff out there, even wrapped in triple layers of kryptonite DRM, it might be cracked and then circulate in unprotected form. For movies, that's lost revenues. For TV shows, that means ads could be stripped out, expiration routines could be removed and  (gasp!) content could be modified or remixed.

All that counts as Very Scary Stuff to industry executives, and as a result they're looking for "strong" DRM before they consider letting their premier content circulate online. This is a mistake, for two reasons:

The first is about the user experience: Any protection technology that is really difficult to crack is probably too cumbersome to be accepted by consumers.

We've seen all sorts of failures of this sort before, from dongles to laborious and confusing registration schemes. Each seems better at annoying consumers than at building markets. The lesson from these examples is that zero-percent piracy is not only unattainable, it's economically suboptimal. If your content is uncrackable, it means you've probably locked the market down so tight that even honest consumers are being inconvenienced.

Instead, efficient software and entertainment markets should exhibit just enough piracy to suggest that the industry has got the balance of control about right: not too loose and not too tight. That number is not zero percent (which requires protection methods so invasive they kill demand), and it's not 100% (which kills the business). It's somewhere in-between.

The second reason the quest for zero-piracy is a mistake is an economic one: piracy can actually let you raise your prices.

I'll give you a surprising example. I was chatting with a former Microsoft manager the other day and he revealed that after much analysis Microsoft had realized that some piracy is not only inevitable, but could actually be economically optimal. The reason is counterintuitive, but intriguing.

The usual price-setting method is to look at the entire potential market, from the many at the economic lower end to the few at the top, and set a price somewhere in between the top and bottom that will maximize total revenues. But if you cede the bottom to piracy, you can set a price between the top and the middle. The result: higher revenues per copy, and potentially higher revenues overall.

(This is, by the way, the opposite of the conventional economic approach to developing-world piracy, which is to lower the cost of a product closer to the pirate version, closing the pricing gap to try to win customers over to the official version. In practice, however, the pirate price is so low that it's rarely possible to close that gap enough to make much of a difference.)

Add to this the familiar (if controversial) argument that piracy helps seed technology markets, and can be a net benefit. Especially in fast-developing countries such as China and India, the ubiquity of pirated Windows and Office have made them de-facto national standards. Few users could have paid for the retail versions at the start, but now that the spread of cheap technology, including free software, has led to an economic boom, Microsoft is finding a nice market for commercial software at the very top, in big companies and government offices.

When all these effects are considered, it appears that there actually is an optimal level of piracy. That right level would vary from industry to industry. Today the estimated piracy rates are 33% for CDs and 15% for DVDs. The industries say that's too high, but most anti-copying technologies they've brought in to lower that figure have proven unpopular. Would even tighter lock-downs help? Probably not. Maybe 15%-30% is simply the market saying that this is the optimal rate of piracy for those industries, and any effort to lower that significantly would either choke demand or push even more people to the dark side.

So the moral for video content holders and others considering DRM: be careful what you ask for, because you just might get it. "Uncrackable" DRM could make the P2P problem worse, by driving more users underground and depressing prices. Don't imagine that if you release content in a relatively weak DRM wrapper (like today's DVDs) and copies get out that the whole market will collapse. Instead, you may find that piracy stays constant at relatively low levels, leaving the rest of the market happier and more profitable.

The lesson is to find a good-enough approach to content protection that is easy, convenient and non-annoying to most people, and then accept that there will be some leakage. Most consumers see the value in paying for something of guaranteed quality and legality, as long as you don't treat them like potential criminals. And the minority of others, who are willing to take the risks and go to the trouble of finding the pirated versions? Well, they probably weren't your best market anyway.

Are niches more profitable than hits?

After I posted my new 80/20 graphic last week (thumbnailed at right; click to expand), a number of readers queried the bar on the bottom right, which shows that in Long Tail markets profits can follow revenues equally into the niches, something that isn't true for traditional retail and is thus a big deal. But readers think I've actually understated the effect and the advantages are even more dramatic than I showed. And it looks like they're right.

I've argued that profit parity is a key advantage of the Long Tail. In bricks-and-mortar stores, economics favor the hits because they use shelf space so efficiently, briskly whizzing off the rack nearly as soon as they're placed there. But in Long Tail markets, where the costs of shelf space are very low, the niches have the same costs as the hits, and potentially the same profit margins. This explains that last profit bar in this graphic, which is the same as the revenue bar that comes before it.

But I underestimated the effect of lower niche content acquisition costs. The readers were right. The economics of niches turn out to be even better than the hits. Way better, as it happens.

To see why, let's look at the DVD retail market. Here's a rough sense of the financial picture (the estimates are based on input from a number of retail experts, including William Fisher of DVDStation):

What you see here is that the economics of new releases these days are simply awful. The studios charge $17-$19 for the DVDs and the "big box" retailers (Wal-mart, Best Buy) sell them for $15-$17 for the first week or two, for an average loss of $2 per DVD (this is before overheads; the actual loss is larger).

After the first month or so, the wholesale price of the DVDs goes down faster than the retail price, and they gradually move into profitability. Yet 70% of DVD sales are of titles within their first two months of release, before they're profitable. Why do stores sell new releases so cheaply? Because for the big-box retailers, at least, they're a loss leader, designed to draw people to other titles in DVD section and elsewhere in the store, where the margins are better.

DVD distributors encourage this by allowing unsold new releases to be returned, lowering the risk for retailers (but increasing it for the studios, as Dreamworks and Pixar have just learned to their cost).

The problem is that while this makes sense of the big-box retailers who have other things to sell, it has the effect of setting the price for everyone else, including the specialty DVD retailers like Blockbuster. The big-box retailers have thus driven down the margins for new releases across the industry, making the economics of the Head even tougher. No wonder Blockbuster's stock is down 50% this year.

But if you could shift demand further into the Tail, creating a market that wasn't so dependent on new releases, you could improve the profit picture immensely. People move in herds, so this doesn't happen overnight, but it's not impossible. This is why recommendations and other filters are so important to Long Tail markets. By encouraging people to venture from the hits world (high acquisition costs) to the niche world (low acquisition costs), smart retailers have the potential to improve the economics of retail dramatically.

(This is, by the way, exactly what Netflix does: It underbuys new releases, despite the fact that such unavailability and delay annoys some customers and increases churn, because it allows Netflix to maintain its margins.)

Note that while I've given the case of DVDs, the exact same is true for music and books, and probably a lot of other things. The bulk of the revenues may still be in the hits, but increasingly the profits are in the niches.

So, chastened by this lesson, I've redrawn that graphic, reflecting the improved profit picture in the Tail (the numbers are for illustration purposes only; they would vary in diferent markets).