The Long Tail

To Anne

This book has
benefited from the help and collaboration of literally thousands of people, thanks to the relatively open process of having it start as a widely read article and continue in public as a blog of work in progress. The result is that there are many people to thank, both here and in the chapter notes at the end of the book.

First, the person other than me who worked the hardest, my wife, Anne. No project like this could be done without a strong partner. Anne was all that and more. Her constant support and understanding made this possible, and the price was significant, from all the Sundays taking care of the kids while I worked at Starbucks to the lost evenings, absent vacations, nights out not taken, and other costs of an all-consuming project. But more than that, she was my sounding board, my first reader, my counsel, confidante, and an endless source of encouragement and advice. (Our young children—Daniel, Erin, Toby, and Isabel—also spent a year without seeing much of their father, and I thank them for rising to the occasion with sterling behavior and, one hopes, no permanent scars.)

In the research and outlining phase of the book, I was fortunate to have had the use of perhaps two of the best working and thinking
spaces on the planet. Louis Rossetto and Jane Metcalfe, our friends, neighbors, and the founders of
Wired
, graciously lent me their beautiful Berkeley offices for several months in the summer of 2005. (I was a “scholar in residence,” a title that made me feel smarter all by itself.) And another dear friend, Peter Schwartz, gave me space in the equally beautiful Emeryville offices of his Global Business Network, where I did much of my later-stage brainstorming and whiteboarding with my crack writing assistant, Steven Leckart.

My other invaluable partners were my team at
Wired
, particularly Bob Cohn and Thomas Goetz, the executive and deputy editors, respectively. They rose to the occasion brilliantly, managing to both encourage me and cover for me as the book consumed more and more of my time. Bob also edited the original Long Tail article, helping me refine the arguments and phrasing, a contribution that continues to pay dividends. Melanie Cornwell’s comments on the manuscript caught many pop culture errors and otherwise made it smarter. Also thanks to Blaise Zerega, who as managing editor kept the wheels from falling off while I was only semi-present, and Joanna Pearlstein, our research director, who helped with many of the early infographics. And special thanks to Si Newhouse for entrusting me with his remarkable platform for ideas in the first place and generously allowing me to take the time off to expand one into a book.

Many academics contributed in important ways to quantifying Long Tail effects and exploring their implications. Erik Brynjolfsson from MIT’s Sloan School of Management and Jeffrey Hu from Purdue’s Krannert School of Management did some of the early work on estimating Amazon’s Long Tail, which gave me both an analytical framework on which to build the theory and the confidence to know that it could be done. Their continuing research in this area is fascinating, and their support for my work is greatly appreciated. At Harvard Business School, Anita Elberse’s work on the Long Tail of Netflix and DVDs has been very helpful, and I look forward to both its publication and future work with her.

At Stanford Business School, Professor Haim Mendelson allowed me to present to one of his classes and make the Long Tail a research
subject. As a result, I was fortunate to work with his students Angie Shelton, Natalie Kim, Saloni Saraiya, and Bethany Poole, who wrote case studies of Yahoo! Music and eBay. On the eBay research we also had the help of Terapeak, which provided invaluable data on the Long Tail of buyers and sellers on that marketplace. And at the University of California, Berkeley, economist Hal Varian has been a font of ideas and advice, along with inspiring me to consider new angles and aspire to greater rigor.

One of the early sources of data and a continuing best-practice Long Tail example was RealNetworks’ Rhapsody. Rob Glaser and Matt Graves there were a source of constant help and encouragement, for which I am eternally grateful. Reed Hastings, the CEO of Netflix, was not only an early supporter and data provider, but also the one who advised me that my “Long Tail” phrase might have legs, which turned out to be wise indeed. Dave Goldberg at Yahoo! helped with insight into the music industry, and Bill Fisher of DVDStation provided both data and wisdom into the changing DVD economy. And Robbie Vann-Adibé, formerly of Ecast, deserves a special thanks for getting me started on this.

Thinkers and writers who contributed to this book in both words and ideas include Umair Haque, who helped tremendously with the House Music section; Glenn Fleishman, who contributed hugely to the Amazon sections; Andrew Blau of GBN, who helped me think through the Long Tail from an incentives perspective; Rob Reid, whose long and brilliant emails on the changing entertainment economy I’ve quoted at length; and Kevin Laws, whose early insights into the power of a mass of niches influenced the original article.

My agent, John Brockman, not only was a great sounding board and advisor, but also invited me into his extraordinary world of thinkers and scientists, and I count the many dinners and meetings I’ve had at his invitation as among some of the most interesting of my life. My editor at Hyperion, Will Schwalbe, helped me tremendously in focusing the book; its current structure is largely due to his wise advice and its completion is due to his constant enthusiasm and gentle guidance.

My parents deserve special thanks. To my father, Jim Anderson, for showing me the importance of a global view and intellectual honesty. And to my mother, Carlotta Anderson, for inspiring me with rhetorical rigor and boundless curiosity.

The research on the book industry was among the most difficult, since the ideal source data (Amazon’s sales records) was unavailable and we were forced to reverse-engineer much of it from third-party data. For that, I owe special thanks to Morris Rosenthal and Tim O’Reilly. Finally, thanks to John Battelle, the author of
The Search
, whose example of blogging a book in progress inspired me to start thelongtail.com, which has been the source of incalculable other good ideas, advice, data, and wisdom from my thousands of smart readers, who deserve the last, and heartfelt, thanks.

The tracking of
top-seller lists is a national obsession. Our culture is a massive popularity contest. We are consumed by hits—making them, choosing them, talking about them, and following their rise and fall. Every weekend is a box-office horse race, and every Thursday night is a Darwinian struggle to find the fittest TV show and let it live to see another week. A few hit songs play in heavy rotation on the radio dials, while entertainment executives in all these industries sweat as they search for the next big thing.

This is the world the blockbuster built. The massive media and entertainment industries grew up over the past half century on the back of box-office rockets, gold records, and double-digit TV ratings. No surprise that hits have become the lens through which we observe our own culture. We define our age by our celebrities and mass-market products—they are the connective tissue of our common experience. The star-making system that Hollywood began eight decades ago has now spun out into every corner of commerce, from shoes to chefs. Our media is obsessed with what’s hot and what’s not. Hits, in short,
rule
.

Yet look a little closer and you’ll see that this picture, which first emerged with the postwar broadcast era of radio and television, is now
starting to tatter at the edges. Hits are starting to, gasp,
rule less
. Number one is still number one, but the sales that go with that are not what they once were.

Most of the top fifty best-selling albums of all time were recorded in the seventies and eighties (the Eagles, Michael Jackson), and none were made after 2000. Hollywood box-office revenue was down by more than 6 percent in 2005, reflecting the reality that the theatergoing audience is falling even as the population grows.

Every year network TV loses more of its audience to hundreds of niche cable channels. Males age eighteen to thirty-four, the most desirable audience for advertisers, are starting to turn off the TV altogether, shifting more and more of their screen time to the Internet and video games. The ratings of top TV shows have been falling for decades, and the number one show today wouldn’t have made the top ten in 1970.

In short, although we still obsess over hits, they are not quite the economic force they once were. Where are those fickle consumers going instead? No single place. They are scattered to the winds as markets fragment into countless niches. The one big growth area is the Web, but it is an uncategorizable sea of a million destinations, each defying in its own way the conventional logic of media and marketing.

ITUNES KILLED THE RADIO STAR

I came of age in the peak of the mass-culture era—the seventies and eighties. The average teenager then had access to a half dozen TV channels, and virtually everyone watched a few or more of the same handful of TV shows. There were three or four rock radio stations in any town that largely dictated what music people listened to; only a few lucky kids with money built record collections that ventured farther afield.

We all saw the same summer blockbusters in the theater and got our news from the same papers and broadcasts. About the only places you could explore outside the mainstream were the library and the comic book shop. As best I can recall, the only culture I was exposed
to other than mass culture was books and whatever my friends and I made up, and that traveled no farther than our own backyards.

Contrast my adolescence with that of Ben, a sixteen-year-old who grew up with the Internet. He’s the single child of affluent parents in the tony North Berkeley Hills, so he’s got a Mac in his bedroom, a fully stocked iPod (and a weekly iTunes allowance), and a posse of friends with the same. Like the rest of his teenage friends, Ben has never known a world without broadband, cell phones, MP3s, TiVo, and online shopping.

The main effect of all this connectivity is unlimited and unfiltered access to culture and content of all sorts, from the mainstream to the farthest fringe of the underground. Ben is growing up in a different world from the one I grew up in, a world far less dominated by any of the traditional media and entertainment industries. If you don’t recognize yourself in the pages to come in this book, imagine Ben instead. His reality is the leading edge of all of our futures.

From Ben’s perspective, the cultural landscape is a seamless continuum from high to low, with commercial and amateur content competing equally for his attention. He simply doesn’t distinguish between mainstream hits and underground niches—he picks what he likes from an infinite menu where Hollywood movies and player-created video-game stunt videos are listed side by side.

Ben watches just two hours or so a week of regular TV, mostly
West Wing
(time shifted, of course) and
Firefly,
a canceled space serial he has stored on his TiVo. He also counts as TV the anime he downloads with BitTorrent, a peer-to-peer file-sharing technology, because it was originally broadcast on Japanese television (the English subtitles are often edited in by fans).

When it comes to movies, he’s a sci-fi fan, so he’s pretty mainstream.
Star Wars
is a passion, as was the
Matrix
series. But he also watches movies he downloads, such as amateur machinima (movies made by controlling characters in video games) and independent productions such as
Star Wars Revelations,
a fan-created tribute film with special effects that rival the Lucas originals.

Some of the music on his iPod is downloaded from iTunes, but most comes from his friends. When one of the group buys a CD, he or
she typically makes copies for everyone else. Ben’s taste is mostly classic rock—Led Zeppelin and Pink Floyd—with a smattering of video-game soundtracks. The only radio he listens to is when his parents turn on NPR in the car.

Ben’s reading ranges from
Star Wars
novels to Japanese manga, with a large helping of Web comics. He, like a few of his friends, is so into Japanese subculture that he’s studying Japanese in school. When I was in school, kids studied Japanese because Japan was a dominant economic power and language skills were thought to open up career opportunities. But now kids study Japanese so they can create their own anime subtitles and dig deeper into manga than the relatively mainstream translated stuff.

Most of Ben’s free time is spent online, both randomly surfing and participating in user forums such as Halo and
Star Wars
discussion sites. He’s not interested in news—he reads no newspapers and watches no TV news—but follows the latest tech and subculture chatter on sites such as Slashdot (geek news) and Fark (weird news). He instant messages constantly all day with his ten closest friends. He doesn’t text much on his cell phone, but he has friends that do. (Texting is preferred by those who are out and about a lot; IM is the chat channel of choice for those who tend to spend more time in their own rooms.) He plays video games with friends, mostly online. He thinks Halo 2 rocks, especially the user-modified levels.

I suspect that had I been born twenty-five years later, my teenage years would have been quite similar. The main difference between Ben’s adolescence and my own is simply choice. I was limited to what was broadcast over the airwaves. He’s got the Internet. I didn’t have TiVo (or even cable); he has all that and BitTorrent, too. I had no idea there was even such a thing as manga, much less how to get it. Ben has access to it all. Would I have watched
Gilligan’s Island
reruns if I’d been able to build a clan with friends in World of Warcraft online instead? I doubt it.

TV shows were more popular in the seventies than they are now not because they were better, but because we had fewer alternatives to compete for our screen attention. What we thought was the rising tide of common culture actually turned out to be less about the triumph of
Hollywood talent and more to do with the sheepherding effect of broadcast distribution.

The great thing about broadcast is that it can bring one show to millions of people with unmatchable efficiency. But it can’t do the opposite—bring a million shows to one person each. Yet that is exactly what the Internet does so well. The economics of the broadcast era required hit shows—big buckets—to catch huge audiences. The economics of the broadband era are reversed. Serving the same stream to millions of people at the same time is hugely expensive and wasteful for a distribution network optimized for point-to-point communications.

There’s still demand for big cultural buckets, but they’re no longer the only market. The hits now compete with an infinite number of niche markets, of any size. And consumers are increasingly favoring the one with the most choice. The era of one-size-fits-all is ending, and in its place is something new, a market of multitudes.

This book is about that market.

This shattering of the mainstream into a zillion different cultural shards is something that upsets traditional media and entertainment no end. After decades of executives refining their skill in creating, picking, and promoting hits, those hits are suddenly not enough. The audience is shifting to something else, a muddy and indistinct proliferation of…Well, we don’t have a good term for such non-hits. They’re certainly not “misses,” because most weren’t aimed at world domination in the first place. They’re “everything else.”

It’s odd that this should be an overlooked category. We are, after all, talking about the vast majority of everything. Most movies aren’t hits, most music recordings don’t make the top 100, most books aren’t best-sellers, and most video programs don’t even get measured by Nielsen, much less clean up in prime time. Many of them nevertheless record audiences in the millions worldwide. They just don’t count as hits, and are therefore not counted.

But they’re where the formerly compliant mass market is scattering to. The simple picture of the few hits that mattered and the everything else that didn’t is now becoming a confusing mosaic of a million mini-markets and micro-stars. Increasingly, the mass market is turning into a mass of niches.

That mass of niches has always existed, but as the cost of reaching it falls—consumers finding niche products, and niche products finding consumers—it’s suddenly becoming a cultural and economic force to be reckoned with.

The new niche market is not replacing the traditional market of hits, just sharing the stage with it for the first time. For a century we have winnowed out all but the best-sellers to make the most efficient use of costly shelf space, screens, channels, and attention. Now, in a new era of networked consumers and digital everything, the economics of such distribution are changing radically as the Internet absorbs each industry it touches, becoming store, theater, and broadcaster at a fraction of the traditional cost.

Think of these falling distribution costs as a dropping waterline or a receding tide. As they fall, they reveal a new land that has been there all along, just underwater. These niches are a great uncharted expanse of products that were previously uneconomic to offer. Many of these kinds of products have always been there, just not visible or easy to find. They are the movies that didn’t make it to your local theater, the music not played on the local rock radio station, the sports equipment not sold at Wal-Mart. Now they’re available, via Netflix, iTunes, Amazon, or just some random place Google turned up. The invisible market has turned visible.

Other niche products are new, created by an emerging industry at the intersection between the commercial and noncommercial worlds, where it’s hard to tell when the professionals leave off and the amateurs take over. This is the world of bloggers, video-makers, and garage bands, all suddenly able to find an audience thanks to those same enviable economics of digital distribution.

THE 98 PERCENT RULE

This book began with a quiz I got wrong. One of the things I do as the editor of
Wired
is give speeches about technology trends. Because I started my career in the science world and then learned economics at
The Economist,
I look for those trends first in hard data. And, fortu
nately enough, there has never been more data available. The secrets of twenty-first-century economics lie in the servers of the companies that are all around us, from eBay to Wal-Mart. Although it’s not always easy to get the raw numbers, the executives at those companies swim in that data every day and have a great intuitive feel for what’s meaningful and what isn’t. So the trick to trend-spotting is to ask them.

Which is what I was doing in January 2004, in the offices of Robbie Vann-Adibé, the CEO of Ecast, a “digital jukebox” company. Digital jukeboxes are just like regular jukeboxes—a big enclosure with speakers and blinking lights, often found in bars—with the difference that rather than a hundred CDs, they have a broadband connection to the Internet and patrons can choose from thousands of tracks that are downloaded and stored on a local hard drive.

During the course of our conversation, Vann-Adibé asked me to guess what percentage of the 10,000 albums available on the jukeboxes sold at least one track per quarter.

I knew, of course, that Vann-Adibé was asking me a trick question. The normal answer would be 20 percent because of the 80/20 Rule, which experience tells us applies practically everywhere. That is: 20 percent of products account for 80 percent of sales (and usually 100 percent of the profits).

But Vann-Adibé was in the digital content business, which is different. So I thought I’d go way out on a limb and venture that a whopping 50 percent of those 10,000 albums sold at least one track a quarter.

Now, on the face of it, that’s absurdly high. Half of the top 10,000 books in a typical book superstore don’t sell once a quarter. Half of the top 10,000 CDs at Wal-Mart don’t sell once a quarter; indeed, Wal-Mart doesn’t even carry half that many CDs. It’s hard to think of any market where such a high fraction of such a large inventory sells. But my sense was that digital was different, so I took a chance on a big number.