The Long Tail

Google Apps and the power of embedded functionality

There been a lot of talk today about Google releasing a suite of hosted applications, starting with basic communications (email, talk) running on your own domain and then more Office-like apps such as word processing and spreadsheets. Most of the discussion has been in casting this as a competitor to Microsoft's Office. But I think there's something entirely different (and a lot more interesting) going on.

As Anil Dash discusses here, these web-based apps are not meant to replace Office but to complement it by doing things online that desktop software just can't do well. What might those things be? I think we have a hint in the spread of embedded video, courtesy of YouTube. The ability to easily embed into any blog page a full-featured videoplayer dedicated to a single video is a large part of YouTube's success. It doesn't require you to go elsewhere or download anything--it just works.

Now imagine the same model working for data. Rather than me posting static jpeg charts and links to Excel spreadsheet files, what if I could post data the way I post videos: as an embedded mini-app that simply displays the data in a useful way, allowing readers to manipulate or copy it at will? This would be a little like what Ray Ozzie (Microsoft's Gates V2.0) calls "Live Clipboard", which is a proposed way to copy and paste code, structured data and even functionality from website to website, just as we currently do with plain text.

That's what I want. Not an online spreadsheet that simply replicates what Excel already does perfectly well on my laptop, but small spreadsheet elements that I can paste into a blog post in the form of a specific data set or graph. The fact that they're hosted elsewhere is what would make them simple enough to use, just as embedding YouTube video is so head-slapping easy today. That's not yet the case for the woefully under-featured Google spreadsheet (there's no graphing and you can't make it open to all), but don't be surprised if Microsoft under Ozzie does better with Office Live. The embedded functionality era has just begun. YouTube is just the start of something much bigger. 

Five things about me that may or may not be relevant

As I do more and more press interviews for the book, I'm starting to get requests for a more personal history. My potted professional background is here, but it's pretty dry. My family history, fortunately, is not. Here are five things about my background that may or may not have made me what I am today.

1. My great-grandfather, Jo Labadie, helped found the American anarchist movment in the late 1800s (that's him at right). His story is told in All-American Anarchist, a biography by his granddaughter, my mother. His writings and library became the foundation of the Labadie Collection at the University of Michigan, which is now a major collection of radical literature, from civil liberties to sexual freedom. It's also the repository for letters to and from Ted Kaczynski, the Unabomber, while he is in prison.
2. My grandfather, Fred Hauser, invented the automatic sprinkler system, the source of a very modest family fortune (here's a later patent; the online patent database only starts in 1976). I spent some blissful summers with him in his workshop in Los Angeles, where we would make a two-stroke engine out of solid blocks of steel and crude castings. Metal lathes are amazing, and the crunchy feel of steel curlicues and shavings underfoot at the end of a good day of machining is something I'll never forget.
3. My father, Jim Anderson, was a Wisconsin telegraph operator's son who just happened to luck out in the draft and get an assignment to work on the Army newspaper in La Rochelle, France during the Korean War (I know, it's unbelievable). He stayed in Europe after the war, ending up working for a newspaper in Berlin at the height of the Cold War and, very John le Carré-like, hanging out with Eastern Bloc "diplomats". In 1960, when CIA U2 pilot Gary Powers was shot down over the Soviet Union, those friendships paid off when he was one of a small handful of western reporters allowed in to cover the trial. It put him in the international spotlight, and led to a distinguished career in journalism that still continues today, albeit from semi-retirement.
4. I was born in London in 1961 and had a cute British accent until we moved to the US when I was five. I've still got dual citizenship, which came in handy when I lived in Hong Kong in the 1990s and needed to enter mainland China regularly without using my US passport, which had a dreaded "journalist" stamp in it. (People may tell you that ethnic minorities and Falun Gong practitioners are the most persecuted people in China, but journalists are certainly up there).
5. Finally, there's that whole punk/new wave band thing. Yes, I really did fail out of college, work as a messenger and otherwise spend a good chunk of my twenties working hard at being a slacker. But I eventually turned the corner, went back to university and did the proper hardcore physics thing, which is where my professional bio above begins.

How shooting digitally changes acting

On Saturday I was in LA speaking at the Directors Guild of America, opening their annual "Digital Day" with a talk about the declining share of blockbusters in much of media and entertainment (Hollywood being a glaring, but perhaps not lasting, exception). Aside from showing a bunch of low-res web video to make the amazing 4k and 3D video that came later in the day look all the better, my main function was to scare the crap out of them with YouTube statistics. 100 million streams a day for flash animations, Apple ad spoofs and other three minute fare from an army of amateurs...and probably not a DGA member among them.

But the really cool thing was the panel discussion that followed me. Tony Bill (Flyboys, Untamed Heart), Rob Cohen (The Fast and the Furious), David Fincher (Fight Club), Rebecca Miller (The Ballad of Jack and Rose) and Brad Silberling (Lemony Snicket) spoke about "How choosing to work in digital affects the directorial process".

The answer: a lot. "I think shooting in digital changes acting as much as film changed stage acting, or as sound changed film," said Bill.

Why? Because film costs a lot and must be used sparingly, while digital tape is practically free. The difference between the scarcity economics of film and the abundance economics of digital is, as Bill put it, "the difference between pointing a loaded gun at someone and a toy gun. You point a loaded gun at them and they're going to act different. A film camera is a loaded gun. Digital is not."

He explained further what he learned shooting Flyboys with the Panavision Genesis. "The old model of acting is that the rehearsal is great and then things change when you say "rolling"--usually for the worse. Now there's no film in the camera. You can shoot everything. So there's no rehearsal. Or perhaps it's all rehearsal. Either way, it's far more natural."

Actors freeze up when they know that there's a cost to failure--a thousand-foot magazine of film costs $1,200 between film and developing. Said Bill: "That slight whirring noise of film running through the camera is the sound of money. And it gets in the way of being real."

I've had to unlearn saying "action" and "cut". I think shooting in digital makes every actor better. You're always in rehearsal and never in performance. There's no "start". It allows for serendipity. Rather than reach an emotional moment and then having to recreate it later with the film running, you capture everything.

Even better, the director doesn't have to make all the important decisions up front. They can apply "makeup" in post production. Rather than build the perfect period-piece set, they can shoot in the real world and age it afterward. For one movie set in the 1970s, one of the panelists said, "We shot on real streets and then spent a half-million dollars erasing Starbucks from every fucking shot."

But there are some downsides to digital, too. For one thing, because it costs far less to do additional takes, directors are prone to making their actors do it again and again--asking for 40 takes rather than 20. Bill described Robert Downey Jr. leaving little jars of urine around his chair because the director wasn't letting him go to the bathroom. "I'm exhausted," Downey complained. Who can blame him?

Another downside: At super high resolution, "you can now see how bad extras are," noted Fincher. Reviewing some recent rushes, he noticed that two extras, crisp and clear in the background, "are supposed to be talking to each other, but nobody's listening. They're both talking at the same time!"

Finally, a note about the photo at the top of this post. It's a shot I took of Doug Trumbull, the legendary visual effects pioneer who did 2001: A Space Odyssey, Blade Runner and Close Encounters of the Third Kind. He's narrating a slide show of shots from the 2001 set, but he isn't actually at the podium. Instead, he appeared via hologram (he was actually at Siggraph in Boston). The picture doesn't do it justice, but up close it was amazing.

My New Wave Hair: An Apology (and a story)

Valleywag found a copy of my long-forgotten band's EP, complete with a picture of me with New Wave Hair (as shocking for the bad taste as it is for the reality that I once had hair--although it wasn't really that color).  But what it hasn't found is the great story of the band's name. So here it is:

It's 1982, I've failed out of college and am working in Washington DC as a messenger and playing in bands at night. This is the heyday of the DC punk scene, which I have just graduated from largely because I couldn't play "Stepping Stone" fast enough (you had to be there). I do, however, have a bass and the aforementioned haircut. So I was a good candidate for an up-and-coming Gang of Four-inspired band called....wait for it.....REM.

We were actually pretty good, entirely thanks to the other band members, and the time came to cut and release an EP. As we were recording the record, the producer mentioned that he'd just heard that there was another band called REM, from some place in the sticks called Athens, Georgia. They, too, had a single coming out. No worries, we thought--we had the name first (I'm not actually sure that was true) and how good could a band from Georgia be anyway?

But the owner of the 9:30 Club, a legendary local new music venue, saw an opportunity to have some fun. Why not have a battle of the REMs? So she invited them up and they accepted. We, meanwhile, thought it would be a great way to build some buzz for our forthcoming record.

There were two rules: A coin toss determines who opens for whom. The winner gets to rename the loser.

The night came, and we lost the coin toss. No worries--we went on and played a killer set, got a lot of applause, then retired to the bar to enjoy our victory.

Then the other REM came on. I'm a little fuzzy about the progression, but I think the first song they played was Radio Free Europe. The crowd went silent, mouths hung agape, and when the last chord was struck, the room exploded. Crap.

After that REM finished their incredible set, most of the band went off to get drunk. Mike Mills, the bass player, was kind enough to stick around long enough to rename us (although he'd had a few, too). Having spent some time with us backstage before the show, he had just the name: Egoslavia.

And that's how this band got its stripes. Too bad we never issued passports!

Wikipedia gets RSS feeds

I'm a huge fan of RSS feeds, in part because they're perfect for niche media. You don't have to publish every day to earn your way to my reading habits. Just make sure you've got a feed and if your focus intersects with my interests, I'll subscribe. When you publish I'll get it--no need for me to bookmark it and make a note to check it regularily. (I've written more about this here).

At this point virtually everything I read is via a feed, with one glaring exception--my Wikipedia watchlist. It's the only bookmark I still use. So I was delighted to discover this morning that finally Wikipedia has feeds! Thanks to Steve Rubel for the heads-up:

Wikipedia has added RSS feeds to the 1.25 million entries in the encyclopedia. This means you can now more easily track the revision history for important articles, such as those about people, brands or corporations. Simply click on the history link at the top of any entry page and you will see the RSS link on the left hand side

And to cap off a fantastic day, we buy Wired News

As if today wasn't big enough, I am absolutely delighted to announce that a long-held dream of ours has been realized: we finally have our web site back. Conde Nast today announced the acquisition of Wired News.

You may be confused by this: Wired.com isn't the website of Wired Magazine? Well, for the last eight years it hasn't been. The magazine and website were separated in a complicated deal in 1998, long before my time (see this Wikipedia entry for more Wired history). The result was an agreement between the two, by which Wired News (wired.com) would host our content on their site (under wired.com/wired) next to their own content, but we, the magazine, were prohibited from doing anything in the digital realm. Aside from being somewhat ironic that Wired Magazine wasn't really wired, it was frustrating for us to be unable to walk the talk, since we didn't control the site.

Now we do. Today we have the opportunity to make Wired (the combined brands) a leader in online media innovation, exactly as our readers expect. We're bursting with ideas and can't wait to get started brainstorming with our cousins once-removed across the hall from us in San Francisco, who have done fantastic work with limited resources for all these years, sharing our name but not our keycards. Welcome to Conde Nast, Wired News! We've been waiting for this day for a long time.

Open Peer Review

The scientific journal Nature is conducting a fascinating experiment in "open peer review", which it describes this way:

In Nature's peer review trial, lasting for three months, authors can choose to have their submissions posted on a preprint server for open comments, in parallel with the conventional peer review process. Anyone in the field may then post comments, provided they are prepared to identify themselves. Once the usual confidential peer review process is complete, the public 'open peer review' process will be closed. Nature will report on the results after the trial period is over.

In parallel, it's conducting a online debate about the future of scientific peer review in the web age, and has asked various scientists and writers to contribute their thoughts. Mine was on how the online definition of "peer" (basically anyone, as in "peer-production" and "peer-to-peer") can help us rethink the academic definition of "peer" (usually a tenured PhD professor) for the purpose of identifying the most important scientific work. Here's an excerpt:

The free online Public Library of Science (PLoS) journals are planning to extend the open peer review model by adopting conventions from the blogosphere: an open comment area for each paper, 'trackbacks' that show which sites are linking to it, and perhaps a reader ratings scheme. Michael Eisen, a genomics researcher at the Lawrence Berkeley National Laboratory and one of PloS's founders, says the hope is to capture some of the post-publication wisdom already found in academia, but rarely accessible to others.

PLoS still uses expert researchers to review papers before publication, but the editors realize that these scientists often have little time to really dig into a paper. By contrast, readers of a paper after publication may also have an opinion, and many (especially graduate students) have the time to evaluate the paper in depth. The online environment means there’s no reason not to record it.

Such a record would have the effect not only of continuing peer review after publication, but also of making it easier to find important work in a blizzard of papers – they’re the ones that are being buzzed about. It is also easier to ignore poor work that slipped through peer review – these are the papers with the withering comments and poor ratings.

Best of all, such an open peer-review process taps into something that already exists: journal clubs. Every day, thousands of researchers and students are discussing the latest papers, but their insights and opinions are not recorded and shared widely. This information needs only to be collected, organized and distributed to become far more useful. It's now possible to tap such collective intelligence online by doing to scientific publishing what the web has already done to mainstream media: democratizing it.

You can read the whole thing here and comment on it here.

What's in Chris' gadget bag?

If anyone's curious, here's what's in my gadget bag: the technologies I own and use every day. These aren't necessarily endorsements, but it may be interesting for people to see what the editor of Wired, who is exposed to an awful lot of gadgetry, actually chose to buy.

Core tech:

• Computer: Lenovo ThinkPad T60 (Core Duo, EvDO)
• Second computer (travel and bedside): Lenovo X41 Tablet PC
• Handheld gameplayer: Nintendo DS (mostly used by the kids, aside from a little Brain Age for me)
• Phone: Treo 700p
• MP3 player: Creative Zen Micro 6GB
• Digital voice recorder (for interviews): Olympus DM-10

Home entertainment tech:

• DVR/media server: Dell Media Center PC
• Videogame console/media extender: Xbox 360s (two)
• Video display: Panasonic AE900 projector
• Main surround sound: Onkyo HT-S777
• Remote speakers: Polk Audio Monitor 40s
• Music streamer for rest of house: Roku SoundBridge M1000
• Universal remotes: Harmony 680s

Photo:

• SLR camera: Canon Digital Rebel XT
• Pocket camera: Canon Digital Elph SD630
• Camcorder: JVC DVM-90 (not recommended; I'll go tapeless next time)

Networking:

• Linksys routers, cable modem, wifi access points and hubs (a mix of gigabit and 100baseT)
• Network backup storage: Buffalo Gigabit Linkstation (400GB)
• Network printer: HP psc 2200 multifunction printer (no longer available)

Services:   

• Music: Rhapsody
• Phone: Sprint
• ISP: Comcast broadband, Earthlink email server
• Blog: Typepad
• Feed reader: Bloglines
• Web hosting: Hostway
• DVD: Netflix

Core application software (aside from standard Office Pro)

• Browser: Firefox and Firefox 2 beta (Bon Echo)
• Charts: Adobe Illustrator CS
• Other graphics: Corel Draw Essentials
• Photo touchup: Photoshop Elements 4.0
• Photo library: Picassa
• Outliner/notes: ActionOutline

As an aside, most of the data analysis for the Long Tail was done with database queries in Access, outputting to Excel.  The charts in the book were done in Illustrator.

Al Gore, rockstar (live!)

As you might have guessed from Wired's cover story last month, Al Gore is one of my heroes. Back in my science days (early 90s) I was lucky enough to see him up close as the chairman of the Senate Science and Technology subcommittee. For all the jokes, his work in promoting what was then called the National Information Infrastructure really was instrumental to the development of the Internet.

I particularly remember several hearings he held with scientists and scientific visualization experts from the National Center for Supercomputing Applications (which would later become the birthplace of the Mosaic web browser, which in turn led to Netscape). None of the other committee members bothered to show up, so he came down off the dais and sat at the witness table with the visiting researchers to geek out over the 3D simulations and advanced (for the time) SGI workstations. This was a Gore we never saw in the 2000 Presidential Campaign: passionate, intensely curious, scientifically sophisticated and, yes, human--even funny.

What's notable about that scene around the witness table is that it combines the three main theme in the life of Al Gore: politics, technology and the environment. The simulations he was being shown at these hearings were climate models, some of which showed early work in projecting the effects of global warming. When he was introducing bills to build out the Internet, one of his main objectives was to make it easier for scientists to collaborate remotely on climate research, to try to understand the effect of human-generated carbon emissions on global temperatures.

Now Gore is back, and his passions have converged in a campaign--movie, book, speaking tour and web effort--to focus the world's attention on global climate change. Gore's a new kind of environmentalist--what we call a "Neo Green". Rather than calling for us to dial back our lifestyle, slow development and return to the land, he looks for technological solutions (both in energy generation and conservation) that allow continued economic growth with less environmental cost. No surprise that he sees technology as a solution rather than the problem: he's on the board of Apple and is a special advisor to Google.

If you happen to be in New York City on Thursday you can see him in action yourself. Wired is holding a town hall discussion at Town Hall with Gore, NASA's James Hansen, Laurie David, and Lawrence Bender, moderated by Wired Contributing Editor John Hockenberry, Thursday, May 25, from 8-10pm. I'll be doing the introductions.

You can get tickets here.

Lego: How they do it

In my post last week on the new Lego Factory, which now only ships you the pieces you need rather than expensive bags of (too many) assorted parts, I promised I'd find out how the did it. Yesterday I spoke to Michael McNally, Lego's Brand Manager, who explained how they cracked the tricky picking and packing problem of total mass customization.

The answer: they pack the kits by hand, piece by piece. In one of Lego's Denmark factories, one packing station is now dedicated to Factory. The 520 pieces available in Factory are a number large enough to be interesting but small enough to be stored in bins that are no more than a step or two away for the packer. (The previous model used prepacked bags from the Creator series, which it could ship from a US warehouse). As it happened, Lego had recently automated some other parts of its Denmark operation so it had a few extra workers, who it was able to reassign to this job.

It's not a big job so far--only about 3,000 kits have been sold from the 75,000 uploaded--but that relatively small number and high upload-to-purchase ratio may be in part due to the high kit prices imposed by the previous inefficient parts strategy, which could easily double to price of a model. Now that Lego has been able to drop the average model price by 60% with this only-what-you-need packing process, I suspect more people will be tempted to buy what they design.

Next steps, McNally says: more pieces (including more minifigs) and making the software a bit easier to use.