Hoop Dreams, Netflix and the long tail

Sports Illustrated's Seth Davis wondered how well the award-winning documentary Hoop Dreams has held up over the years, so he decided to watch and review it again (he still likes it). The film follows the lives of two young basketball players through high school and college. Both are very good players with dreams of making it in the NBA. Neither makes it.

This should not come as a great surprise. Of the many, many players who start out on high school teams, only a small portion play college ball, and of those only a small portion make it to the pros, whether the NBA, the D-league, overseas or elsewhere. Of those, few survive more than a couple of years and only a very select group make the big money that the high school players are dreaming of. That leaves a lot of very good players looking for some other way to make a living.

What we have here, then, is a long-tail distribution. As with athletes, so too for musicians, actors, entrepreneurs, would-be movie moguls, politicians or, for that matter, bloggers. In all these cases, rather than finding a way to "monetize" the long tail, the market has allocated the lion's share to the very top and much of the rest to the almost-top. Not a problem for those of us playing pick-up after work or blogging as a hobby, but a harsh reality check for a 21-year-old on draft day.

The web was supposed to change all that, at least in the media world. And actually, maybe it has to some extent. How did Davis dig up a copy of a decade-old indie documentary, one long since out of the theaters and not available at the local big-chain video store? Netflix, of course.

14:59 left on the fame clock

A friend of mine writes an occasional blog on, well, whatever pops into his mind at the moment. He's done absolutely nothing to publicize it. In fact, he prefers anonymity, which is why I haven't provided a link. Occasionally people stumble across it anyway, typically once every day or so. Given the number of potential readers out there, that's a nearly perfect record of obscurity. He's even farther out on the long tail than I am.

A couple of years back, this friend posted an album review of the latest offering from a band with a fairly large cult following. Nothing much came of it, but even so it came to account for a significant portion of the site's traffic. Then, one day not too long ago, the post seemed to catch the attention of the band's fan base. In one day, that one post got hundreds of hits. Still not much compared to a major commercial site, but well more than the entire site had garnered in the previous year.

Of the hundreds of visitors, almost all bounced right back off the site. A few followed the tag for that band. Since there was only one post with that tag, this didn't get far. Few, if any, stayed to browse the other posts. Two left comments. The first completely disagreed with the review. The second completely agreed.

And that was it. The site has since settled back into obscurity. If he'd been concerned that a horde of fans was going to overwhelm the site and flood it with comments, that concern has passed.

The lesson here, I think, is that if the long tail idea has legs (interesting image, that), and a small site can become modestly profitable if only it can find its audience, it needs to focus tightly on its particular niche. A site dedicated to care and feeding of 1948 Oldsmobiles might well find a following of rabid Olds fanciers. A blog on anything and everything, done for the author's amusement, will pass largely unnoticed. My friend assures me this suits him fine.

Red fruit in square bins

While looking for something else, I happened to encounter, or possibly re-encounter, Google Squared. Google doesn't exactly advertise it prominently. I came across it by selecting Even more ... from the More ... drop down on Google's home page, then Labs, where Squared is third among many. The lack of fanfare is probably appropriate. In its present form at least, it seems more like an interesting experiment than anything like a showcase product, thus the placement on Labs.

I started throwing some of the Baker's Dozen questions at it, but it soon became clear that it's just not aimed at answering plain English queries. However, if you give it red fruit as a query, the results are pretty nice; better, in fact, than any from the original baker's dozen.

I have to say I'm not completely sure what Squared is meant to do, beyond assembling search results into a grid based on some sort of vague and probably dynamic criterion. Still, it seems like a fairly neat hack.

My $0.00 worth

I have a confession to make. I hadn't actually read Chris Anderson's book before discussing a review of it.

I was OK with that because, judging from Gladwell's review, I've already seen the basic thesis put forth, and because I didn't see any pressing need to buy a book about how everything is going to be free.

But just in case, I just read a shorter version of the same spiel, appearing online in Wired. On the one hand, it's more tightly reasoned than the strawman version that forms in one's head in the absence of the original item (for example, Anderson has of course heard of King Gillette). On the other hand, I still side with Gladwell on this one, and it doesn't seem likely that anything in the book but not the article would change that opinion.

The main beefs I have with the argument are:

Free stuff is not a new idea. Anderson acknowledges as much. Indeed he opens with Gillette inventing the "give away the razor, sell the blades" model. But this puts the burden of proof on Anderson to show how the web is different from other historical examples.

For example, the net offers near-zero marginal cost in some contexts, but so has broadcast radio/TV. The default hypothesis is therefore that the web offers basically more of the same, albeit with localized disruptions such as occur with any technological change -- my not-so-disruptive-technology hobby horse. I simply don't see the clear-cut case that things are radically different.

There are (at least) two kinds of "free" in play here. One is "free" in the marketing sense, which really means that the cost is hidden somewhere else. Anderson gives a taxonomy of six ways this can happen, none of which seems particularly new or peculiar to the web. The other is "near-zero marginal cost," which as Gladwell points out, really means "near-zero marginal cost for part of the puzzle".

In the marketing world, there is a big psychological difference between free and cheap. In the technical world, Moore's law and its cousins provide some hope that certain costs will get exponentially close to zero, but will never actually get there. Implicit in the whole argument is the idea that near-zero costs ("free" in the second sense) enable giving things away ("free" in the first sense). But giveaway marketing doesn't depend on the inputs being free. It depends on being able to hide, defer or otherwise shift the costs. Granted, this is easier when said costs are low, but the two are definitely not the same thing.

Moore's law doesn't apply to everything. When disk space starts getting tight -- yes, it still happens -- it's traditional for development to complain to IT. "C'mon ... you can get a terabyte for a hundred bucks these days. What's the holdup?" The holdup is that the cost of the disk drive is only part of the cost of the storage. Time spent ordering and installing the drive costs something. Adjusting the various administrative parameters/scripts/whatever costs something. Ideally not much, but something.

Google and company have found ways to reduce those costs, too, making it attractive to move storage out "into the cloud". But such a move costs time and disruption. Once you're in the cloud, some of the administrative costs go away, but some don't. New costs might even arise from not being able to just go look at the hardware. Ideally not much, but something.

All the while certain costs are tending asymptotically toward zero, there's generally something in the system resistant or immune to the trend. A big something in many cases is the content -- the movie, song, text or whatever that's being conveyed at near-zero cost. Content costs to produce and the money has to come from somewhere. If it doesn't, some content just won't get produced (rest assured, dear reader, that this blog is definitely not that sort of content).

Gladwell's point about electricity too free to meter, that the important question is why this didn't and couldn't happen, is dead on.

Ah, the irony. It's easy to nitpick, but to me it seems more than curious that an author arguing that prices are going inevitably to zero should choose a good old-fashioned hardback book, and not a particularly cheap one, as a medium [hmm ... have I beaten that point into the ground yet?]. Likewise, if digital technology has driven costs of movies and such to near zero, how can one cite a $15 DVD as a loss leader?

If "[b]roadcast commercials and print display ads have given way to a blizzard of new Web-based ad formats", why am I still bombarded with broadcast commercials and print display ads, not to mention billboards, signage and, for that matter, junk mail? One could argue that "given way" is just a figure of speech here, execpt that it clearly fits an overall pattern of claims that the web has supplanted the old order. Calling out more clearly when the web has and hasn't actually supplanted older forms might have been tedious, but might also have been instructive.


In sum, while I don't dispute that hardware and bandwidth getting cheaper and that this is bound to have some economic effect, and while I agree that the web has had and will continue to have some disruptive effects, I remain deeply skeptical of claims that the web is bound to revolutionize business or economics.

If the claim is merely that the web will cause an increase in various forms of freebie marketing, that's a testable hypothesis and one that, in my own shaky estimation, might well be confirmed. However, I don't get the impression that this sort of gradualism is what the headline "Free! Why $0.00 Is the Future of Business" is trying to convey.

That's all the virtual ink I care to spill on Free: ... at least for the time being. On to something more geekly now, I hope.

[Seven years on, it's pretty clear this has come and gone ... maybe long enough gone that someone will bring it back soon --D.H. Jan 2016]

Zero as a special price

The main thesis of Chris Anderson's Free: The Future of a Radical Price is that "free" is a special price. Reducing a price from two cents to one cent is not the same as reducing it from one cent to free.

That much, at least, doesn't seem implausible (though much of the rest does). Behavioral responses in general are often non-linear, and in any case people are not rational. Better yet, there is some evidence for it. In Zero as a Special Price: The True Value of Free Products, Kristina Shampanier, Nina Mazar and Dan Ariely describe a series of experiments exploring people's reactions to free stuff vs. merely cheap stuff.

In the experiment that seems to get the most press, Shampanier et. al. offered subjects (MIT students, to be precise) the choice of Hershey's kisses at one cent or Lindt truffles at fifteen cents. The students chose Hershey's 14% of the time, Lindt 36% of the time and nothing the other 50%.

The choice was then changed to free Hershey's* kisses vs. Lindt at fourteen cents. The ratios changed from 14:36:50 to 42:39:19. That is, many more people went for the free Hershey's, many fewer went for the fourteen-cent Lindt and somewhat fewer opted for nothing.

One thing to note here: The experiment was conducted in a booth at MIT's student center as people came and went. In such a setup it's hard to tell to what extent you're getting the same people before and after the change. The experimenters imposed a 30 minute break between changes in an attempt to minimize the overlap, since if you can't do a direct comparison with the same subject being presented each choice, it's best not to muddy the waters and instead try to make the samples as independent as possible.

This little distinction seems worth noting, since you're not measuring the response of the same person seeing the same price difference but with different prices. The people who saw the fourteen-cent Lindt would generally not have had the impression that Lindt was "supposed to" cost fourteen cents more than Hershey's. One might plausibly argue that the comparison in both cases was between Lindt at some random price and Hershey's either free or at one cent.

However, when the choice was given between free Hershey's and Lindt at a dime, the ratios were 40:48:12. Effectively (assuming the effect is statistically significant, and I'll take their word on that), lowering the price of Lindt caused people who would have paid more to instead do nothing.

Did I mention that people aren't rational?

The authors argue that the results are best explained by the theory that people see free Hershey's as being worth more than one-cent Hershey's, and that generally people assign a premium to free goods purely on account of their being free, as when people spend hours of their time standing in line for a free coffee promotion.

The authors go to some lengths to eliminate other possibilities. For example, they compare prices of two cents and 27 cents vs one cent and 26 cents vs free and 25 cents. The first two give basically the same results. The second is dramatically different, suggesting that whatever non-linearity there is lies close to the zero price.

They also conducted an experiment on Halloween in which the kiddies were given the opportunity to switch chocolate bars in various ways. Frankly, I didn't take the time to grok that one completely, but it seemed to support the general hypothesis.

The discrepancy between Lindt at a dime and Lindt at fifteen cents is an example of the more thoroughly studied tendency, well exploited by clothing designers and others, of people to perceive higher-priced goods as being worth more.

They also mention research suggesting that when the price becomes free, social norms take over from economic concerns. For example, students would take an average of four Starburst candies for a penny, but would only take one if they were offered for free.

So ...

Lots of interesting stuff here. Clearly "free" is a different price from others. This has been known to marketers forever, but the authors provide interesting data and insights.

The next obvious question is what effect a negative price would have. Would more people have gone for a free Hershey's vs. a fourteen-cent truffle, or for a Hershey's and a penny vs. a thirteen-cent truffle?

Meanwhile, back at the web, does this mean that giving stuff away online is the way of the future and previous business models are obsolete? Looking at the specific results from the candy-pricing experiments, that seems like a bit of a leap.

* All trademarks are property of their owners. Please don't sue me for leaving off the "TM" or whatever I was supposed to include.

Hive mind vision care

Here's another datapoint on search engines, even less rigorously obtained.

As I write this, the image on the screen is nice and sharp, but my eyes are telling me that the left side of my monitor is maybe 10% taller than the right side. They're also telling me that horizontal surfaces slope significantly downward to the left (the opposite of what perspective would suggest for such a trapezoid -- odd).

In other words, I'm astigmatic and I just got a new eyeglass prescription.

It's been a while since I last had my prescription updated. I don't recall the effect being so dramatic last time and the optometrist had said my prescription hadn't changed much. So I googled for astygmatism[sic -- I was sure there was a 'y' in it somewhere] new presecription [that's just a typo -- I was looking at my fingers because the screen was all wonky] tilt.

Despite that dodgy spelling, I found a helpful page from Ask MetaFilter. Granted, it said what I was thinking anyway (some degree of tiltiness for a few days is normal, but don't be shy about consulting the optometrist). Still it was still nice to get some kind of confirmation.

Ask MetaFilter is a "hive mind" site that lets you ask questions of the crowd and which gathers the answers together. I speculated previously that there are probably some sorts of question a hive mind query service would do well on. This seems like one of them. It's a common problem, and the experiences of others in the same situation are useful. Of course, this is one cherry-picked data point. See also confirmation bias.

Did MetaFilter outperform Google? Well, I found the MetaFilter page through Google. It was the second hit. The first was from some random eye doctor in DC, with roughly the same advice.

In defense of mad schemes

In these pages I've cast a jaundiced eye on such matters as the Attention Economy and the notion that information wants to be free, generally arguing that when Everything is Different Now, it probably isn't. Pixel advertising, anyone?

It may come as a surprise then that I still think it's worth something for people to try to re-imagine the basic tenets of economics and society. Occasionally something does come of it, albeit not always what the innovators expected. When it does, the whole mechanism lurches forward another small notch.

For example, the GNU Manifesto didn't kill commercial software or supplant the model of software developers making a living by writing proprietary code as employees of commercial software houses. It has, however, led to the development of a lot of good code that's readily available, and the ecosystem is better for it.

It's also worth noting that the prospective shaker-uppers are generally acting on their own initiative, putting up their own money and using their own time, and/or convincing others to do so willingly.

That said, it still seems important to remember that of all the arrows shot at the moon, most fall to ground harmlessly (and some harmfully), some snag a wayward goose, and few if any actually reach their intended target.

Required Reading

In just under 3000 words, Malcom Gladwell takes apart Wired editor Chris Anderson's Free: The Future of a Radical Price, lays the pieces out on the floor for examination, then sweeps them up and tosses them in the nearest dustbin. If I ever write such a pithy and devastating critique, of anything, I hope I may have the wisdom to quit while I'm ahead.

Along the way he articulates one of the major reasons that disruptive technology can end up being not-so-disruptive technology:

Anderson begins the second part of his book by quoting Lewis Strauss, the former head of the Atomic Energy Commission, who famously predicted in the mid-nineteen-fifties that “our children will enjoy in their homes electrical energy too cheap to meter.”

“What if Strauss had been right?” Anderson wonders, and then diligently sorts through the implications: as much fresh water as you could want, no reliance on fossil fuels, no global warming, abundant agricultural production. Anderson wants to take “too cheap to meter” seriously, because he believes that we are on the cusp of our own “too cheap to meter” revolution with computer processing, storage, and bandwidth. But here is the second and broader problem with Anderson’s argument: he is asking the wrong question. It is pointless to wonder what would have happened if Strauss’s prediction had come true while rushing past the reasons that it could not have come true.

And why isn't electricity too cheap to meter?

As Gordon Dean, Strauss’s predecessor at the A.E.C., wrote, “Even if coal were mined and distributed free to electric generating plants today, the reduction in your monthly electricity bill would amount to but twenty per cent, so great is the cost of the plant itself and the distribution system.”

Indeed. It's easy -- human nature, probably -- to get so caught up in a particular technical advance as to forget that said advance is only part of the picture and that, rather than falling like dominoes, the other pieces of the system are more likely to shift slightly and go on doing more or less what they ever did.

I have only two quibbles with the piece.

The article quotes the Credit Suisse analysis I mentioned putting YouTube's annual loss at around $470 million. As I said, the actual figure might be closer to $170 million, but I don't think the distinction is that significant to the overall argument.

More disappointing is the last sentence:

The only iron law here is the one too obvious to write a book about, which is that the digital age has so transformed the ways in which things are made and sold that there are no iron laws.

As far as I can tell, there are no more or fewer iron laws of business and economics than there were fifty years ago. On the tail of a convincing argument that digital technology isn't going to change the cold fact that people have to and are willing to pay money for things of value, it's strange to see an assertion that it has completely changed the game after all.

But no matter. It's a great review.

Free: ... is available in print for $26.99.

Step right up! You can't win if you don't play!

Suppose I'm raffling off a $1500 laptop and there are 3500 entrants in the raffle. How much should you pay for a ticket? My calculator says about $0.43.

Now suppose I'm auctioning off a $1500 laptop. Bidding starts at $0.01. You can bid the price up in $0.01 increments for the low, low price of $0.60 per bid. The winning bid is $35.

Personally I like those odds, but then I'm the one hypothetically running the auction. At $0.60 per raffle ticket -- I mean bid -- I'm getting an extra $0.17 or so per ticket, plus the $35 winning bid, for a profit of $635 on a $1500 item. And some lucky winner gets a laptop for $35. Such a deal!

There are apparently several auction sites out there doing just this sort of thing. The particular one I heard about was Swoopo, but as is so often the case the names will come and go while the game remains the same.

Web and print journalism: Curiouser and curiouser

By now everyone knows that print journalism is in serious trouble. By "everyone knows" I mean "the story has been aired widely enough that it's time to file it away under a shorthand version and move on to something else." Might or might not mean it's actually the case.

Enter Politico.com, which looks like a typical case of new media uprooting the old. Founded by ex-print journalists, it is now a go-to place for anyone who wants to find out lots and lots and lots of the minutiae of Washington politics. It came to public notice during the 2008 election but has since carved out a niche, with more White House reporters (8) than any other single organization.

As a business, it is more or less at break-even, not bad for such a young startup in an inherently cut-throat field. The punchline? About half its revenue comes from its print edition. Without it, Politico would be losing fairly serious money.

Folks, I'll admit I really don't know what to make of all this. On the one hand, pushing bits over the internet has got to be much, much cheaper than putting ink on paper and distributing the paper, even over a geographically small area. So I would tend to expect a natural evolution from print toward bits with -- and this is where I beat my not-so-disruptive technology drum yet again -- largely the same structures forming online as off, and largely the same players.

On the other hand, are the facts. The facts (in my reading, at least) bear out the not-so-disruptive part nicely. The print-to-bits part, however, seems to be going remarkably slowly, even backsliding in some instances (Politico and books published from blogs, for example). Kindle does not seem to have taken the world by storm, yet.

The obvious next step is to try to quantify all this. How much money is going to bits and print, respectively as the years go on? The analyses are out there, doubtless. Perhaps Alpha can turn them up?

How much money is YouTube not making?

I previously mentioned that YouTube is losing money. This was based mostly on having run across Credit Suisse's estimate of $470 million in losses. John Paczkowski of the Digital Daily disputes this, saying that Credit Suisse vastly overestimated YouTube's infrastructure costs and that Google has no incentive to correct this impression since doing so would lead its partners to demand a bigger share of revenue.

The argument seems plausible. One of Google's competitive advantages is its ability to buy bandwidth in bulk and its expertise in hosting hordes of servers cheaply. So, taking this into account, the picture changes considerably: YouTube is probably losing more like $170 million (the article says $174 million, but I'm not convinced I see three significant digits).

Now, if I had just told you that YouTube was losing $170 million, that might have sounded bad. But given that they might have been losing $470 million but it turns out it's only $170 million, surely that's not so bad, right? Kind of like getting a $470 pair of shoes for only $170. Such a deal, no?

The key question here is whether Google can afford it and if so, whether they want to. As far as I can tell, the answer to both is yes. With net income of around $4 billion, $170 million looks relatively small. $470 million would be harder to ignore.

Wolfram Alpha on NPR

A while ago I was given a link to an interview between Robert Siegel and Stephen Wolfram about Wolfram Alpha (thanks, Earl!). I finally got around to listening to it (thanks, mp3 player!).

I liked both the questions and the answers. I particularly thought it was interesting that Alpha arbitrarily decides that Bill James should be answered by a statistical comparison between the two first names Bill and James, while Henry James turns up a statistical analysis of the author Henry James's output.

Given that Alpha can do both, it has to decide which to do, or offer a choice. Offering too many choices makes a powerful system hard to use, which is obviously not what they're after. Most likely Alpha has Henry James in its database of authors, but not Bill James, the small irony being that Bill James is a renowned baseball statistician.

There's also a cool if somewhat obvious easter egg at the end.

Anyone(*) can become president ...

... but only 44 have. If that doesn't work out you can always put together a hit viral video on YouTube. Chris Wilson explores the odds on that one:

On Friday, May 22, I used Web-crawling software to capture the URLs of more than 10,000 YouTube videos as soon as they were uploaded. Over the next month, I checked in regularly to see how many views each video had gotten. After 31 days, only 250 of my YouTube hatchlings had more than 1,000 views—that comes out to 3.1 percent after you exclude the videos that were taken down before the month was up. A mere 25, 0.3 percent, had more than 10,000 views. Meanwhile, 65 percent of videos failed to break 50 views; 2.8 percent had zero views. That's the good news: Your video is slightly more likely to get more than 1,000 views than it is to get none at all.

The theory behind the "long tail" is that it can be just as profitable to go after (in this case) the many videos with few hits as the few with many hits. Maybe. But the 280 with no views are unlikely to bring in much. Given that there's at least some overhead per video, it's not even clear that the 6500 or so with 50 views or fewer are worth going after. So that leaves the top 32% or so, pretty much the opposite of the long tail.

Note also that Wilson is sampling all YouTube videos, including videos from record labels, clips from commercial broadcasts and so forth, not just potential viral videos.

So how does YouTube make its money? Well, actually, it doesn't.

*I feel compelled to state the obvious: Quite apart from the hurdles of actually getting elected, or even nominated by a major party, this particular sweepstakes is only open to "natural born citizens" of the US who are 35 or older (If you were a citizen when the constitution was ratified, you're also eligible regardless of where you were born. You'd probably be also able to draw a lot of YouTube hits.).

The new NASA/METI map

NASA and Japan's Ministry of Economy, Trade and Industry (METI) have just released the most complete topographic map of Earth, ever. The dataset covers 99% of the Earth's surface, from 83 degrees north to 83 degrees south, with elevation measured every 30 meters. Put this together with a good GPS and adequate storage, and you've got a never-lost device that would have seemed like sheer magic during most of recorded history.  Or so one might think.

In purely technical terms, cartography and navigation have seen revolutionary changes in the past few decades. In practical terms, the picture is a bit less clear. Most of us spend most of our time in a few, fairly small, well-known areas, a fact which leads to some interesting implications on privacy. Outside that comfort zone, a GPS can certainly be useful in finding one's way through unfamiliar streets, but a printed map will do in a pinch, or in cases of absolute desperate need, stopping somewhere and asking directions.

Similarly, commercial shipping and aviation follow well-known routes and for decades have had specialized equipment for staying on course. It's not clear what difference a better map of the Earth's surface would make.  In the air and at sea it's more important to know where you are than what's below you, except when you're landing or putting in to port.  Unless it's a dire emergency, you'll be doing that in a well-known place, and if it is a dire emergency, the new map is probably the last thing on your mind.

Radio navigation is about a century old. Satellite navigation works better and has been taking over, particularly on the seas. I'm not an expert on shipping and transport, but I have no doubt it has opened up and will continue to open up new possibilities.  In the air, the FAA and its sibling agencies are working to enable point-to-point "air taxi" services which forgo the established flight corridors and fly directly between small airports. (This has been in the works for years, though. The hurdles are not only technical, but legal and economic).  In either case, technology is making a difference, but the real difference is in navigation, not mapping.

So ... the new map database is definitely cool, and making it freely available to the world is even cooler. If I'm planning an expedition to Patagonia or the Yukon, I'll definitely want to consult it. For my morning commute, not so much.

[Rereading, it seems to me that elevation every 30 meters is only of so much use without some indication of roads, place names and so forth, and 30 meters is plenty of room to hide all sorts of hazards.  As wonderful a research tool as I'm sure the map is, it's probably adds little or no value over previous maps when considered as a navigation aid.   The NASA/METI press release in the news tends to confirm this: the gaps filled in were mainly in "very steep terrains and in some deserts", as one might expect --D.H. May 2015

This is the rare article that I've edited more than lightly on re-reading.  The original version shifted abruptly to radio and satellite navigation without really saying why.  I've tried to make that clearer.  I may not have reconstructed the original argument, but I think the current version at least makes some coherent argument --D.H. Jan 2016]