Bandwidth is the new coverage

Something caught my ear on the radio today: an ad from a major cell provider twitting the competition for making all sorts of claims about speed (and then claiming to be the fastest themselves ... so there!)

Not so long ago the battle was over coverage, but evidently coverage is now about as good as it's going to get.  The internet connectivity of a phone is now a bigger selling point than the mere fact that you can call people with it.  If you're outside the US, or just more into smart phones than I am, this may well be old news.  I tend to use my phone predominately as a phone and an alarm clock, probably because I almost always have access to WiFi and a decent-sized screen and keyboard [Hmm ... this one might be worth a followup -- D.H. Dec 2015].

So far I've found a smartphone most useful as an easily portable GPS, an application for which I'd gladly trade speed for coverage.  A GPS function without a usable map is not much use if you're lost out in the boonies.  But maybe that's just me.



What also caught my ear about the ad was that carriers appear to be claiming to be "X times faster."  Faster than what, I'm not really sure, but it sounded reminiscent of internet providers claiming to be "X times faster" than dialup.  Even with that nice low bar set, providers had trouble providing quite what they promised.  Perhaps cell providers will do better in that regard?

Ajai Chowdhry on IT in Africa

Real journalism is not an easy gig.  Your job is to report news, which means things nobody knows about yet.  Old news is no news, so you need to figure out what's going on as quickly as you can.  Extra time spent researching or editing is time that the story's not getting published.  Then you get to go on to something else you don't know about, though at least it will usually be in the same general vein.

So I feel a little bit bad as a casual blogger about picking on the CNN headline writer who characterized Indian IT entrepreneur Ajai Chowdhry's comments on African infrastructure as "Why broadband not roads will transform Africa."  But only a little bit.

From what I read, Chowdhry isn't saying that broadband is more important than roads.  The main assertions I get are that Africa's problems and solutions are the same as India's; India has had an advantage in being a single state instead of 53; because of their similarities and decades of close relations, doing business in Africa is not difficult for an Indian company; Africa represents a huge business opportunity; African manufacturing has a huge native market to supply; African unity will only help Africa's economy and stature in the world; and, yes, broadband and the web could and should play a major role in addressing African poverty.

Chowdhry mentions roads in one passage at the end:

But the one area where Africa can make a big difference is by not just looking at putting up roads -- it should look at putting up internet broadband-type infrastructure.

In other words, both are important, and perhaps broadband is being overlooked.

My point here is not to bash on CNN.  As I said, putting this all together is harder than it looks.  Rather, it's that Chowdhry's broad and well-developed view of the situation, from the standpoint of a key player in IT, provides a good perspective of how the web fits into the overall picture of economic development.

Given that Chowdhry is in the business of IT and clearly and openly hopes to gain from helping develop African IT, it's particularly notable how broad a view he presents.

More cheesy movie goodness

Since I've admitted to watching watching Ghostbusters II recently, I suppose it will do no harm to admit to having watched Hackers as well. Hey, I missed it the first time around. Skipped it, actually, on the grounds that I would have a hard time accepting its depiction of computers and networks or the inevitable Markov chain of random technical terms.

I was right about the cargo cult computing but, being perhaps older and wiser, much more able to bear it. It's a fun movie to look at, with plenty of whizzy graphics. Come to find out much of it was done with motion-controlled models, as the CGI of the time would have looked too artificial. In movie logic, dancing equations and morphing false-colored talking virus heads on 90s-era hardware are the most natural thing in the world, because they provide atmosphere.

The movie does a surprisingly good job of capturing the gray hat hacker ethos, however much it fails to convey the paint-drying dullness of most forms of hacking to (non-hackish) spectators. I've always found it unfortunate that "hacker" has come generally to mean not hacker programmer but black hat hacker or even script kiddie (fortunately, we have "geek" these days covering roughly the same territory that "hacker" used to). On that front Hackers may have done more good than harm. They also manage to mention the Dragon Book, so props for that.

I didn't pay much attention to the plot because, well, it didn't seem like that kind of movie, but I did react to two of the film's more famous howlers. One was the RISC/CISC confusion in the Pentium scene (if you want to know more detail than that, you should probably just watch the film). The other was when the main characters gush over a 28.8bps modem.

Ah yes, this is a blog about the web, isn't it? I'm getting there ...

As IMDB duly points out, they meant 28.8kbps, not 28.8bps. But that's just a typo and I heard it as 28.8kbps at the time. What caught me was that, more than the cassette tapes, minifloppy disks, haircuts and rollerblades, it's the bandwidth that sends the whole thing horribly off the rails. Never mind the rest of the implausibilities. They're supposed to be doing all this over 14.4 or slower? Oy. Guessing passwords, sure, if they're weak and easily guessed, but if they ever have to download more than a few hundred K, they're doomed.

It's the bandwidth that makes the modern web what it is. Thanks to YouTube and company, the video of the villain's face on the hero's laptop screen seems perfectly plausible until you remember what size pipe it's all supposed to be going through. In fact, much of the cyberstuff, except for the phreaking (the technology has moved on) and the social engineering (ageless and evergreen), is probably more plausible now than it was then, thanks to the web.

TWC and Fox in the new year

As of this writing it appears that the game of chicken that Time Warner Cable and News Corp have been playing is likely to end up with the two splatting into each other at high speed. News Corp wants more cash for carrying its popular TV programming, including US college football bowls and sitcoms such as It's Always Sunny in Philadelphia. Time Warner Cable (now independent of parent Time Warner) claims it would have to pass along $1 per subscriber and is leery of setting a precedent that the other major TV networks could then use to their advantage.

So far there has been no agreement and there's every possibility that Fox channels such as FX, Speed, Fuel and Fox Soccer Channel -- but not Fox News or Fox Business, which have separate contracts -- could simply go dark to TWC subscribers at midnight tonight.

So what does this have to do with the web? For one thing, TWC is also a major ISP (full disclosure: they're mine, in particular). For another thing, Hulu (part-owned by Fox) will be happy to show you Sunny and other FX shows (not sure about the sports), whether over TWC's pipe or someone else's. That's particularly interesting since Hulu is supported by ads, and declining ad revenue is one of the reasons Fox wants to charge cable companies like TWC more. Another might possibly be that Fox wouldn't mind cable subscribers dumping cable in favor of an online service Fox has a piece of.

Nah.

Surfing up and down the east coast

In a previous life, I made a couple of long-haul bus trips across the American southwest. It was an option worth considering for someone with more time than money -- an undergraduate, say -- provided the traveler was someone -- an undergraduate, say -- who didn't mind sitting and occasionally half-sleeping in cramped quarters in the company all manner of interesting people. On the one hand it's slow, something over 40 hours of nearly continuous driving to get about halfway across the continent. On the other hand, you see all sorts of things you'll completely miss in flyover mode. That cuts both ways.

Here in the 21st century it's a whole different game, at least in the Boston-DC corridor. Two carriers, BoltBus and megabus.com, are offering service between major cities aimed not only at scruffy undergrads but also at business travelers. What makes it work?

• Speed is not such a problem in the target market. Flying will still be considerably faster for the longer routes, even counting time to and from airports, but DC - New York is nowhere near as long a haul as LA - Albuquerque.
• The bus is still way cheaper.
• These new busses are spiffy, new, less cramped than Ye Olde Greyhound and ... you knew there it had to be in here somewhere ... webby.

Not only do you book online and bring an email confirmation as your ticket, you can surf while you're on the bus at no extra charge. Check your social networking site, fire off a few emails and tweets, who knows, maybe even get some work done while you're en route.

How does it work? I wasn't able to track down the exact mechanism, but I would have to guess a series of WiMax towers along the route feeding the on-board WiFi.

How well does it work? Well, BoltBus's FAQ cautions that "This technology is new, and there are spots on the trip where the service may be unavailable. We also do not advise downloading large files, as the speed will be relatively slow [...] Plug in and Wi-Fi disclaimer: BoltBus makes every effort to provide these services free of charge to every passenger. However, if, for whatever reason, the service is unavailable we are unable to supply a refund."

Well, whaddya want for a nickel?

I'm not sure which company's transparent attempt to sound like Bus 2.0 works better. You'd think that either CamelCaseNames or domainname.com would be about to fall out of fashion any day now, so I'll give the edge to BoltBus for not using mega.

[Looks like both of these outfits are still in business --D.H. Dec 2015]

The shape of the backbone

I was all set to write a somewhat philosophical post comparing the internet backbone to the phone system that came before it, when I realized I know very little about the internet backbone. Since I claim not just to be writing about the web, but actually figuring it out (however slowly) as I go along, that seemed like a pretty embarrassing lapse. So I've started doing a little basic digging and, so as to have something to post while I'm doing it, pass along some of my findings.


There was a British general once who, whenever he was sent to a new theater of battle, would start off by making a crude, not-painstakingly-to-scale map of the major roads and rail lines in the territory, noting the distances between the most important cities, junctions and other points of interest. This exercise would pay dividends later, not just in the day-to-day running of the campaign, but in preventing blunders on the order of depending on troops to cover a week's worth of distance in two days. It would also give some idea of the political structure of the place -- who was likely to be trading with whom, who might not care too much about what was going on where, and so forth -- and help greatly in deciding which supply lines to guard where, which enemy supply lines to attack where, etc., etc.

Along those lines (minus the military perspective), it seems useful to start with a very large-scale look at the international internet backbone, with an eye toward who's connected to whom and with what capacity. Physical distance is not as important here, though latency can matter in some cases.

Here's just such a map, dating from 2005, prepared by TeleGeography research for the International Telecommunications Union. Please don't look at the part that says "Proprietary and Confidential". There are probably more recent maps available* but this should give a reasonable impression. A few things that jump out:

• North America <-> Europe dominates, followed by North America <-> Asia with roughly 40% of that capacity, followed by everyone else, far behind. In particular, there is very little capacity, relatively speaking, directly linking Europe and Asia. If you're in Bangalore talking to Bucharest, you may well be going by way of San Francisco.
• Capacity to and from Africa is almost negligible on this map, but see below.
  
• Capacity on these scales is currently measured in Gbps (the document gives 5- and 6-figure numbers in Mbps, but most of that precision is probably spurious). That's pretty big compared to my home connection, but not really stunningly large considering how many people are involved. By comparison, the human visual system appears to have a capacity on the order of 1Gb/s, so the 500Gb/s pipe between North America and Europe is, in some notional sense, roughly equivalent to 500 pairs of eyes. Put more realistically, though, 500Gb/s is about 125,000 DVDs playing simultaneously.
• What these numbers actually mean is a different question entirely. Suppose for example that everyone in the US wants to follow the European football leagues online -- not that that's going to happen anytime soon. Would those millions of viewers saturate the pipe? Hardly, since the main pipe only has to carry a few video feeds at any given time.

As I said, the big missing piece in the 2005 picture is bandwidth between Africa and the rest of the world. As of this summer, that bandwidth has gone from practically nothing to about 1.3 Tb/s ,or 1300Gb/s to put it in the same units used above. That's over twice the size of the Europe <-> North America pipe, a classic example of leapfrogging technology assuming we're comparing apples to apples. Even if we're comparing apples to pears it still looks like a pretty big pipe.

(*) Back in the days that bang paths roamed the earth, one could subscribe to a newsgroup which published frequently updated maps of the backbone as it then existed, meaning mainly a bunch of T1 and similar lines. A T1 line could carry about 1.5 Mb/s, or .0015 Gb/s, or about 0.0003% of the Europe <-> North America link.

Global sneakernet? Not likely.

Previously on Field Notes: A pigeon is found to outperform a broadband network. This turns out not to be an isolated or implausible incident. Further, it looks like Moore's law is on the side of the pigeon; the amount of information a pigeon could reasonably carry is increasing much faster than broadband speeds.

How far can this go? Will the skies soon be clogged with data-laden birds (and the streets below with their by-products)? Will startups soon be offering a higher-tech equivalent, perhaps autonomous mini-drones carrying terabytes or more at a time?

I'm guessing not, and for one reason that should be pretty clear: latency. Deutch's fallacies strike again. The physically-moving-medium approach has always had its niche, namely when volume is more important than latency. This is one reason the mails still run -- another being that sometimes you just need to send a physical object. Most of the time, though, latency wins. If you're sending the message, "attack at dawn", it hardly matters whether you could get the entire contents of your video library halfway around the world by noon the next day.

Just where the cutover occurs depends on a variety of factors. For an interesting case, see the earlier note on kinescopes at the bottom of this post.

Classic software engineering and the mighty pigeon

A diligent member of my army of stringers, researchers, fact-checkers and miscellaneous hangers-on forwarded me a BBC article on a contest between South Africa's broadband carrier and a carrier pigeon. The pigeon carried a 4GB memory stick 60 miles in 2 hours. In the same amount of time, the broadband connection had transmitted 2% of the data. Pigeon 1, broadband nil.

Is this really news? First, how fast was the broadband connection? 2% of 4GB is 80MB. 80MB/7200s = 11KB/s. OK, that's pretty slow. For comparison, I just ran a speed test against a server about 60 miles away. The download speed was about 14Mb/s, or about 2MB/s. That would bring me my 4GB in about 40 minutes. Pigeon 1, broadband 1.

But wait. I can only download as fast as the other end can upload. If the other end has my broadband connection, it can upload at about 360Kb/s or about 45KB/s. You read that right. My upload speed would appear to be about a fortieth of my download speed. That's about four times the speed of the South African connection, meaning I couldn't even get 10% of the data transmitted by wire before the pigeon reached its destination. Pigeon 2, broadband 1.

Hmm ... my ability to send large quantities of data -- movies, for example -- to the world at large is severely limited, but my ability to access said data from whoever can make it available quickly isn't. And my internet connection is provided by a Cable TV/old-school media company ... but I digress.

When I got the link about the South African pigeon, I immediately thought of Jon Bentley's classic Programming Pearls and More Programming Pearls. If you're interested in software engineering you could do worse than to stop reading this right now, go hunt down copies of these books and inhale them. The code samples are variously given in C, C++ and a procedural pseudocode reminiscent of Old High Algol, but just as Chaucer is worth the trouble of reading in the original, so too Bentley.

If you don't want to hunt down paper copies, check out the web site [The web site doesn't carry the entire book, but it's still well worth visiting. The book itself has been extensively updated in the recent second edition. There's even a little Java here and there, but only a little. The Labs is still the Labs, after all].

I assume you're back now and along the way noticed problem 11 in column 1 of Pearls:

11. In the early 1980's Lockheed engineers transmitted daily a dozen drawings from a Computer Aided Design (CAD) system in their Sunnyvale, California, plant to a test station in Santa Cruz. Although the facilities were just 25 miles apart, an automobile courier service took over an hour (due to traffic jams and mountain roads) [Ah, Highway 17. Good times, good times.] and cost a hundred dollars per day. Propose alternative data transmission schemes and estimate their cost.

The solution Lockheed came up with?

The computers at the two facilities were linked by microwave, but printing the drawings at the test base would have required a printer that was very expensive at the time. The team therefore drew the pictures at the main plant, photographed them, and sent 35mm film to the test station by carrier pigeon, where it was enlarged and printed photographically. The pigeon's 45-minute flight took half the time of the car, and cost only a few dollars per day. During the 16 months of the project the pigeons transmitted several hundred rolls of film, and only two were lost (hawks inhabit the area; no classified data was carried). Because of the low price of modern printers, a current solution to the problem would probably use the microwave link.

Pigeon 3, broadband 1.

For several obvious reasons I doubt that pigeons are going to be the optimum solution for most high-volume data transmission problems, but it certainly gives one pause to note that a generation after the Lockheed story, in the face of Moore's law and all that, pigeon power is still a plausible solution. At least compared to what passes for broadband here in the States.

Why should this be? Moore's law cuts both ways. In fact, it currently favors the pigeon. The South African test was done with a 4GB memory stick. Sticks of 16GB are now available. Leaving aside the question of whether a pigeon (or two) could carry more than one stick, even a single pigeon with a single 16GB stick could beat my download speed over a 60 mile course.

Memory sticks are getting bigger much faster than broadband is getting faster, if only because switching to a larger stick is much, much easier than switching broadband technologies.

This is all reminding me of my early posts on Jim Gray.

The BBC's Broadband World

I just ran across this piece while browsing headlines. I haven't watched the videos but the chart below the map, showing average advertised speeds, seems just as interesting. In effect, you have

• Japan, with close to 100Mbps
• South Korea, with about 90Mbps
• France, with around 50Mbps
• Everyone else, with 20Mbps or less

The US is low on the list at around 10Mbps, or about an order of magnitude less than Japan. The caption on the video for the US puts the (relatively) pathetic state of things down to the free market approach taken, but I don't see how the chart data bears that out. We're not slow because we're capitalist. We're just slow.

March Madness, baby!

Every spring here in the US, the NCAA hosts its college basketball tournaments, and every year some many-digit number of dollars of productivity is lost as employees (and bosses) try to keep track of how their brackets are doing. The effect is particularly pronounced in the first round, a glorious orgy of 32 games, roughly in batches of four, spanning a Thursday and Friday in late March. The games go from about noon to midnight on the East Coast, and thus pretty much through the workday on the Pacific side.

What does this have to do with the web? Every year there's a bit more Madness online. This year, CBS is streaming every game for "free" on ncaa.com, using Silverlight. The quality is good (not that anyone at my place of work would have checked, of course), but "free" means you get a commercial at the beginning, an obnoxious animated ad off to the side, and the usual further commercials during what used to be the "TV time outs" and are now the "media time outs."

There's also the "boss button" that throws up an imitation spreadsheet. This must be just for laughs. If it actually fools your boss, you obviously need hoops to liven up what must otherwise be a stunningly dull workday.

All of this has been in place for a while, but each year's version is a little slicker and a little smoother as the media players get better and the net becomes better able to handle large loads of streaming video.

Can't watch the video? Well, you could always stream audio, or just check an automatically updating scoreboard. Playing the office pool? You can do that online, too. The site will do the busywork of keeping score. Some will even show updated scores of games in progress. Nice.

Frivolity? Sure (unless the Jayhawks are playing). But it's also a nice test case and driver for the technology. You have everything from streaming media down to nice UIs for showing scores and tracking brackets, all aimed at a mass market that, collectively, is very picky about usability. If your site is clunky, millions of sports fans that should rightfully have been your customers will find one that isn't.

Just the sort of testbed for improving the infrastructure to pave the way for ... um, well, March 2010 comes to mind ...

How disruptive is online advertising?

A Forrester Research report quoted in a Wall Street Journal article, neither of which seems easy to access on line, says that Kids These Days spend more time online now than they do watching TV (apparently a good chunk of that time is spent gaming). Widespread adoption of broadband connections (or at least, considerably-faster-than-dialup connections) has been a big driver for this.

This is causing both major advertisers and major online advertising players to re-think how best to reach consumers in the new higher-bandwidth net.world. In the case in point, Proctor & Gamble and Google are going so far as to exchange employees, the better to understand each other's cultures and outlooks. An odder couple you couldn't ask for, and yet it appears to make business sense.

On the one hand, this is just another chapter of the "How do we make money off this 'web' thing?" saga that's been playing itself out slowly but surely for the last decade or so. But on the other hand, it gains a bit more urgency when rephrased as "We need to make money off this 'web' thing. The other stuff is drying up!"

My feeling continues to be that the web will have much more impact on how companies make money than on which companies are making it. That certainly seems to be the lesson from the dot.com boom and bust: WebVan folded but brick-and-mortar grocery stores still take orders online. EToys got bought out by KB Toys and Walmart and Target went online. Not to say that new companies haven't sprung up -- Google, Amazon and EBay come to mind -- just that old companies adapting has been more the norm.

In fact, the how is not necessarily changing that much. P&G still makes money selling soap, grocery stores still sell groceries and toy stores still sell toys. The big difference is in how they reach their customers. And even then, an ad still looks pretty much like an ad and an online catalog still looks a lot like a catalog.

Another data point on immersion

A while ago I estimated the bandwidth required for 3-D Imax at 13GB/s, uncompressed. Today I got the latest newsletter from the California Academy of Sciences bragging about the new and improved Morrison Planetarium. They say this bad boy can blast 300 million pixels per second onto its hemispherical screen. At 32 bits per pixel, that's about 1.2GB/s, or about a tenth of my IMAX estimate (I think I used 32 bits per pixel to ensure a conservative estimate of the bandwidth required. 24 ought to be good enough).

Take out a factor of two since 3-D requires two images and assume that the frame rate is 24 frames/s in both cases. The remaining difference is down to spatial resolution. IMAX is about 10K by 7k, or 70 megapixels, so the Morrison is more like 14 megapixels. In the earlier article I guessed that the 13GB/s could probably be compressed down to 1GB/s, partly because the two 3-D images would be largely redundant. Planetarium-level 2-D video would also compress, but not quite as much. Bottom line, you're still looking at gigabits per second to be able to handle this stuff.

The Academy also claims that the panetarium would hold 2,402,494,700 M&Ms, but I'm skeptical about either the volume of an M&M or the planetarium, much less both, being known to a part in 10 million.

Bringing Greene County on line

Here's a fairly heartening story, reported by NPR:

Five years ago, the US government stopped subsidizing tobacco farming. At the time, Greene County, North Carolina was more dependent on tobacco than all but one county in the US. Faced with disaster, the county government made what seemed like a most improbable choice for a place with little money and big problems coming down the road. Instead of trying to attract a factory to a work force with few skills outside tobacco farming, they would bring in broadband and increase internet coverage from practically nothing to the entire county. They would also supply laptops to all high school students and make other educational reforms not detailed in the story.

The results were dramatic: many more students applied to and attended college, teen pregnancies fell, business improved and instead of falling off the map, the county was able, in the words of county government worker Misty Chase, to "jump over the industrial age and move straight into the technology age." On a less tangible level, the kids had iTunes and folks were able to email their relatives outside the county. No one is claiming that the internet and laptops were the only factor in the turnaround, but it's clear it wouldn't have happened without them.

In a world of hype about the net and the web changing everything, it's easy to be skeptical of such claims and easy to forget that, just as with rural electrification and universal phone service, technology really can make a difference.

Broadband: A somewhat broader view

CNN reports on a recent Pew study on broadband penetration. The thrust of the article, headlined "Broadband Internet? No thanks," is that surprisingly many dial-up users don't want broadband. The body of the article, however, suggests that it's not that people don't want broadband, but that it costs too much. Or, slicing it a bit finer, they don't see enough added value for the added price.

If you look at the numbers in the study, you get yet another somewhat different picture. The great majority of people who have had dial-up have now switched to broadband, and evidently a fair number have skipped dial-up entirely and gone straight to broadband. If broadband is stalling at all, it isn't stalling for lack of interest or even because of price, but because the low-hanging fruit has already been picked.

The CNN article is focusing on the minority who still have dial-up or who don't have any internet connectivity at all. The article makes sense in that context, but it's a pretty narrow context. The headline writer still loses either way: Of the reasons given for not switching, "nothing would convince me" ranks a distant second to "the price would have to fall."

The Pew study's headline reflects the luxury of not having to write for a mass market on a tight deadline: "Adoption Stalls For Low-Income Americans Even As Many Broadband Users Opt For Premium Services". Not quite as snappy as "Broadband Internet? No thanks," but it does seem more in line with Pew's numbers.

Right at the top of the Pew report is a graph showing

• Broadband penetration rising from near-zero in June 2000 to 55% now.
• Dialup penetration rising from 35% in June 2000 to 40% in April 2001, then falling to 10% now.
• Total internet penetration rising from 35% in June 2000 to 65% now.
• Broadband adoption rising faster from March 2007 to March 2008 than it did from March 2006 to March 2007.

On the last point, note that the time axis of the graph is not to scale, and that there seems to have been some leveling off since December 2007.

As the CNN article and the Pew headline suggest, in the cases where people don't have broadband, price is a factor when availability isn't. It would be interesting to know for what portion of people who can't afford an internet connection is it because they can't afford a computer to hook up to it.

There's one more point I'd like to pull out here. While broadband still costs more, broadband has been getting cheaper over the past few years while dial-up has been getting more expensive. This doesn't seem surprising. Dial-up, which converts digital data to an analog signal for transmission over infrastructure that's now mainly digital, is inherently less efficient, and at this point the economies of scale are no longer on its side.

The new black box in town

Netflix has finally announced its long-rumored set-top box [reviewed here]. The box, made by Roku, is $99 and the service is $9 a month (it's included with any Netflix subscription except the $5 entry-level special). There are a couple of catches:

• Obviously, you'll need broadband. The service promises "near DVD-quality", that is, somewhere around a megabit. I don't know if they can send you HD if you've got the bandwidth for it but if you live in the States, you probably don't so the question is moot.
• The selection is limited. In particular, the studios are still leery of putting out their latest releases on the internet, so you won't get those. In all, about 10,000 titles are available, compared to the 100,000 they offer by mail.

Hmm ... anyone remember when movie theaters had one (count it, one) screen and there were three TV networks plus the odd UHF station playing Our Gang, The Munsters and Laurel and Hardy re-runs? Now 10,000 titles is "limited". Progress, I suppose.

"Broadband" in the states won't support live HD, but if the box had enough storage on board you could get HD sort-of-on-demand. Ask for your selection when you leave for work and it'll be there when you get home, probably, assuming no one else wants to do much with the internet connection during the day. Faster than waiting for a DVD in the mail, but interestingly, not a lot faster.

I would be surprised, though, if the box had anywhere enough storage on board for a full HD movie. My guess is it's enough to buffer a few minutes in case of net.hiccups, and Wikipedia's entry on Roku (the manufacturer) seems to support this.

Even at near-DVD resolution and a "limited" selection, I'm expecting the things to sell like hotcakes. For about the price of a premium movie channel package you get orders of magnitude more selection with comparable picture quality (if the quality isn't at least comparable, the whole thing will be dead on arrival). If they do sell, the projected video flood of the internet comes one sizable step closer.

In the case of households with cable, a chunk of video traffic that now goes over the provider's system and then down the cable to the TV will shift to going over the backbone to the cable provider and thence to the TV. In other words, the net shift is from the provider's system onto the backbone. The obvious solution is to put Netflix's servers at the upstream end of the cable (but see this post for a slightly different take).

Whether this happens depends on how the cable companies feel about their newfound "co-opetition". If you're in the bandwidth business, you love Netflix's box. If you're in the content business, maybe not so much.

The National New York Geographic Times

A little bit ago I mentioned a late-80s Media Lab brainstorming session in which someone suggested that, given "effectively infinite" bandwidth of 500Mb/s, one could have "a daily National Geographic-quality New York Times with today's news rather than yesterday's, manufactured at the breakfast table."

I wanted to take another look at how that stacks up with where we really are, 20 years later, because I can see two opposite conclusions one could draw, equally not-quite-convincing:

• It happened. Printed newspapers and magazines are not gone, but it's quite possible to get one's news entirely from online sources, up to the minute, with pictures and even video (The video's a bit on the small and grainy side, but at least you don't have to have someone standing by the TV set with a coat hanger, a roll of aluminum foil and one hand out the window. Progress.)
• It's nowhere near happening. No one even makes a mass-market printer that will produce broadsheet-sized, glossy magazine-quality copy quickly and (just as vital) cheaply. A few dozen pages at magazine-quality resolution is probably running into gigabytes even with compression (photos take way, way more space than text, so the question is what portion of the page is photos). That's a daunting number for most of us.

I'd synthesize these thus: It didn't happen exactly as envisioned, but the basic shift toward real-time, media-rich news is very much ongoing. The high-definition version hasn't happened partly because the bandwidth isn't there, but arguably one reason the bandwidth isn't there is that high definition is not nearly as important as immediacy.

How much bandwidth does a man need, Mr. Tolstoy?

Earl comments that we probably already have enough, or nearly enough bandwidth to saturate human capacity, but not enough to satisfy our human desire for new shiny pretty things. Or at least I hope that's a reasonable paraphrase.

I would say it depends on what kind of information you're trading in. If you're dealing in metadata like who knows whom, or dealing in random facts like the score of every major sports event in the world, or in text like newspaper copy, even a dial-up connection can feed you information faster than you can process it.

If you're dealing in sound, a dial-up connection can certainly carry voice (that's what it was designed for, after all). Decent stereo sound requires more like 100kb/s (mp3 takes about a megabyte a minute). That's a bit beyond dial-up, but cable and DSL can handle it.

Analogously, if you want YouTube-quality video, you can certainly get that over cable or DSL, but plain DVD quality, at 4Mb/s, needs what's still an uncommonly big pipe. And that still leaves plenty of room before we hit the limits of perception. Those finely-tuned eyes of ours are bandwidth-hungry.

On the other hand, as Earl points out, you hit diminishing returns well before you hit the limits of perception. That last Mb/s of bandwidth doesn't improve the experience nearly as much as the first one. The difference between regular TV and HD is not nearly the difference between TV and no TV (and some would argue whether that last one represents an improvement, either).

Storage hardware is already entering a new regime of plenty. You can now get a terabyte disk off the shelf. My spell checker flags "terabyte", but soon enough we'll be throwing around (I'm guessing) "T" like we now throw around "gig" and "meg". Borrowing from those displays you see in the camera/music player department, a terabyte can hold:

• A million minutes, or more than two solid years, of music.
• Hundreds of thousands of high-quality photos.
• Hundreds of hours of DVD-quality video.

For less than the cost of a humongous monitor to watch it on, you can now get enough disk to store your entire DVD collection and (likely) every home video you ever shot, plus your entire song collection. Or you could store every phone conversation you've ever had in your life (phone quality requires much less than mp3, in case you've spent more than two years of your life chatting).

If you have dial-up at 56Kb/s, it would take 4-5 years to fill a terabyte, assuming the connection is running full-tilt, constantly, with every bit recorded on the disk. When you're done with that, you'll probably be able to pop in a 10TB disk for the same price and keep going ... The question is, will you still have a 56Kb/s dial-up connection?

In the big happy family of Moore's law, bandwidth into the house is the poor cousin (processing power is the crazy uncle in the attic, but maybe I'll get to that later). At some point, bandwidth will reach the point that disk is reaching now, where we'll actually have to think a bit about how to use it all. Anomalously, that point still seems fairly far off.

But then, how much do we really need? In Tolstoy's famous story How much land does a man need? (which I haven't read), it turns out you need about three feet by six feet. By six feet deep.

Brand and The Media Lab, 20 years after

Ah, here's the quote: Stewart Brand's The Media Lab, page 68. Media Lab's director, Nicholas Negroponte, has just described a half-gigabit channel as "effectively infinite" and asks what one would do with that much bandwidth:

The Media Lab's best and brightest stuttered. One volunteered you could have a combination of television immediacy and newspaper depth and detail available at any time through such a medium. Fine, that would occupy a teensy fraction of the bandwidth. What else? Uh, every house acquires its own broadcast capability. Uh, you could broadcast solids by having them fabricated at the receiving end. Uh, everything would be instantly available as super fax -- a daily National Geographic-quality New York Times with today's news rather than yesterday's, manufactured at the breakfast table.

That was written 20 years ago. It's tempting to take potshots at past speculation about what was then the future, but let's not. Predicting the future is hard. Myself, I'm hard put to predict the present.

On the other hand, some interesting questions do arise:

First question: What happened to the bandwidth? The Media Lab folks were stuttering 20 years ago because representatives of Bell (I mean Lucent ... I mean AT&T .... I mean Alcatel ... I mean ...) had just told them that they could run 500Mb fiber optic to everyone's home. That's around 60 megabytes/second, both ways.

There is no official definition of broadband, but current reports (such as this one from the OECD) use 256Kb as a working definition. By that measure Denmark and the Netherlands are leading the way (with Switzerland, Korea and Norway rounding out the top five). In Denmark, there is just over one broadband subscription per three inhabitants. If you factor in households with more than one inhabitant, this probably means that almost everyone who wants a broadband connection has one.

However, most of those subscribers have DSL (it doesn't say what flavor). If you're actually looking for fiber, Korea has about 9 subscriptions per 100 inhabitants, Japan about 8, Sweden about 5, Denmark about 3 and the Slovak Republic about 1. So much for fiber to every home. Most of us, it appears, are slogging along with DSL and cable. Your mileage may vary depending on a number of factors, but even the theoretical maximum listed here is nowhere close to 500Mb/s for either.

How did this (not) happen? That's an interesting story having more to do with economics than technology ...

Next question: Suppose we eventually all get a half-gigabit hookup. What could we possibly do with that? The answer seems very clear in hindsight: Blow it all on extra pixels. Late-'80s TV is one thing. HD-TV is quite another. Blu-Ray and HD-DVD play back at about 40Mb/s, easily handled by our infinite pipe, but they're not the ultimate.

Even HD-TV has visible artifacts if you look closely enough. Personally, I don't much care, but the true videophile is going to want more. How much more? Recall that IMAX is multiple gigabytes per second uncompressed, and (at a guess) around a gigabyte compressed, still over twice infinite capacity.

I doubt, though, that higher definition is why most people will want a bigger pipe. I suspect most of us who want a bigger pipe just don't like to wait. Sure, I could watch HDTV over a mere 50Mb/s connection, but who wants to wait for a program to download in real time? We want it now (because we're going to watch it later -- go figure). Now even the infinite pipe doesn't look so infinite. It's still going to take 10 minutes to download a feature film in high definition. Of course, if I can watch a movie while I'm downloading 10 more, that might take some of the sting out of it.

If you buy the "no-waiting" theory then we don't want or need infinite sustained bandwidth, but we would really like infinite peak bandwidth. Give me a gigabit pipe, and I promise I'll hardly ever use it. Or maybe I will. Usage has a way of catching up with capacity, and it seems risky, at the least, to predict otherwise.

Finally, what about the speculation in the original quote? The broadcasting solids idea has an obvious weak point: you'd need a pretty special printer to go with that (modern machine tools can reproduce solids described digitally, by any of several processes, but they're not available for home use).

As to the others: We actually do have something like the "television immediacy and newspaper depth," at least by one plausible definition, in the blogosphere and in conventional news sources. The high-def New York Times is certainly feasible. You could even print it out if you really wanted to. It's another question, and an interesting one, why no one seems to be putting out that sort of content to the mass market.

IMAX 3D vs. broadband

Watching a concert film on IMAX 3D is an interesting experience. You get a much better view, for a stadium show, than anyone actually in the house -- sort of like you're somewhere close to the stage and really tall and able to float up level with or above the performers at random intervals. The sound is great, and perfectly synchronized, unlike sitting up in the nosebleeds with a slight-but-noticeable lag between the big screen and the speakers. You can hear the virtual crowd roar around you, even if the actual crowd around you is quiet.

The screen is big enough and close enough that you can't quite take it all in at once. If something darts forward off to one side, you have to turn (slightly) to see it in focus. You don't have to turn as far as you would in real life, another slight but unavoidable disconnect between being fully immersed and sitting in a theater with a huge screen and great speakers.

Note to directors: The 3-D cameras love the drum kit -- all those cylinders poised at various angles -- but don't overdo it. There's also another slight disconnect here. The drumsticks strobe noticeably since even IMAX is still 24 frames per second.

Now for the interesting question: How many bits?

IMAX film has a resolution of approximately 10,000 by 7,000. Assuming 32-bit color, 24 frames per second and 2 cameras, that comes out to about 13 gigabytes per second, uncompressed. There's ample room for compression, particularly in 3D since the two images are largely identical, but you're still talking on the order of a gigabyte per second. Picture throwing two blu-ray DVDs into the maw of the beast every minute and you're in the ballpark.

Leaving aside the small matter of installing an IMAX home theater, could you at least stream the bits into your house? If you happen to have 10-gigabit ethernet or better coming in, you're good to go. 75-year-old Sigbritt Löthberg of Karlstad, Sweden does, thanks to her son, Peter (see here for slightly more details). I don't, and you probably don't, either.

On the other hand, gigabytes are getting cheaper every day. Last I looked, hard drives were running around $0.30/GB. A 90-minute movie would require about 5TB of disk (5400GB at 1GB/second), over $1000 retail. That's probably viable for theaters now -- IMAX film reels are massive -- but not quite ready for home use.

Lies, damn lies and benchmarks

Computeractive magazine in the UK recently put up a broadband speed checker and asked readers to send in their results. Come to find out, typical speeds are a far cry from the "speeds of up to" numbers given by the providers.

More than half of respondents reported getting less than half the advertised speed. Naturally, users are not best pleased. Spurred by the reaction Computeractive is calling for government regulation of broadband advertising.

On the one hand, truth-in-advertising laws can help provide clarity when sellers have little incentive to provide it themselves. On the other hand, I'm curious just what sort of measurement will end up being required. The whole problem with the current advertising is that it provides a single, easy-to-digest number instead of a more nuanced picture that allows for (among other things) congestion at any of several points.

I wouldn't be surprised to see some sort of standard "speed rating" hashed out, perhaps with separate flavors for DSL-like, cable-like or WiMax-like services. I wouldn't mind being a fly on that wall.