The future still isn't what it used to be: Doug Engelbart

If you're exploring the origins of the web and pondering how early ideas ended up incorporated -- or not -- into what we see now, sooner or later you're going to run into "The Mother of all Demos", a 1968 presentation by Doug Engelbart and company demonstrating the oN-Line System, or NLS for short, produced by the Augmentation Research Center at the Stanford Research Institute.

I watched it a few months ago. Having somehow missed it for all these years, I went in cold and -- deliberately -- without researching what I was supposed to be looking at. I took copious notes, in some cases replaying to try to make sure I got things right ... and then got distracted by other things. What follows is not fresh and detailed, but I want to at least include something about the demo, so here goes ...

NLS was directly inspired by Vannevar Bush's Memex concept, so if nothing else it's an interesting case study in what happens when you try to turn someone's architectural vision into a real system.

Everything's in glorious black and white, the text on the screens is UPPERCASE and clunky, with actual capital letters indicated by an OVERBAR, and some of the participants look more than a bit like the 1980s stereotype of nerds, because that's where the 1980s stereotype comes from. Actual 1980s nerds were hopelessly uncool in completely different ways (so I hear).

On the other hand, Engelbart is wearing a headset mic, moving a cursor (the team called it a "bug") around on the screen using a mouse, pointing, clicking, narrating the whole way through, linking up with colleagues in a different county in real time, cross-fading and compositing live video and generally giving the appearance of an ordinary livestream, just in black-and-white and what now qualifies as period costume, and without a discernible boss level.

About 20 minutes in,  Engelbart says "I'm going to do something called 'jump on a link'" and explains that a link points to a particular location in a particular file with a particular "view type" saying what kind of content you're looking at -- yeah, that's hypertext.

Physically, NLS is nothing like Bush's original description of a desk-shaped piece of furniture holding a library of microfilm and some sort of magnetic recording medium. More than twenty years had passed since Bush's original article and whole new technologies had developed in the interim, particularly in computing hardware.

NLS was running on a 24-bit SDS-940, clocked at somewhere on the order of 0.1MHz, with approximately 200KB of main memory, 5MB of swap space and 100MB of disk(-ish) storage. It had a text and graphic-capable display and a 1200-baud modem for connectivity.  It also ran QED (Quick EDitor), the very first text editor I learned to use (a bit later, though), and a direct ancestor of vi, which I still occasionally use.

Tiny as this might seem today, I'd argue that there's a bigger gulf between it and 1945's ENIAC than between it and equivalent commercially-available equipment from 1991 (as many years from NLS as NLS was from ENIAC): say a farm of a couple dozen 80MHz SPARCstation 2s, each with 128MB of ram and a few GB of disk. Yes, the SPARCstations could do a lot more, but the SDS has essentially the same pieces: Enough RAM to be meaningfully programmable (as opposed to requiring switches to be set manually), offline storage, connectivity^*, a graphical display, the ability to connect a variety of peripherals and so forth, which the ENIAC has essentially none of. The hardware available when NLS was developed was essentially a smaller, slower version of the hardware that Web 1.0 was built on. Nothing of the sort was available when Bush was thinking up Memex.

NLS is clearly recognizable, almost 60 years later, for what it is: a computer system. Memex, just as clearly, is a thought experiment, though one with enough detail to make it the next best thing to a demo.

To be sure, even if the difference between NLS hardware and today's is mainly a matter of quantity, the quantities have changed by a lot. Today's hardware is several orders of magnitude beyond the SDS, and there are several orders of magnitude more computers in the world now, and they are connected by networks several orders of magnitude faster than 1200 baud. That all has to make a difference, and it has. I hope to dig into this a little more deeply at some point.

There is another key difference between NLS and Memex. Memex is intended for a single person working alone. NLS is inherently multiuser and connected. While the actual demo takes place in San Francisco, the system itself is running in Menlo Park, about 50km/30mi away and probably about as much of a pain to drive to and from then as now. After a fair bit of introductory material, Engelbart is joined by Bill (English, or possibly Paxton?) in Menlo Park, and the two proceed to edit shared documents together.

In the post on Memex, I argued that this difference is one of the main reasons that as far as I'm concerned Bush did not invent the Web. I wouldn't say that Engelbart's team invented the Web, either, but just as the SDS-940 has much more in common with modern computers than with ENIAC, NLS has much more in common with the modern Web than with Memex.

Along with demonstrating the NLS hardware and software, Engelbart made a point of discussing the project that produced it and the approach behind it. That approach was

• Empirical: Try things out and see what works
• Evolutionary ("steepest ascent"): You can't see everything in advance, so take a heuristic approach and look for highest-impact changes at every point
• Whole system: While different people had different skills and responsibilities, the project itself encompassed hardware, software and anything else needed to produce a working system
• Bootstrapping (later called "eating you're own dogfood"): The people developing the system used the system. Past a certain early point, further development of the system was done in the system itself.

"It's a struggle doing it that way," Engelbart said, "but it's beginning to pay off."

I get the feeling both parts of that were knowing understatements.

What we now call dogfooding is a powerful approach that provides a constant reality check, but with the pitfall that it can skew toward the researcher's use cases. A system that's useful for developing software might or might not be useful for other things. Engelbart appears to have been aware of this hazard. One of his prominent examples is his own shopping/errand list. With its repetitions and loose categorization it looks natural. Maybe it wasn't his real list, but it feels like it could have been.

Besides making the system more relatable by giving an example that the audience would be familiar with, constructing a loosely-organized shopping/errand list on the fly demonstrates the system's ability to deal flexibly with free-form content.

This wasn't a given. I could easily imagine someone trying to demo a more rigidly structured system: "OK, now first I create a shopping list template ... Oh, I forgot about vegetables. Let me go back and add a vegetables section to the template. Now where was I?" as the audience's eyes glaze over. Engelbart just puts a list together. It doesn't hurt that he's a fairly engaging speaker and is intimately familiar with the system, but even taking that into account, it doesn't seem like the system is putting arbitrary barriers in his way.

That's not to say that using NLS would be simple for someone encountering it for the first time. After a while, you come to understand that Engelbart and company have memorized quite a few one-letter commands and established quite a few conventions to help find their way around. This is a classic ease-of-use vs. ease-of-learning tradeoff (it's not always a tradeoff, but I'll leave it at that before I start mumbling things about Pareto optimality).

There are some high-level goals that read a lot like what we would now call mission statements:

• Improve the effectiveness with which individuals work at intellectual tasks
• as a subgoal: better use of human capabilities
• Develop a system-oriented discipline for designing the means by which greater effectiveness is achieved

Again, Engelbart makes a point of calling that last point out. Just as important as producing a usable product is producing a way of developing such products.

NLS is meant to be "an instrument/vehicle helping humans to operate within the domain of complex information structures". While it's not explicit in this statement, the demo makes clear that, unlike Memex, NLS is intended for collaboration. It's also subtly larger in that it aims at "humans" rather than "men of science".

Engelbart puts forth several principles that sound surprisingly modern:

• "content represents concepts" Note the use of "content". It didn't start life as a marketing term.
• "structure represents relationships, or human-thought product"
• said structures are "too complex for direct human study", that is, the computer is taking on some of the weight of dealing with a complex structure
• there is a "control metalanguage" note use of "meta". Again, not a new term (well, it goes back to Aristotle's Metaphysics at least).

So it's 1968, Doug Engelbart has just demonstrated hypertext, computers are starting to get networked together, a bunch of standards ending in "TP" for "Transport Protocol" are about to be developed (FTP is announced in 1971) ... so we can expect the Web as We Know It to appear sometime around ... 1991? Wait, what? That's like, 20 years later.

Audience member Andries (Andy) van Dam's assessment of the impact of the demo was that computing mostly went on unchanged afterwards. Besides writing one of the most important early textbooks in computer graphics, van Dam, along with Xanadu founder Ted Nelson -- and I only just now learned this while researching this post -- developed HES, the Hypertext Editing System, at Brown University in 1967. As far as I can tell this was independent of the NLS work, though clearly the two teams would have known about each other.

Van Dam's assessment notwithstanding, several NLS alumni ended up at Xerox PARC (Palo Alto Research Center), which produced, among other things, the influential object-oriented language Smalltalk and a bitmapped mouse-driven GUI ... and these went nowhere until, so the story goes, Steve Jobs stopped by and basically grabbed the whole concept for the Apple Lisa.

The Lisa also went nowhere for a while, but was later reworked to the Macintosh, which eventually took off. I had the fortune of stumbling across a demo of a Lisa at school. This being a tech school, the hardware-oriented folks talked the Apple rep into taking the cover off so they could poke around. The Apple rep, though visibly nervous, was generally a good sport. But I digress.

The famous Macintosh ad was in 1984, 16 years after Engelbart's demo, itself 18 years after the project was initiated. We'd like to think we move much faster now, but do we, really, once you measure comparable points of development: someone's initial concept, start of project, first major demo, first real product, with error bars representing variability in how long things take?. I hope to dig a bit deeper into that as well.

*Though NLS used its modems to allow people to connect to it, computers were already connecting to each other. The ARPANET project was already underway and would be live in a year or two.