Tools of choice

In real life I'm a software developer. That doesn't figure in much here, probably because as far as the web is concerned I'm an ordinary user, not a developer. However, one place I use the web is at work. No, not to browse fascinating articles from the blogosphere, unless the article happens to answer a particular vexing question I'm dealing with. My web use at work basically boils down to

• gmail
• a web-based bug tracking system
• searches now and then for answers to vexing questions
• researching and downloading open source software

This last bullet item has significantly changed the way software developers work, at least in the Java corner of the world where I dwell. Your mileage may vary, but for a large and still-growing set of typical problems, downloading a package and using it is likely to be a better option than rolling your own. You can't beat the price, and the time between "that looks interesting" and actually using the package is generally measured in minutes. There's no obligation and if the package is not quite right, the source code is right there.

Some examples from the toolkit I use at work

• Java itself and its libraries are now essentially open source.
• The Eclipse IDE. Now, I realize that IDE wars are to our time what editor wars were to the previous generation (um, that would be my generation, I guess), but Eclipse is the one I happen to use for a variety of reasons. One caveat: Eclipse is not just an IDE. It's really a whole platform. It slices. It dices. It has distros like Linux has distros. If you're not careful you can end up with a bloated mess. If you pick and choose, though, you can end up with a very nice, usable, though still memory-hungry tool.
• Subversion for version control. Again, other worthy choices are available.
• JUnit. The value here is not so much the code as the mere fact of putting something out there as a framework for writing unit tests. That said, I've had no complaints about the code.
• Apache Ant for builds. I actually don't use Ant directly these days, but I rely on it behind the scenes. Having seen one too many Makefiles that ate Chicago, I have no plans to go back to make.
• Apache in general for a variety of useful libraries, including networking (Mina) and general utilities (Commons)
• A new favorite for taming Swing: MiG Layout. If you've ever considered fleeing to a tall mountain in Nepal rather than hassle another mysterious problem with GridBagLayout and its little friends, check MiG out. Your life will become better.

Naturally this is just a particular, idiosyncratic view of what's out there. If you run Linux (as I do at home), there's the whole GNU/Linux/git/gcc/autoconf/gmake/gcc/... toolchain. If you like Perl, or Ruby, or Python, each is its own little universe. Any way you slice it, the amount of stuff out there is impressive.

Back at the blog, is this a real live example of disruptive technology? If so, what is the disruptor? Is it the concept of open source? Is the enabling technology the internet, the web, or some combination of both? How much does it matter that much of the internet and web as we know it rests on open source/free software? Why am I carefully saying "open source" here and not "free"? How many threes are there in a dozen?

All interesting questions except perhaps the last, but not ones I'm going to tackle just now.

[I still use Java, Eclipse and JUnit.  I'd now recommend git over Subversion for version control.  For various reasons, I don't have much occasion to use the rest of the list these days. --D.H. Dec 2015]

The point of the postscript

In a previous rumination over Open Source, I quoted Linus's original Usenet post announcing the beginnings of Linux. I left it sitting there somewhat ambiguously, and since a certain amount of ambiguity can be useful, and since this is a blog, not a wiki, I'm going to leave it that way in the original and comment on it here instead.

The point I was making is this: The post strongly suggests Linus thought he was just throwing stuff at the net, but this is highly ironic in light of what actually played out. Little did he know ...

One might go further and speculate about why one would want to put something like that out and invite feedback. My personal guess is that Linus's post cautiously understates what he thought his as-yet-unnamed OS might become. This, in turn, vastly underestimates what it actually has become.

Revolution OS and thereabouts

OK, so I just watched Revolution OS (on the Roku/Netflix box, of course), which I'd been putting off out of concern it might be more propaganda than information. The opening minute or so did little to allay that, but it turned out to be a pretty good documentary, and as even-handed as you could expect from something that interviewed Open Source folks entirely. It did this by getting in touch with several of the principals, including rms, esr and Linus, and pretty much just letting them talk. This is often a good idea, especially when the principals involved are thoughtful, creative, articulate and intellectually curious.

What emerged was a clear picture of the history of Free Software/Open Source, how "Open Source" came to be the dominant name, and the essential differences between the two: Free Software advocates want all software to be free because it's a Good Thing. Open Source advocates want particular software to be open because it's a Useful Thing. It may not surprise the attentive reader that I tilt toward the latter.

There are ironies along the way, for example a small one in Netscape adopting Open Source not through grassroots activism by engineers -- though this did occur -- but because it was eventually imposed from the top down by management; a large one in that the entire Open Source movement, which is at best indifferent to rms' s central goal of making all software free, depends crucially on GNU code and even more crucially on the GPL [more precisely: on the GPL and licenses directly influenced by it]. Rms himself points this out in his acceptance of the Linus Torvalds award at the 1999 Linux world. Linus's daughters trot back and forth behind him on stage all the while.

If you're looking for a spirited debate over Open Source vs. not-open, you won't find it -- except for an early quote from Bill Gates (who did not directly participate), there are no dissenting voices. If you're looking for knock-down, drag-out Linux vs. Windows, as the marketing collateral implies, you won't find that either. And a good thing. Revolution OS is a much more a chance to put a human face on the names you see floating around and get an idea of what they were thinking. Considered from that point of view, it succeeds nicely.


But I didn't really set out to write a movie review here. I really just wanted to share something amusing I ran across while chasing a link from a link from a page I looked up out of curiosity after watching the movie. This is from Jamie Zawinski, who has done more Open Source development than most of us, I would wager. Zawinski says:

But now I've taken my leave of that whole sick, navel-gazing mess we called the software industry. Now I'm in a more honest line of work: now I sell beer.

Specifically, I own the DNA Lounge nightclub in San Francisco. However, it takes quite a lot of software to keep the place running, because we do audio and video webcasts twenty-four hours a day, and because the club contains a number of anonymous internet kiosks. So all that code is also available.

This all sounds fine and noble, and I like the design decisions, but I have to wonder: Just how anonymous can an internet kiosk be in a nightclub full of webcams? Checking sports scores without having to establish an account anywhere? Sure. Plotting world domination? Maybe not so much.

By the way, this turns out to be post number 300. I made a production of 100 and 200, but from here on out I probably won't until some more significant milestone. Hmm ... 100π is about 314 ...

Open source vs open source

This piece arose out of a discussion on a mailing list I'm on. The discussion was about, among other things, why open source projects fail. By the time I finished a particular email, I realized I was basically blogging (not coincidentally, my output to that particular list dropped considerably when I started blogging here). Here's what I wrote, toned down a bit and edited lightly to take the later discussion into account:

It occurs to me there are two kinds of open source. First, there is stuff someone put together and threw out to the web. Much of this of limited value. There might be a good idea in there, but chances are it's not really new, or it's not fully worked out. It might or might not be well coded. It's probably indifferently documented. The developer works on it whenever, so there's no release schedule. For my money, such projects don't fail -- they never really start.

Then there are projects like Apache, or Mozilla, or Eclipse or Red Hat, or the various Google offerings, that are developed and supported full time by some sort of durable entity [Later in the discussion I unwisely called this a corporate entity, in a legalistic sense. This proved to have too many overtones, so I switched to "institution," which is still not ideal.] Generally there's commercial money in it one way or another, but the important quality is that there are numbered, scheduled releases and there are multiple people working on it as part of their day job.

But what about Linux, Python, Perl, the GNU tools and such? They may have started out in the first category [Re-reading, no. They weren't just thrown out on the web. They were generally well along before the world saw them -- but see the Postscript below for an interesting twist], but at some fairly early point the person behind them made a conscious decision to move to the second. Linus could have decided "Hey, that kernel thing was cool. I think I'll do something else," but instead he spearheaded the move from 0.9.x to 1.0.x (note -- release numbers) and has stuck with it up to 2.6.x. The Linux kernel arguably has one of the least formal and most distributed development processes of the major projects I'm aware of, but even then there is a single gateway for significant changes and if Linus should get hit by a bus, there are known people who could take over.

Python's Benevolent Dictator for Life, Guido van Rossum, works at Google, where he spends half his time on Python (evidently 50% for Guido is 20% for everyone else). BDFL Larry Wall's early development of Perl took place during his employ at JPL. While Perl is another of the less formal examples, there is still an elaborate structure around the development and release of Perl.

Rms took the more formal route, founding a foundation and drafting the famous GNU licenses. Being rms, he secured his own money for it, some of it thanks to a grant from another foundation, the MacArthur foundation. This is not a path lightly traveled, but the destination, again, is an institution dedicated to supporting the software.

In short, whenever something significant has happened in open source, it's because someone explicitly pushed for it. The actual coding, testing, documentation etc. might be distributed and more or less volunteer, but at the bottom (or top, if you prefer), there is a small, single point of control. This point of control tends to become institutional, that is, an entity distinct from any individual, fairly quickly.

There is a "religious" aspect to open source that many people, including myself, instinctively distrust. It relates to the notion that open source stuff "just happens" and that if you just throw stuff out on the web, or maybe even just hope it will happen, you'll magically get a robust, coherent and useful product. Experience shows, not surprisingly, that this just doesn't happen. Generally, a project has to start with a robust, coherent and useful product before a culture can emerge around it.

PostScript: My pondering on this topic keeps returning to Linux. On the one hand, it definitely supports the idea that good free software doesn't just emerge out of the web, but requires the ferocious dedication of a single person or small group. On the other hand, it has had a less romantic version of the idea of informal, distributed development in it from the beginning. Linus's GIT source control system, which deliberately has no central repository, is a more recent manifestation. In this light, it's interesting, not to mention somewhat amusing, to read Linus's original Usenet post to the world:

Hello everybody out there using minix -

I'm doing a (free) operating system (just a hobby, won't be big and professional like gnu) for 386(486) AT clones. This has been brewing since april [i.e., a few months], and is starting to get ready. I'd like any feedback on things people like/dislike in minix, as my OS resembles it somewhat (same physical layout of the file-system (due to practical reasons) among other things).

I've currently ported bash(1.08) and gcc(1.40), and things seem to work. This implies that I'll get something practical within a few months, and I'd like to know what features most people would want. Any suggestions are welcome, but I won't promise I'll implement them :-)

Linus (torvalds@kruuna.helsinki.fi)

PS. Yes – it's free of any minix code, and it has a multi-threaded fs. It is NOT portable (uses 386 task switching etc), and it probably never will support anything other than AT-harddisks, as that's all I have :-(.