A couple of updates on Satoshi

In the previous post I said it seemed "really odd" that Wright hadn't publicly demonstrated he had Satoshi's private keys by signing a message¹.  Wright has since said that he lacks the "courage" to do this, and at least one of the Bitcoin experts who had previously said that Wright was certainly Satoshi has since walked that back.

The simplest explanation here is that Wright doesn't have the key.



Re-reading that post, I see I commented to the effect that, while Bitcoin is often described as a means of anonymously transferring money, that wasn't necessarily so.  I said I'd get back to that, but didn't really, so here's a bit more:

Bitcoin's blockchain very publicly ties a given transaction to a timestamp and the keys of the sender and recipient (and probably other stuff I'm leaving out).  That's quite a bit of information to mine, even though the keys don't have real-world identities attached to them.  For example:

• If you see a series of transactions with the same sender and recipient over time, you can assume they're doing business.
• If you know that someone ordered a doomsday device online for one million dollars, and you see exactly one Bitcoin transaction for $1,000,000 in a reasonable time period around the purchase, that's a pretty good clue that that wallet is associated with the sale of a doomsday device.

The doomsday device is a contrived example, of course.  A real evil genius wouldn't be so obvious (though a henchman might ...)  The buyer and seller could, say, break the transaction down into unremarkably-sized pieces and use separate wallets for each part.  Nonetheless, the cardinal rule of anonymity on the web is that you've probably given away more than you think you have, if your adversary is really looking.  There are protocols built on top of Bitcoin to mitigate this, but Bitcoin itself makes no guarantee.

Another weak point is that at some point you need to get reserve currency in and out of a the system, unless you really, really believe in Bitcoin as a currency in its own right.  The easiest way to do this is via an exchange, which ties your transactions to a particular bank account.  It would not be wise to assume that these accounts can be kept perfectly anonymous.   Also, if there are relatively few people converting Bitcoin to and from reserve currency, the fact that everyone else has to go through them will leave its mark on the transactions in the blockchain one way or another.  Again, I'm sure there are ways to mitigate this, but they're not built into Bitcoin.

But then if you're buying and selling doomsday devices online, you really don't need my advice.




¹In Bitcoin circles the preferred method would be to transfer some small amount of Bitcoin from Satoshi's wallet.  This process includes producing a digital signature using Satoshi's key.

On the interwebs, no one knows you're Satoshi Nakamoto

Several major news outlets have recently announced that Australian Craig Wright has come forward as the person behind the pseudonym Satoshi Nakamoto, inventor of the Bitcoin protocols.

I have no particular reason to doubt this, but more on that in a bit.  First, an update on Bitcoin.

In previous posts I've expressed skepticism that Bitcoin, considered as a currency, would come to supplant reserve currencies such as the Dollar, Pound or Yen.  I think that case is made by now.  The main points are

• The current Bitcoin money supply is a fraction of a percent of the dollar money supply
• The dollar value of Bitcoins actually spent in a day is an even smaller percentage of the number of dollars spent in a day
• While several major retailers have announced that you can "pay in Bitcoin", this generally means they're partnering with someone who converts Bitcoin to and from a reserve currency.
• The merchants themselves quote prices in a reserve currency and take payments in that currency.
• Bitcoin's exchange rate with reserve currencies is much, much more volatile than reserve currencies' exchange rates with each other
• As a result, there is little evidence that anyone holds on to Bitcoin except as a speculation
• All of this has been the case for years now

That doesn't mean that the whole exercise has been a waste of time or, for that matter, that a few people haven't made large piles of (reserve currency) money from it.  Bitcoin may not be a viable currency, but it's interesting considered as a payment method.  In particular, the "blockchain" used to provide a public hard-to-forge record of transactions, has seen interest from a number of players, including existing banks.

It's worth noting that the blockchain doesn't provide the anonymity that Bitcoin is rightly or wrongly known for.  After all, by design it's a public record.  But before we get to that, let's do a quick review of public key cryptography with the caveat that, although I've studied cryptography with some care, I'm far from a cryptography expert.

Public key cryptography uses a publicly known algorithm and two keys: a public key that everyone can see and a private key that only you should see.  In the usual version, RSA, the two keys are "inverses" of each other.  If you take a message and use one key to apply the algorithm, you get gibberish.  If you apply the other key to the result, you get the original message back.  This means you can use the system in two ways:

• Encryption:  If I run a message through the algorithm using your public key then (as far as we know), only you (or at least, only someone in possession of your private key) can turn the resulting gibberish back into the original message.
• Signing: If you run a message through the algorithm with your private key to get gibberish, then anyone with the public key can recover the original message but (as far as we know) only you (or at least, only someone in possession of your private key) could have produced the gibberish that you sent out.

Typically these are used together, along with a cryptographic hash function that can take any message and boil it down to a largish number -- a few hundred digits -- in such a way that (as far as we know), no one could have produced a fake message that boils down to that same number.  Putting it all together, if I want to send you a message securely, and assure you that I actually wrote that message:

• I use the hash function to boil my message down to a largish number, called the hash value, that (as far as we know) could only have been produced from my message.
• I use my private key to turn that number into a different largish number, that (as far as we know) only I (or ... you get the picture) could have produced and (as far as we know) could only have been produced from my message.  I add this "signature" to the bottom of my message.
• Finally, I use your public key to turn the whole thing¹ into gibberish that (as far as we know) only you can decrypt, using your private key.
• After you do this, you use my public key to turn the signature back into the original hash value, and you use the cryptographic hash function to verify that the message hashes to that same value.

What do I mean "as far as we know"?

All of these algorithms and protocols are widely published and widely studied by everyone from undergrads to renowned experts to the folks at No Such Agency and its counterparts.  This is a key tenet of modern cryptography.  Only private keys are private.  Everything else is public.  "Security through obscurity" is frowned upon.  While it is possible that someone has cracked one or more pieces of this system, it's unlikely for several reasons:

• The same entities that would be cracking the system are using it for highly sensitive information (they might also be using some secret system for really really sensitive information, but the information known to be secured using the widely-known systems is pretty sensitive -- military communications, money transfers ...).  If you know your system can be cracked you have to assume your enemy could figure that out as well.
• There are known ways to test whether your crypto has been hacked, mainly noting carefully who acts on what secret information (whether real information or carefully selected false information).  Military history has several famous cases of battles deliberately lost to avoid giving away that the enemy's code had been cracked.  Eventually, though, that information will be put to use.  Not long after that, the jig is up.
• No academic researcher has published a significant crack of the current "best practice".  By contrast, several no-longer-widely-used systems have been cracked in the literature (MD5, anyone?).  Cracking a widely-used system is a pretty good way to advance on the tenure track and/or line up a lucrative job and/or develop serious street cred.

If someone has cracked the current public-key encryption infrastructure, they're using that information very carefully.

OK, how about my other disclaimer, "or someone in possession of your private key".  Key management is where we separate the pros from the amateurs -- and even the pros get it wrong way more often than they'd like.

Even if you're using some sort of uncrackable-in-principle quantum cryptography, at some point you're going to want to actually read a message encrypted to you and put your signature on something you're sending to someone else, and to do that you need some sort of key.  With current technology, typically that key is a file that's been encrypted using a passphrase.  If I want to decrypt or sign something, I put in the passphrase and the crypto software decrypts my private key to use for the real task at hand.

There are several weak points here.  An attacker could steal my key file and guess my passphrase.  An attacker could install a hacked version of the crypto software that sends the decrypted private key to the attacker, or encrypts the message with the attacker's public key along with the real recipient's.  An attacker could install a hacked version of the crypto software that uses a not-so-random number generator in places where we expect an unguessable random number to be used (see the footnote below for more on how those last two would work).

This is just off the top of my head.  Real experts spend a lot of time coming up with more sophisticated attacks and ways to prevent them.  The point is that no matter how strong the actual encryption algorithm, there needs to be a key of some sort, and there are plenty of ways to steal or tamper with the keys.

So if someone comes forward claiming ownership of a private key that was created years ago and used in a very visible way and that could be worth hundreds of millions of dollars, you may want to be a little careful before taking such a claim at face value.

Apparently, the only publicly available cryptographic evidence of the claim that Craig Wright is Satoshi Nakamoto is a digital signature that could have been copied from the publicly available blockchain.  This seems really odd.  You don't prove you have a private key by showing something already signed with that key.  You prove it by taking some new message of someone else's choosing, signing that and having the signature checked, ideally by yet a third person.  And anyone claiming to be Satoshi ought to know that.

On the other hand, reliable resources report that Wright has done essentially that using a different key that only Satoshi should have.  Additionally, Bitcoin experts who have met Wright claim he's the real deal -- he acts like Satoshi, he has the technical knowledge that Satoshi would have, he knows historical details that only Satoshi is likely to know, and so forth.

This is not to say that Craig Wright is lying or otherwise up to no good.  I have no reason at all to believe that or imply that.  My point is that that it would be a mistake to judge his or anyone's claim to be Satoshi Nakamoto based on cryptographic evidence alone.

So ... who knows?  At this point, we will probably never know for sure who Satoshi Nakamoto is or was, or even whether it was only one person.  Which seems somehow appropriate.

Postscript: The private key published by Craig Wright was used in an early block -- block 9 -- in the Bitcoin blockchain, in which a quantity of Bitcoin was transferred to the late Hal Finney.  As it happens, Hal Finney commented on a post on this blog.  The post was on anonymity, but his comment led me to his blog, where I saw a bunch of interesting stuff on digital rights management (DRM).   Naturally, I posted a reply to that, rather than his actual comment.  You can see the whole exchange here.

¹In practice, you don't use the public key system directly for large messages.  Rather, you use the public key system to encrypt a randomly generated key for a faster, non public-key system that is then used to encrypt the actual message.  Besides being faster, this also allows you to send a message to multiple recipients without having to generate a separate encrypted version of the whole message for each recipient key.  Instead, you encrypt the randomly generated key for each recipient.  I trust you can see why I left all this out of the main post.

On the responsibility of "flash mobs"

Hmm ... where did I put that big honking I AM NOT A LAWYER disclaimer?  Ah ... here it is.

OK ... where was I?

In an old post about a pillow fight that got out of hand, I speculated about the responsibilities of flash mobs.  One point that the original post only mentions in passing is that the pillow fight was only a flash mob activity by the loosest of interpretations.  It was, after all, already an annual event, pretty much the opposite of a spontaneous occurrence.

So why call it a "flash mob"?  While the event was scheduled for a definite place and time (Valentine's Day in Justin Herman Plaza) and now even has a Facebook page, the event itself is open to anyone who happens to show up.  There are no tickets and there is no official organizer or organizing body.  If you show up with a pillow on Valentine's day and start swinging, you're in.  Otherwise you're not.

Leaving aside some interesting questions of identity and language usage for the other blog, it seems that the key point here is that people sometimes gather unofficially to do things, they've been doing that forever, and, most important, being unofficial does not absolve anyone of responsibility.  If I get together with ten close friends and twenty people they invited and 35 people those people invited, it doesn't matter whether we did this over the web, or whether I know everyone there.  It matters what we do.

If we decide to go clean up a city park, good for us.  If we decide to trash the same park, we're responsible for that instead.  Which is why the original headline, "S.F. may crack down on 'flash mob' antics" misses the point.  As the article itself made clear, the city had a particular case of how to deal with a not-officially-sanctioned group of people making a mess.  Nothing particularly flash-mobby or webby about it.

Print ... yet again still not dead

A few years ago I speculated on what it might mean that politico.com was getting a good share of its revenue from its local print edition in DC -- evidently it was the print revenue that was keeping the magazine afloat.  Politico is still around, including the print edition.  I don't know whether the print edition is still critical to the operation, but it says something that it's still around years later.

There's an even more blatant example, though, one that I don't recall noticing earlier even though it's been around since 2005: WebMD has a print edition.  I've seen it in a couple of doctor's offices in the past few months.  I'm sure I've seen it many times before and just not registered it.

It makes perfect sense, of course.  Patients waiting in doctor's offices are the classic captive audience.  Even today, when people are likely to have smartphones and/or tablets to read from -- or could just bring a book like in the olden days -- it's clearly still worth it to have a pile of paper around to browse through.  A medical magazine aimed at the general reader makes perfect sense.  If you're into sports or celebrity gossip you've probably already read the stories in those months-old magazines, but chances are you haven't browsed through WebMD, no matter how old it is.  Being in a doctor's office, you might well be in the mood to.

As with Politico, it's particularly interesting that a primarily web-based outlet -- you can't get much webbier than WebMD -- is choosing to publish a print edition, and sticking with that decision for years at a time.

Prediction update

In mid-2009 I said:

Personally, I still don't see I Robot coming to life any time soon, but I do see things that got written off as impossible during the dead-and-debunked phase starting to stir to life again. I'm thinking, say, competent machine translation or robots that can pick things up and carry them around a house, happening gradually in the next decade or so.

So we're about 70% of the way there now.  I retried the round-trip translations from this post with a popular translation tool and got considerably better results, maybe even what you might call "competent".  I don't see any signs of thing-lugging robots appearing in the stores as mass-market consumer products in the next couple of years, but I wouldn't be surprised if someone somewhere is selling such a thing before long -- or has already.

That's the nice thing about vague predictions -- it all depends on how you count.

Wikipedia considered harmful ... or not

In an old post on Wikipedia, I said that

[I]t's easy to spot a backwater article that hasn't seen a lot of editing. This is not necessarily a bad thing. Obscure math articles, for example, tend to read like someone's first draft of a textbook, full of "Let x ..." and "it then clearly follows that ..." The prose may be a bit chewy, but whoever wrote it almost certainly cared enough to get the details right.

My feeling was that if you were really interested in, say, the functoriality of singular homology groups,  you'd probably have enough context to chew through prose like "This generality implies that singular homology theory can be recast in the language of category theory."*

In a recent ars technica article, John Timmer argues that impenetrable technical articles are actively harmful: "The problematic entries reinforce the popular impression that science is impossible to understand and isn't for most people—they make science seem elitist. And that's an impression that we as a society really can't afford."

I think that's a good point, but I'm not sure how bad the problem really is in practice.  A hard-to-read article is most likely to be harmful if a lot of people are seeing it, which also makes it more likely that someone will be able to improve it.  This is a fundamental assumption of Wikipedia in general, I think.  As such, it would be interesting to see some data behind it -- is there a strong correlation between the number of times a page is landed on and the number of edits (or edits per word, or such)?

Assuming that correlation holds, then someone coming to Wikipedia to learn about, say, physics should have a good chance at a gentle introduction.  Let's try:

• The main article on Physics seems like a perfectly good Wikipedia page.  It starts with a general introduction, goes into history, core theories, relation to other fields and so forth.  Let's look at one of those fields:
• Condensed matter physics still seems to be in good shape.  The first sentence doesn't seem completely useful at first: "Condensed matter physics is a branch of physics that deals with the physical properties of condensed phases of matter," but the next paragraph goes on to explain nicely what a condensed phase of matter is.  The rest of the article continues in a well-structured way to give an outline of the field.  Let's look at one of the theoretical aspects:
• Symmetry breaking "needs attention from an expert in Physics".  I'd agree with that assessment.  The general idea is still there, but we're definitely getting technical: "In physics, symmetry breaking is a phenomenon in which (infinitesimally) small fluctuations acting on a system crossing a critical point decide the system's fate, by determining which branch of a bifurcation is taken."  For example, what's a bifurcation?  Well, we can at least chase the link and find out:
• Bifurcation theory is actually a better-structured article, but no less technical.  It's actually a math article, not a physics article.

I would say that either of the last two would be intimidating to a non-physicist/mathematician.  I don't know if you could say the same about the first two.  Yes, there are still technical terms and concepts, but it's pretty hard to get away from that and still cover the material.  I would also say that a non-physicist interested in physics in general would be far more likely to land on the Physics article than the other three.

I also noticed that, while I have run across a few really impenetrable technical articles in Wikipedia, it didn't seem -- in this particular random walk, at least -- that the quality of the articles dropped off steadily as one went off the beaten path.  Fields intersect, and perhaps you're never too far from someone's beaten path.  I did chase a link from Bifurcation Theory to Stationary point, which was marked "The verifiability of all or part of this article is disputed", not something one expects in math articles, but it didn't seem particularly better or worse than the previous two, the warning notwithstanding.

Let's say that the random walk above is fairly representative -- and I think it is, based on other experience browsing Wikipedia.  What of Timmer's claim that general interest articles such as The Battle of the Wilderness are accessible, while technical articles such as Reproducing kernel Hilbert space are hostile?

I suspect Selection bias.  Timmer (and myself, and anyone who browses a lot of technical articles) sees a lot more technical articles than the average reader.  In fact, we should broaden that a bit and say "specialized" instead of "technical".  Just as math geeks might read a lot of math articles, history geeks will read a lot of history articles, sports geeks a lot of sports articles and so forth.  Should we really expect Reproducing kernel Hilbert space to be as newbie-friendly as The Battle of the Wilderness when the battle site has a plaque on a public highway talking about it?

Let's try following History like we did for Physics above, since Timmer's example of an excellent article is from that field.  The first area of study is Periodization, and already we see a notice that the article's sources remain unclear.  A large chunk of the article covers Marxian periodisation, giving the almost certainly false impression that this is the only significant way that historians divide history into periods.  This section seems to be a copy of the corresponding section in Marx's theory of history, suggesting that one of the most basic Wikipedia cleanups -- making sure that the bulk of the information is in one definitive place and everything else links to that place -- hasn't been done.

Just as impenetrable math or physics articles may give the impression that scientists are a bunch of elitists, the article on Periodization -- the second history article I tried -- may give the impression that historians are a bunch of leftists.  So maybe Wikipedia as a whole is unfriendly to academics as a whole?  That would be a sad irony indeed, but I doubt that's really what's going on.  Wikipedia is filling more than one function, after all.  It's a general introduction in some places, and a specialized reference work in others.  That seems fine.

There's a lot more to be said here.  Maybe survey articles like the ones on physics and history are the wrong place to look -- not that we've really defined what we're looking for.  I tried putting "The battle of" into the search bar and clicking on a few of the battles that came up.  All the articles seemed quite well written.  Perhaps the Wikipedia process works better for documenting specific events?  Or perhaps the search bar was just showing me the most popular articles, which in turn would be popular for being well written and well-written for being popular?

Overall, the proposition that Wikipedia articles on the sciences are bad for the sciences seems like a testable hypothesis, at least in principle, but properly testing it requires a lot more machinery -- methodology, statistical models, surveys, etc. -- than you'll find in a blog post or op-ed.  Without a more thorough study, we should take assertions like Timmer's as good starting points, not conclusions.

* I should point out that the article I linked to is not necessarily the kind of article that Timmer is complaining about.  I just picked a handy example of a technical article that would probably not be too familiar to most readers.  Yep, I ... just happened ... to be reading up on singular homology theory over my winter break.  What can I say?

Field notes: Proudly answering none of your pressing questions

Browsing through old posts and statistics, one thing that jumped out is that a fair chunk of traffic comes from people seeking answers to questions I specifically don't answer.  Examples:

• Information age: Not dead yet was a reply to someone else's blog post, arguing that you couldn't really say when an "age" began or ended, and in any case "ages" tend to overlap rather than each new age supplanting the last.  If you came there searching for "When did the information age begin/end?", as several people have, you may be disappointed.
• Off topic: Welcome to the new decade was, well, first of all, off topic.  It's a pet peeve of mine that people will argue that everybody gets some fine point of usage wrong, in this case when decades start.  Somewhere I ran across someone's argument that a decade is just a block of ten years and you can start one whenever you want, and that clicked.  I really should have linked to wherever I read that, but then, I doubt it was their original idea either.  In any case, if you landed there looking to settle a bar bet about whether 2010 was the first year of a new decade, you may be disappointed.
• Now what happened to my bookmarks? Was further speculation on a previous post about why I no longer found myself using my browser's bookmark feature so much and never really ended up doing much with deli.cio.us (Remember them? They still seem to be around, actually, under plain delicious.com).  Naturally, pretty much anyone who ended up there was searching for help in recovering lost browser bookmarks.  I added some links for that up at the top a few years back, and even updated them a couple of times, but I have no idea if they're still useful.  As I said, I don't really use bookmarks myself anymore.
• "When I use a word, it means just what I choose it to mean" is the most popular Field Notes post of all time.  I'm not sure exactly what brought people to it -- the bulk of the activity seems to have scrolled off the easy-to-find summaries -- but it's just a rant on an annoying customer service involving spectacular overuse of boilerplate macros.  If you were thinking Alice in Wonderland, you would probably have given up a little past the title.

Two of these are genuine Field Notes posts -- analysis of things webby.  But those are the two that people seem to land on mainly in search of something else.  The other two, that is, the ones people might end up on more-or-less on purpose (depending on what was driving traffic to the last one), don't really convey what this blog is mostly about.  Of those two, the one that people probably land on looking for answers doesn't really answer the question.

And I'm not even disappointed.  To the contrary, there's something in all this that's deeply reflective of the web as I understand it.

My phone, then and now (or: Maybe coverage is the new coverage?)

(August? Really?  This must be a record gap even for the new, unhurried Field Notes.  Oh well ... it's been busy)

Re-reading through the blog, I ran across a short post on wireless carriers' advertising having shifted from coverage to bandwidth.  Most old posts seem to hold up well, but this one seemed remarkably out of date:

• With 4G building out, coverage is very much an issue -- at least in ads
• At the time I was using a "feature phone", mainly as an alarm clock and ... as a phone.  I finally took the plunge with a proper smart phone a while back.  I use it as ... 
• An alarm clock
• A phone
• A GPS
• A way to check email
• A camera (but I also used the old feature phone as that)
• A way to browse news stories, check sports scores, weather etc.
• A way to text -- it's noticeably easier to text with autocomplete, though not miraculously so
• A way to schedule appointments and to check my schedule and reminders
• A way to look up stuff quickly on the web
• A few other random applications

It's interesting that even feature phones had several of those -- phone, clock, text, camera, calendar -- and probably more or less the right set of them to get the most out of the limited bandwidth and CPU.  The newer iterations are generally better (auto backup of photos to the cloud comes to mind), but not mind-blowingly, night-and-day better.

My attitude at the time was "Meh ... I'm usually near my laptop and it has a bigger screen"  While I find I use my phone quite a bit during the day, I'm still near my laptop most of the time, and I'd only really miss the phone for really "mobile" applications, like

• phone
• GPS
• camera
• receiving texts anywhere
• checking email everywhere

Oh ... and as an alarm clock.

Margaret Hamilton: 1 New Horizons: 0

A bit more on Pluto, from a compugeek perspective if not a full-on web perspective ...

The New Horizons flyby was not completely without incident.  Shortly before the flyby itself, the craft went into "safe mode", contact was lost for a little over an hour and a small amount of scientific data was lost.  The underlying problem was "a hard-to-detect timing flaw in the spacecraft command sequence".  This quite likely means what's known in the biz as a "race condition", where two operations are going on at the same time, the software behaves incorrectly if the wrong one finishes first and the developers didn't realize it mattered.

Later investigation concluded that the problem happened when "The computer was tasked with receiving a large command load at the same time it was engaged in compressing previous science data."  This means that the CPU would have been both heavily loaded and multitasking, making it more likely that various "multithreading issues" such as race conditions would be exposed.

Now, before I go on, let me emphasize that bugs like this are notoriously easy to introduce by accident and notoriously hard to find if they do creep in, even though there are a number of well-known tools and techniques for finding them and keeping them out in the first place.

The incident does not in any way indicate that the developers involved can't code.  Far from it.  New Horizons made it through a ten-year, five billion kilometer journey, arriving within 72 seconds of the expected time, and was able to beam back spectacularly detailed images.  That speaks for itself.  It's particularly significant that the onboard computers were able to recover from the error condition instead of presenting the ground crew with an interplanetary Blue Screen of Death.  More on that in a bit.

Still ...

It's July 20, 1969.  The Apollo 11 lunar lander is three minutes from landing on the Moon when several alarms go off.  According to a later recounting by the leader of the team involved

Due to an error in the checklist manual, the rendezvous radar switch was placed in the wrong position. This caused it to send erroneous signals to the computer. The result was that the computer was being asked to perform all of its normal functions for landing while receiving an extra load of spurious data which used up 15% of its time.

This is a serious issue.  If the computer can't function, the landing has to be aborted.  However,

The computer (or rather the software in it) was smart enough to recognize that it was being asked to perform more tasks than it should be performing. It then sent out an alarm, which meant to the astronaut, I'm overloaded with more tasks than I should be doing at this time and I'm going to keep only the more important tasks; i.e., the ones needed for landing ... Actually, the computer was programmed to do more than recognize error conditions. A complete set of recovery programs was incorporated into the software. The software's action, in this case, was to eliminate lower priority tasks and re-establish the more important ones.

This is awesome.  Since "awesome" is generally taken to mean "kinda cool" these days, I'll reiterate: The proper response to engineering on this level is awe.  Let me try to explain why.

Depending on where you start counting, modern computing was a decade or two old at the time.  The onboard computer had "approximately 64Kbyte of memory and operated at 0.043MHz".  Today, you can buy a system literally a million times faster and with a million times more memory for a few hundred dollars.

While 64K is tiny by today's standards, it still leaves plenty of room for sophisticated code, which is exactly what was in there.  It does, however, mean that every byte and every machine cycle counts, and for that reason among others the code itself was written in assembler (hand-translated from a language called MAC and put on punch cards for loading).  Assembler is as low-level as it gets, short of putting in raw numbers, flipping switches or fiddling with the wiring by hand.

Here's a printout of that code if you're curious.  The dark bands are from printing out the listing on green-and-white-striped fanfold paper with a line printer such as used to be common at computer centers around the world.  The stripes were there to help the eye follow the 132-character lines.  Good times.  But I digress.

Just in case writing in assembler with an eye towards extremely tight code isn't enough, the software is asynchronous.   What does that mean?  There are two basic ways to structure a program such as this one that has to deal with input from a variety of sources simultaneously: the synchronous approach and the asynchronous approach.

Synchronous code essentially does one thing at a time.  If it's reading temperature and acceleration (or whatever), it will first read one input, say temperature from the temperature sensor, then read acceleration from the accelerometer (or whatever).  If it's asking some part of the engine to rotate 5 degrees, it sends the command to the engine part, then waits for confirmation that the part really did turn.  For example, it might read the position sensor for that part over and over until it reads five degrees different, or raise an alarm if doesn't get the right reading after a certain number of tries.

Code like this is easy to reason about and easy to read.  You can tell immediately that, say, it's an error if you try to move something and its position doesn't reach the desired value after a given number of tries.  However, it's no way to run a spaceship.  For example, suppose you need to be monitoring temperature continuously and raise a critical alarm if it gets outside its acceptable range.  You can't do that if you're busy reading the position sensor.

This is why high-performance, robust systems tend to be asynchronous.  In an asynchronous system, commands can be sent and data can arrive at any time.  There will generally be a number of event handlers, each for a given type of event.  The temperature event handler might record the temperature somewhere and then check to make sure it's in range.

If it's not, it will want to raise an alarm.  Suppose the alarm is a beep every five seconds.  In the asynchronous world, that means creating a timer to trigger events every five seconds, and creating an event handler that sends a beep command to the beeper when the timer fires (or, you can set a "one-shot" timer and have the handler create a new one-shot timer after it sends the beep command).

While all this is going on, other sensors will be triggering events.  In between "the temperature sensor just reported X" and "the timer for your beeper just went off", the system might get events like "the accelerometer just reported Y" and "the position sensor for such-and-such-part just read Z".

To move an engine part in this setup, you need to send it a command to move, and also create a handler for the position sensor's event.  That handler has to include a counter to remember how many position readings have come in since the command to move, along with the position the part is supposed to get to (or better, a time limit and the expected position).

A system like this is very flexible and doesn't spend time "blocked" waiting for things to happen, but it's also harder to read and reason about, since things can happen in any order and the logic is spread across a number of handlers, which can come and go depending on what the system is doing.

And then, on top of all this, the system has code to detect and recover from error conditions, not just in the ship it's controlling but in its own operation.  Do-it-yourself brain surgery, in other words.


I report my occupation as "software engineer" for tax purposes and such, but that's on a good day.  Most of us spend most of our time coding, that is, writing detailed instructions for machines to carry out.  True software engineering means designing a robust and efficient system to solve a practical problem.  The term was coined by Margaret Hamilton, the architect of the Apollo 11 control systems quoted above and a pioneer in the design of asynchronous systems.  As the story of the lunar landing demonstrates, she and her team set a high bar for later work.

New Horizons ran into essentially the same sort of problem that Apollo 11 did, but handled it less robustly (going to "safe mode" and then recovering, as opposed to automatically re-prioritizing), all building on techniques that Hamilton and her team helped develop, and using vastly more powerful equipment and development tools based on decades of collective experience.  So, with all due respect to the New Horizons team, I'd have to say Apollo 11 wins that one.

Latency, bandwidth and Pluto

As you may well know, the New Horizons spacecraft flew by Pluto last month, dramatically increasing our knowledge of Pluto and its moons (let's not even get into whether Pluto and Charon jointly constitute a "binary dwarf planet" or whatever).  There are even a few pictures on the web.

But wait ... that's not very many pictures for a ten-year mission.  Even worse, if you were watching at the time you'll know that New Horizons went completely dark for most of a day, right when it was flying by Pluto.  Isn't this the modern web, where everything is available everywhere instantly?  What gives, NASA?

Part of the problem is the way New Horizons is designed.  It's expensive to accelerate mass to the speed New Horizons is going, and since you can't exactly send a repair crew out to Pluto, it's good to have as few moving parts as possible.  As a result, the ship has a small battery and both the antenna and the cameras are mounted firmly in place.  If you want to turn the antenna toward Earth, you have to move the whole ship, using some of the small store of remaining fuel dedicated to course corrections and attitude control.  If you want to point the cameras toward Pluto, you have to turn the ship that way.  You can't do both.

That explains why the ship went dark for the duration of the flyby, but actually it effectively went dark for considerably longer than that.  It takes about 4.5 hours for a signal to travel the distance between Earth and Pluto.  That means the sequence of events is, more or less

• t + 0: Flight control sends commands to New Horizons to point the cameras at Pluto, take pictures, orient the antenna toward Earth, and report back.
• t + 4.5 hours: New Horizons gets the command and starts re-orienting and taking pictures
• t + 9 hours: Last time at which any signal from before the pointing operation will reach Earth
• t + 25.5 hours (more or less): New horizons, now with the antenna pointed back toward earth, sends "Phone home" message reporting status.
• t + 30 hours (more or less): "Phone home" message arrives

The important thing to note here is that, while the ship is actually out of contact for 21 hours, it's 25.5 hours from the time the command is sent to the time the ship is reachable again, and 30 hours before the ground crew knows it's reachable again.   If the phone home signal hadn't arrived, it would be 9 more hours, at a minimum, before they knew if any corrective action they'd taken had worked.  By internet standards this is ridiculously high latency, but anyone who's played a laggy video game or been on a conference call with people on the opposite side of the world has experienced the same problem on a smaller scale.

So part of the reason pictures have been slow in coming is latency.  It's going to take a minimum of 4.5 hours to beam anything back, and longer if the ground crew has to send instructions.

The other problem is bandwidth.  Pluto is about 5 billion kilometers (about 3 billion miles) away.  Signal strength drops off as the square of the distance, so, for example, a signal from Pluto is about 160 times weaker than a signal from Mars and, since the power source has to be small (around 15 watts), the signal is not going to be extremely powerful to start with.  Lower power means a lower signal to noise ratio and less bandwidth (or at least that's my dim software-engineer understanding of it -- in real life "bandwidth" doesn't exactly mean "how many bits you can transmit", and I'm sure there's lots more I'm glossing over).

Put all that together and on a good day we have about 2Kbps coming back from Pluto.  That's about what you could get out of a modem in the mid 1980s.   Internet technology has progressed just a bit since then, but internet technology doesn't have to cope with vast distances and stringent mass limitations.  At 2Kbps, one raw image from LORRI (the hi-res black-and-white camera) takes close to two hours to transmit.  This is why, if all goes well, we'll be getting Pluto pictures (and other data) well into 2016.

I'd still say that New Horizons is "on the web" in some meaningful sense, but the high latency and low bandwidth make it a great example of Deutsch's fallacies in action.

[Update: Not only is New Horizons sending back data slowly, it's not sending back particularly pretty data at the moment.  From their main page:

Why hasn’t this website included any new images from New Horizons since July? As planned, New Horizons itself is on a bit of a post-flyby break, currently sending back lower data-rate information collected by the energetic particle, solar wind and space dust instruments. It will resume sending flyby images and other data in early September.

-- D.H. 24 Aug 2015]

A decoy within a decoy ...

It's not news that more-or-less legitimate news sites like to surround their articles with enticing links to somewhat-less-legitimate news sites: 20 reasons your dog may be considering a new career ... You'll never believe this simple secret for removing egg yolk from pine trees ... Read this before you buy a moose ... you know, those sorts of things.

Click through on one of those and there's a good chance there will be some sort of slide show surrounded by a minefield of garish ads.  Sure, why not?  Putting together pictures of whatever sort of fluff for people to page through is somewhere around neutral on my personal scale of good or bad ways to make a living.

Except, when you lay out the page with a large ad at the top with an arrow icon next to it, and white space between, looking for all the world like it's going to take you to the article you're really interested in (well, at least maybe kinda interested in).  Except that arrow icon takes you to the advertiser's page.  That tilts ever so slightly toward "bad" in my book.  For the reader, it's a straight-up bait and switch.  For the advertiser, it's a steady stream of annoyed readers and, if it's a pay-per-click model, they get to pay for the privilege.  True, there's not supposed to be any such thing as bad publicity, but still.