Spam spam spam

A while ago I turned off CAPTCHA for comments on this blog, so you wouldn't need to read a nearly-unreadable blob of gibberish in order to post a comment.  "Looks more professional," they said.  "Spam filters work fine," they said ...

It did work fine for a few months, but lately there has been a rash of spam comments and the filters haven't caught up yet.  I considered my options:

• Spammers gonna spam.  Just let it be and hand-delete the comments every so often.  That seemed OK when it was one or two a week, but ignoring the problem hasn't seemed to make it go away.
• Limit comments to people with Google accounts.  A large portion of the spam content, though not all, was anonymous.  Requiring a Google login gets rid of that, at the cost of making anyone who really wants to post anonymously here put together a fake account.  Like, um, a spammer would.
• Turn CAPTCHA back on and go back to the inconvenience that entails.
• Turn on moderation and review comments by hand.  Comments would not show up until I happened to check through them, which would probably be somewhat more often than I actually post.

For now, I'm trying the second option.  If you want to post a comment, you'll have to be logged in to your Google account, but that doesn't seem like a showstopper.  Almost all of the legitimate comments here are from logged-in users.  Again, if you feel the need to post anonymously, I'm sure you'll know what to do.

If that doesn't at least keep it down to a dull roar, I'll probably go with moderated comments.  Re-instating CAPTCHA would be a last resort.

More Silly UX

Searching for "Humphrys Entwistle interview" (OK, full disclosure.  I actually searched for "Humphreys ...") to see one of the BBC's own interviewers "shred" its director general on air, I ran across this page from the Grauniad.  It's a fine example of why web pages should not try to re-invent features, in this case displaying text, that are already widely implemented.

The text transcript of the interview, set in a typewriter font presumably so you can pretend that the Grauniad's staff of crack transcribers just typed it up fresh for you, was a little small for me to read comfortably.  Not to worry, though.  There's a zoom slider.

Slide the slider and the whole document gets bigger within the fixed-sized window on the page.  On the plus side, the text is now easily legible.  On the not-so-much side, half of it is now hidden.  Not to worry, though.  There's a horizontal scroll bar so you can scoot back and forth for every single line in order to read what's there.

Sorry, no, thanks.  Life is too short for horizontal scroll bars on text.  You'd think an organization with its roots in printing text in narrow, readable columns would get that.  I can't think of any situations where, if the idea is just to get the content of the text across, the right answer is anything other than formatting it into whatever column is available.  If the reader wants larger text, then make the words bigger and wrap the text -- like you would anyway -- with fewer words per line.

Or you could spend a fair bit of effort implementing a clever-looking but near-useless solution to a simple non-problem.

Getting Smart with email

It appears that two participants in a prominent scandal -- if you're reading this now, you know which one, and if you're reading this later, it won't really matter -- tried to cover their email tracks by not actually sending email at all.  Instead, they shared an email account and would write messages but save them to the Drafts folder for the other to read.

I'm a bit unclear on how this helps significantly, particularly since it doesn't seem to have worked all that well in this case.

The act of sending email itself is reasonably secure.  If you and your recipient are both using one of the major providers (the same provider, that is), then sending email just means copying some bits, if that.  Nothing need go out over the public internet.  Likewise, reading that email just means logging in and viewing it.  You are using HTTPS, aren't you? Probably, even if you don't know it, but it's worth checking your email settings just in case.

If you're up to no good and storing email on an unencrypted local drive, you deserve to lose.

So it really comes down to how many passwords you would have to crack to get at the messages.  Consider two scenarios:

1. Alice and Bob have separate accounts with MyEmail.com, which supports two-factor authentication.  That means that it's not enough just to know the password.  When you log in, you give not just the password, but a magic number from a text message sent to your phone, or from some other kind of device that produces single-use magic numbers.
2. Alice and Bob share a TOP SEEKRIT "drop box" account with just a password.

In scenario 2, if I can crack that one password, I can see the whole correspondence, so long as I think to check the Drafts folder.  Alice and Bob basically have a password plus a bit of security through obscurity, otherwise known as "no additional security".

In scenario 1, I have two passwords to try to guess, which means two chances at success instead of one.  So far, so good. I crack one of the passwords and log in.  The login screen then says "enter the magic number we just sent to your phone".  Oops.  Not only do I not have the magic number to log in with, Alice (or Bob, as the case may be) now knows that someone is trying to log in.

I suppose it would be possible for Alice could set her phone to forward magic number messages to Bob (or vice versa, but not both!) and use two-factor authentication that way, assuming no one will ask why Bob is getting strange texts with random numbers in them for no apparent reason.  I'd then have to crack the shared password and steal a phone, more or less what I'd have to do in scenario 1, except instead of having a choice of passwords to crack, I have a choice of phones to steal.

Note that some two-factor authentication schemes use a cryptocard or something similar as a second factor.  That would make sharing the account physically impossible, unless Alice and Bob are in the same room, in which case the Cone of Silence is probably the better option.

All bets are off if The Man is able to force MyEmail.com to give up access to the account, but that applies equally well in either scenario.

How to tell the web has really, truly woven itself into our lives

The cashier at the pizza place today had tattooed on her wrists:

<love>
</hate>

It hardly seems fair to point out that that's not valid XML (or HTML).

More stuff to be uneasy about

Previously I mentioned a couple of attempts to mine the Twitterverse for signs of what's going on in the world.  It's also perfectly possible to mine the conventional media.  One such effort is globalincedentreport.com, a "global display of terrorism and other suspicious events". [The link is now broken. I'm not going to chase down when the site shut down, but I don't remember ever running across a reference to it after stumbling on it for this post -- D.H. July 2025]

Global Incident Report gathers together news stories and plots their locations on a global map using eye-catchingly garish icons.  There are several categories of incident, such as disease outbreaks, gang-related activity, drug interdictions and terrorist threats (you'll need to establish an account to view those and a couple of other categories, which seems reasonable).  Basically everything that makes your local TV news the angst-fest it is, right there in one handy web site.

Comparing this to Twitter, I'd say the geo-tagging is more accurate (though the site seems to think Miami, OK is on the Kansas-Missouri state line) and of course the information is more reliable.  Sorry, Web 2.0 fans, but I've been up and down this one.  Mainstream media have their own problems, but I'll take "rush-job vetting and curation aimed at selling ads" over no vetting and curation at all.

There's a clear sampling bias.  Since the reports are all in English, incidents tend to cluster in the US, UK and India.  Canada, Australia, New Zealand and other parts of the Commonwealth are sparser, but they're also generally less populated.  Beyond the bias toward English, it's not clear which feeds the site samples, which it self will be a subset of which feeds are made available.  So, definitely, caveat lector.

Even with the caveats, it's an interesting effort.  The mapped incidents link directly to primary sources -- transparency is good -- and the global view lets you get the big picture of what's going on.  Or at least what's being reported.  And tracked by the site ... anyway, still interesting.

More ESRI and Twitter

A while ago I posted about a map from esri.com that incorporated social media such as Twitter with more traditional sources of fire information.  My conclusion was that the social data was not that helpful.

ESRI has just put out another set of maps, this time on acceptance same-sex marriage.  The first map uses a proprietary demographic model to try to rate how likely a given county is to favor same-sex marriage.  As far as I can tell, this model isn't particularly based on, say, polling data or election results from ballot measures, but more on factors like how urban or rural the county is, how many people have gone to college and so forth.

The second map shows state laws.  It corresponds roughly with the first map.  With the interesting exception of Iowa, the first map shows significant support where same-sex marriage is legal, as one would expect.

The third map is based on Twitter data.  It says there is strong support for same-sex marriage across the country, opposition in five states (including Minnesota, despite significant support in the populous Twin Cities), and very strong opposition in exactly one state: North Carolina.  Idaho, Wyoming and Vermont had insufficient data.

The support map shows North Carolina as fairly similar to neighboring Virginia, which the Twitter data shows as strongly supportive, and as more supportive than neighboring South Carolina and Tennessee. The Twitter data show both of those states as moderately supportive.

Clearly something is out of line here.  Two possible explanations:

• Concerning the overall map, the Twitterverse is not a representative sample.  Overall, Twitter traffic is much more supportive of same-sex marriage than the country as a whole.  This probably shouldn't come as a surprise.
• Concerning North Carolina, the Twitter data covers May 9 through June 30, 2012.  As the map explanation notes, North Carolina had voted on May 8 against a proposition supporting same-sex marriage, by a roughly 60-40 margin after a very intense campaign.  It would be interesting to know what portion of the Twitter traffic surveyed is from the immediate aftermath of that election.  My guess would be a large portion.

Along with the fire map I mentioned above and a study on Twitter rumors after the London riots, which was presented in the Guardian as confirming the notion that Twitter is a good, self-correcting source of information but in fact shows anything but, this is the third piece of evidence I've run across suggesting that you should treat any inferences drawn from Twitter traffic with a grain of salt.

Twitter may well be a great way to find out what people, at least those with Twitter accounts, are paying attention to at the moment, but it's risky, to say the least, to draw conclusions about objective facts from that.

Curiosity online

The Curiosity rover, formally the Mars Science Laboratory, landed on Mars late on August 5th (or early on the 6th, depending on your time zone).

This was a major accomplishment.  Mars has a habit of eating space probes.  The majority have failed, sometimes quietly and sometimes spectacularly, at least once due to the kind of basic coding error that would send a regular geek cursing back to the keyboard.  Except when it comes to interplanetary travel there's generally no "next release".

The Curiosity landing was more complex than any Mars landing before it, and parts of it had never been realistically tested, much less the overall sequence.  This was not out of negligence or cut corners but a simple necessity.  Mars's atmosphere is much, much thinner than Earth's.  At its thickest it's equivalent to Earth's atmosphere at about 35 kilometers (about 20 miles, or about four times as high as the summit of Mount Everest).  Mars's gravity is about 40% of Earth's.  There's simply no practical way to re-create those conditions at the necessary scale anywhere near Earth.

This thin atmosphere is a real problem.  There's just enough of it you can't ignore it, but not enough for a parachute to take the lander all the way to the surface safely.  To land Curiosity, which is considerably larger and more massive than the Spirit and Opportunity rovers before it, NASA put together a landing sequence that Rube Goldberg might have appreciated:

• The probe slams into the atmosphere, protected by a heat shield and pulling upwards of 10g of deceleration, possibly as high as 15g (This is from NASA's press kit from before the landing.  I don't know what the actual numbers were, but clearly it's not going to work with humans aboard.)
• Once the the probe has slowed enough not to need the heat shield, the heat shield is jettisoned.
• The probe then deploys a parachute, which slows it to about 300 km/h (about 180 mph).
• The parachute is then jettisoned and a rocket-powered descent stage takes over, carefully avoiding the parachute and the back shell it's attached to.  If the rockets don't work, the probe will hit the surface, hard, in about 20 more seconds.
• The descent stage lowers the rover on nylon cords while it slowly descends, since landing completely under rocket power risks kicking up enough dust to damage the rover.
• While the descent stage is doing this, the rover gets its landing gear into place.
• When the rover is safely on the ground, the cords are cut loose, using explosives, and the descent stage flies off to crash land some distance away.

What could possibly go wrong?  Mind, this is a simplified description (and any inaccuracies are mine).  The full details include several more maneuvers, through six different configurations in all, with 76 pyrotechnic devices, ballast jettisoned at various points and dozens of people sweating bullets in the control room and elsewhere.

"Seven minutes of terror", they called it.  Damn impressive engineering, I call it.

OK, so now that we've paused to admire NASA/JPL's chops, what does this have to do with the web? 

I'm old enough to sorta kinda remember Apollo and grainy black-and-white TV coverage.  And Tang.  Cool, and, it must be said, more culturally significant than today's missions, but technically not even close to what we have today.  This is true not only of the vehicle itself, but of the communications technology.  In the 70s we had grainy black-and-white video.  For better imagery you had to wait for the astronauts to bring back the film.

Now, using essentially the same technology as a cell phone camera, NASA is able to capture digital images and put them up on its web site for the world to see immediately (as many have pointed out, much more quickly than NBC saw fit to broadcast Olympic events).  The web gallery includes not only the pretty press-release pictures but also the raw frames they were made from, including pictures of Curiosity staring at its feet and other such that didn't make the cut.  There are plenty of other goodies as well.  I particularly like the "white-balanced" images, which have been post-processed to show what the same terrain would look like under sunlight on Earth.

And there are, of course, a Twitter hashtag (#MSL) and (obligatory plug) updates on NASA's Google Plus page.

As inspiring as it may have been to watch thrilling news coverage with Walter Cronkite narrating, there's something much more intimate about being able to visit a web site from time to time and watch the story unfold directly from the source.

Dear Reader,

When I was in school (a while ago, admittedly), we learned how to write a letter:

Dear So and So, 

Blah, blah, blah ... 

Sincerely,
David Hull

(or, with a close friend or family member, "Love, David" or "Yours, David", or such)

In email these days, if there's any salutation or closing at all, it's something like:

Hi So and So, 

Blah, blah, blah ... 

Thanks!

Some of this, I'm sure, is because times change.  The canonical letter of my youth is miles and miles less formal than one from, say, the 1700s.  Nonetheless, I think the switch from snail mail to email has had an influence.

For one thing, it feels a little funny to add a salutation and closing at all, when the email header includes the names of the sender and recipient.  On the other hand, it can feel a little funny not to have at least something.  So, I think, we tend to revert to what we'd say in a conversation, with both participants standing right there and knowing who the other is.

Beyond the switch to a new medium giving an opportunity for new forms, the medium itself influences the forms.  This has been true with letters as well.  Formal letters with half a page of salutations and closings make more sense when hand-writing and sending a letter is difficult and therefore reserved for occasions like petitioning the royal court, where you are quite possibly addressing an official you only know by name and trying to establish your reputation by means of any titles you may have.

Y'r Humble & Ob'd't S'v't
Etc., etc.

Answering my random question

I recently asked whether there were more than a Britannica worth of Britannica-quality articles in Wikipedia.  Looking into it a bit, I'd have to generally agree with Earl that no, there aren't.

Britannica has about half a million articles (according to Wikipedia's page on Britannica).  English Wikipedia has about four million.  I would not say that one in eight Wikipedia articles is up to Britannica standards.

Granted, the famous Nature study of 2005 found that Wikipedia science articles are nearly as accurate as Britannica articles -- and that Britannica is far from flawless.  One can dispute the methodology and conclusions of that study, and Britannica did, but the overall conclusion seems at least plausible.

However, apart from science articles only being part of the picture, the writing in Wikipedia is uneven and full of Wikipedia tics.  Britannica, with full-time writers and editors, ought to be a bit better.  I tend to think this is where Wikipedia generally falls short. Factually, the two are comparable.  In style and organization, not so much.

Taking content and writing together, there are probably relatively few Britannica-quality articles in Wikipedia, but there are more than enough that are close enough.

Now CAPTCHA-free

A while ago I turned on "Word Verification", which makes people leaving comments read a hard-to-read word in order to prove they're not a bot.

This seems to have done more harm than good.  I still get the occasional spam comment, and it's a pain for people to leave real comments.  To see what to do about it, I Googled blogger captcha, and up popped this post urging bloggers to "kick Captcha to the curb".  The gist is, no, that extra inconvenience to real readers isn't really worth it.  Spam filters catch spam even if word verification is turned off.

And, of course, "It flags your blog as less professional".  If there's anything this blog stands for, in tone, subject matter and publication schedule, it's iron-clad professionalism.

So I'm turning word verification off.  If it turns out to be a horrible mistake, I can always turn it back on.  Otherwise, no news is good news.

More rumblings in the world of academic publishing

I've written before about the use of online outlets for quick publication of informal (that is, non-peer-reviewed) results, and arXiv in particular.  In The Case for Books, Robert Darnton expresses concern about the state of academic publishing and the power that the major publishers hold over academic researchers and libraries and wonders what will come of it all.

Now it seems things are heating up.  There is a boycott in progress against Elsevier, the academic publishing juggernaut that owns such publications as Lancet.  A number, and evidently a growing number, of academics are simply refusing to publish in or otherwise participate in Elsevier publications, on the grounds that Elsevier's high prices and profit margins and their overall practices are harmful to those who must publish in them, the institutions who must buy the publications, and to the free exchange of ideas itself.

At this writing, 12,558 people have signed up, giving their full names and affiliations in a searchable list.  These are not random people taking potshots from behind pseudonyms.  These people are putting their reputations on the line publicly and, by walking away from one of the major sources of recognition and exposure, potentially hindering their academic careers.  Their names may be found on thecostofknowledge.com.


The basic issue here is that to have a career in academia, one must produce a steady stream of work.  The universal standard for measuring that stream of work is the number and quality of papers one publishes.  "Publish or perish."

Since anyone at all can print up a paper on a topic of research (and many do), there has to be some mechanism to determine whether a result has any real merit.  In the academic world, that mechanism is peer review.  If you submit a paper to a refereed journal, the editors will select a set of reviewers in your field to go over it.  The reviewers will either reject the article outright or accept it, likely with revisions.

Different journals have different standards for inclusion.  This allows readers to have some idea up front how worthwhile an article is, and provides some means of rating a researcher's output beyond the sheer number of articles published.  In principle, and for the most part in practice, the peer review process ensures that articles in journals are accurate and relevant, at least as far as the reviewers can tell at the time.  Essentially, journals provide brand names.

Peer review is clearly a valuable service, beyond printing and distribution of paper volumes, which is, of course, on the wane.  But there are problems.  In the call to action which started the current boycott,  Timothy Gowers puts forth several complaints:

• Journals cost too much, particularly since the authors and reviewers are paid by their institutions, not the publisher, and it's largely the same institutions that pay for subscriptions to the journals they're paying to produce.
• Online access is behind expensive paywalls.
• Publishers drive the overall cost up by bundling, that is, requiring institutions to buy large numbers of journals, many of which literally go unread, in order to subscribe to the ones they really care about.  An institutional bundle from a given publisher can run into the millions of dollars per year.
• While many publishers produce expensive journals and require bundling, Gowers calls out Elsevier in particular for several reasons, including supporting legislation that restricts access to published results and playing hardball with institutions that try to resist bundling.

In short, publishers are in serious danger of losing their relevance, and in the view of those joining the boycott, Elsevier is one of the worst offenders.

It's all well and good to object to publishers' behavior and organize a boycott, but the academic world also seems actively engaged in building a more open, web-enabled alternative.  This includes

• Blogging as a means of informal sharing and discussion.  Indeed, Gowers' call to action appeared on his blog (which, with a mathematician's precision, he calls "Gowers's Weblog")
• Sites, notably arXiv, for collecting unrefereed preprints.
• New online refereed journals aiming to take the place of old ones.  Normally establishing a brand can be difficult, but if the editorial board of the new journal is made up of disaffected board members from old journals, their reputations come with them.

While writing this, I was wondering what would be a really webby way to do this.  Here's a sketch:

• Articles would be published in something more like a wiki form, with a full revision history and editors making changes directly.
• Since reputation is particularly important here, changes would ideally be digitally signed.
• Individuals could put their imprimatur on (a particular revision of) an article they thought worthy.
• The quality of papers could be judged by the reputation of those approving of them, which in turn would be judged by the quality of the papers they'd produced ...

And then it occurred to me that in practice there would probably come to be groups of people whose approval was particularly significant within particular fields.  It would be good to be able to establish groups of, say experts in homology or complex analysis.  It would also be good to have people who were good at steering new works to the appropriate groups of experts.

Hmm ... except for the revision history and digital signatures bit, this sounds an awful lot like a peer-reviewed online journal.