Monday, December 10, 2012

Spam spam spam

A while ago I turned off CAPTCHA for comments on this blog, so you wouldn't need to read a nearly-unreadable blob of gibberish in order to post a comment.  "Looks more professional," they said.  "Spam filters work fine," they said ...

It did work fine for a few months, but lately there has been a rash of spam comments and the filters haven't caught up yet.  I considered my options:

  • Spammers gonna spam.  Just let it be and hand-delete the comments every so often.  That seemed OK when it was one or two a week, but ignoring the problem hasn't seemed to make it go away.
  • Limit comments to people with Google accounts.  A large portion of the spam content, though not all, was anonymous.  Requiring a Google login gets rid of that, at the cost of making anyone who really wants to post anonymously here put together a fake account.  Like, um, a spammer would.
  • Turn CAPTCHA back on and go back to the inconvenience that entails.
  • Turn on moderation and review comments by hand.  Comments would not show up until I happened to check through them, which would probably be somewhat more often than I actually post.
For now, I'm trying the second option.  If you want to post a comment, you'll have to be logged in to your Google account, but that doesn't seem like a showstopper.  Almost all of the legitimate comments here are from logged-in users.  Again, if you feel the need to post anonymously, I'm sure you'll know what to do.

If that doesn't at least keep it down to a dull roar, I'll probably go with moderated comments.  Re-instating CAPTCHA would be a last resort.

Monday, November 26, 2012

More Silly UX

Searching for "Humphrys Entwistle interview" (OK, full disclosure.  I actually searched for "Humphreys ...") to see one of the BBC's own interviewers "shred" its director general on air, I ran across this page from the Grauniad.  It's a fine example of why web pages should not try to re-invent features, in this case displaying text, that are already widely implemented.

The text transcript of the interview, set in a typewriter font presumably so you can pretend that the Grauniad's staff of crack transcribers just typed it up fresh for you, was a little small for me to read comfortably.  Not to worry, though.  There's a zoom slider.

Slide the slider and the whole document gets bigger within the fixed-sized window on the page.  On the plus side, the text is now easily legible.  On the not-so-much side, half of it is now hidden.  Not to worry, though.  There's a horizontal scroll bar so you can scoot back and forth for every single line in order to read what's there.

Sorry, no, thanks.  Life is too short for horizontal scroll bars on text.  You'd think an organization with its roots in printing text in narrow, readable columns would get that.  I can't think of any situations where, if the idea is just to get the content of the text across, the right answer is anything other than formatting it into whatever column is available.  If the reader wants larger text, then make the words bigger and wrap the text -- like you would anyway -- with fewer words per line.

Or you could spend a fair bit of effort implementing a clever-looking but near-useless solution to a simple non-problem.

Wednesday, November 14, 2012

Getting Smart with email

It appears that two participants in a prominent scandal -- if you're reading this now, you know which one, and if you're reading this later, it won't really matter -- tried to cover their email tracks by not actually sending email at all.  Instead, they shared an email account and would write messages but save them to the Drafts folder for the other to read.

I'm a bit unclear on how this helps significantly, particularly since it doesn't seem to have worked all that well in this case.

The act of sending email itself is reasonably secure.  If you and your recipient are both using one of the major providers (the same provider, that is), then sending email just means copying some bits, if that.  Nothing need go out over the public internet.  Likewise, reading that email just means logging in and viewing it.  You are using HTTPS, aren't you? Probably, even if you don't know it, but it's worth checking your email settings just in case.

If you're up to no good and storing email on an unencrypted local drive, you deserve to lose.

So it really comes down to how many passwords you would have to crack to get at the messages.  Consider two scenarios:

  1. Alice and Bob have separate accounts with, which supports two-factor authentication.  That means that it's not enough just to know the password.  When you log in, you give not just the password, but a magic number from a text message sent to your phone, or from some other kind of device that produces single-use magic numbers.
  2. Alice and Bob share a TOP SEEKRIT "drop box" account with just a password.
In scenario 2, if I can crack that one password, I can see the whole correspondence, so long as I think to check the Drafts folder.  Alice and Bob basically have a password plus a bit of security through obscurity, otherwise known as "no additional security".

In scenario 1, I have two passwords to try to guess, which means two chances at success instead of one.  So far, so good. I crack one of the passwords and log in.  The login screen then says "enter the magic number we just sent to your phone".  Oops.  Not only do I not have the magic number to log in with, Alice (or Bob, as the case may be) now knows that someone is trying to log in.

I suppose it would be possible for Alice could set her phone to forward magic number messages to Bob (or vice versa, but not both!) and use two-factor authentication that way, assuming no one will ask why Bob is getting strange texts with random numbers in them for no apparent reason.  I'd then have to crack the shared password and steal a phone, more or less what I'd have to do in scenario 1, except instead of having a choice of passwords to crack, I have a choice of phones to steal.

Note that some two-factor authentication schemes use a cryptocard or something similar as a second factor.  That would make sharing the account physically impossible, unless Alice and Bob are in the same room, in which case the Cone of Silence is probably the better option.

All bets are off if The Man is able to force to give up access to the account, but that applies equally well in either scenario.

Tuesday, November 13, 2012

How to tell the web has really, truly woven itself into our lives

The cashier at the pizza place today had tattooed on her wrists:


It hardly seems fair to point out that that's not valid XML (or HTML).

Wednesday, October 17, 2012

More stuff to be uneasy about

Previously I mentioned a couple of attempts to mine the Twitterverse for signs of what's going on in the world.  It's also perfectly possible to mine the conventional media.  One such effort is, a "global display of terrorism and other suspicious events".

Global Incident Report gathers together news stories and plots their locations on a global map using eye-catchingly garish icons.  There are several categories of incident, such as disease outbreaks, gang-related activity, drug interdictions and terrorist threats (you'll need to establish an account to view those and a couple of other categories, which seems reasonable).  Basically everything that makes your local TV news the angst-fest it is, right there in one handy web site.

Comparing this to Twitter, I'd say the geo-tagging is more accurate (though the site seems to think Miami, OK is on the Kansas-Missouri state line) and of course the information is more reliable.  Sorry, Web 2.0 fans, but I've been up and down this one.  Mainstream media have their own problems, but I'll take "rush-job vetting and curation aimed at selling ads" over no vetting and curation at all.

There's a clear sampling bias.  Since the reports are all in English, incidents tend to cluster in the US, UK and India.  Canada, Australia, New Zealand and other parts of the Commonwealth are sparser, but they're also generally less populated.  Beyond the bias toward English, it's not clear which feeds the site samples, which it self will be a subset of which feeds are made available.  So, definitely, caveat lector.

Even with the caveats, it's an interesting effort.  The mapped incidents link directly to primary sources -- transparency is good -- and the global view lets you get the big picture of what's going on.  Or at least what's being reported.  And tracked by the site ... anyway, still interesting.

Friday, September 21, 2012

More ESRI and Twitter

A while ago I posted about a map from that incorporated social media such as Twitter with more traditional sources of fire information.  My conclusion was that the social data was not that helpful.

ESRI has just put out another set of maps, this time on acceptance same-sex marriage.  The first map uses a proprietary demographic model to try to rate how likely a given county is to favor same-sex marriage.  As far as I can tell, this model isn't particularly based on, say, polling data or election results from ballot measures, but more on factors like how urban or rural the county is, how many people have gone to college and so forth.

The second map shows state laws.  It corresponds roughly with the first map.  With the interesting exception of Iowa, the first map shows significant support where same-sex marriage is legal, as one would expect.

The third map is based on Twitter data.  It says there is strong support for same-sex marriage across the country, opposition in five states (including Minnesota, despite significant support in the populous Twin Cities), and very strong opposition in exactly one state: North Carolina.  Idaho, Wyoming and Vermont had insufficient data.

The support map shows North Carolina as fairly similar to neighboring Virginia, which the Twitter data shows as strongly supportive, and as more supportive than neighboring South Carolina and Tennessee. The Twitter data show both of those states as moderately supportive.

Clearly something is out of line here.  Two possible explanations:
  • Concerning the overall map, the Twitterverse is not a representative sample.  Overall, Twitter traffic is much more supportive of same-sex marriage than the country as a whole.  This probably shouldn't come as a surprise.
  • Concerning North Carolina, the Twitter data covers May 9 through June 30, 2012.  As the map explanation notes, North Carolina had voted on May 8 against a proposition supporting same-sex marriage, by a roughly 60-40 margin after a very intense campaign.  It would be interesting to know what portion of the Twitter traffic surveyed is from the immediate aftermath of that election.  My guess would be a large portion.
Along with the fire map I mentioned above and a study on Twitter rumors after the London riots, which was presented in the Guardian as confirming the notion that Twitter is a good, self-correcting source of information but in fact shows anything but, this is the third piece of evidence I've run across suggesting that you should treat any inferences drawn from Twitter traffic with a grain of salt.

Twitter may well be a great way to find out what people, at least those with Twitter accounts, are paying attention to at the moment, but it's risky, to say the least, to draw conclusions about objective facts from that.

Friday, September 7, 2012

Curiosity online

The Curiosity rover, formally the Mars Science Laboratory, landed on Mars late on August 5th (or early on the 6th, depending on your time zone).

This was a major accomplishment.  Mars has a habit of eating space probes.  The majority have failed, sometimes quietly and sometimes spectacularly, at least once due to the kind of basic coding error that would send a regular geek cursing back to the keyboard.  Except when it comes to interplanetary travel there's generally no "next release".

The Curiosity landing was more complex than any Mars landing before it, and parts of it had never been realistically tested, much less the overall sequence.  This was not out of negligence or cut corners but a simple necessity.  Mars's atmosphere is much, much thinner than Earth's.  At its thickest it's equivalent to Earth's atmosphere at about 35 kilometers (about 20 miles, or about four times as high as the summit of Mount Everest).  Mars's gravity is about 40% of Earth's.  There's simply no practical way to re-create those conditions at the necessary scale anywhere near Earth.

This thin atmosphere is a real problem.  There's just enough of it you can't ignore it, but not enough for a parachute to take the lander all the way to the surface safely.  To land Curiosity, which is considerably larger and more massive than the Spirit and Opportunity rovers before it, NASA put together a landing sequence that Rube Goldberg might have appreciated:
  • The probe slams into the atmosphere, protected by a heat shield and pulling upwards of 10g of deceleration, possibly as high as 15g (This is from NASA's press kit from before the landing.  I don't know what the actual numbers were, but clearly it's not going to work with humans aboard.)
  • Once the the probe has slowed enough not to need the heat shield, the heat shield is jettisoned.
  • The probe then deploys a parachute, which slows it to about 300 km/h (about 180 mph).
  • The parachute is then jettisoned and a rocket-powered descent stage takes over, carefully avoiding the parachute and the back shell it's attached to.  If the rockets don't work, the probe will hit the surface, hard, in about 20 more seconds.
  • The descent stage lowers the rover on nylon cords while it slowly descends, since landing completely under rocket power risks kicking up enough dust to damage the rover.
  • While the descent stage is doing this, the rover gets its landing gear into place.
  • When the rover is safely on the ground, the cords are cut loose, using explosives, and the descent stage flies off to crash land some distance away.
What could possibly go wrong?  Mind, this is a simplified description (and any inaccuracies are mine).  The full details include several more maneuvers, through six different configurations in all, with 76 pyrotechnic devices, ballast jettisoned at various points and dozens of people sweating bullets in the control room and elsewhere.

"Seven minutes of terror", they called it.  Damn impressive engineering, I call it.

OK, so now that we've paused to admire NASA/JPL's chops, what does this have to do with the web? 

I'm old enough to sorta kinda remember Apollo and grainy black-and-white TV coverage.  And Tang.  Cool, and, it must be said, more culturally significant than today's missions, but technically not even close to what we have today.  This is true not only of the vehicle itself, but of the communications technology.  In the 70s we had grainy black-and-white video.  For better imagery you had to wait for the astronauts to bring back the film.

Now, using essentially the same technology as a cell phone camera, NASA is able to capture digital images and put them up on its web site for the world to see immediately (as many have pointed out, much more quickly than NBC saw fit to broadcast Olympic events).  The web gallery includes not only the pretty press-release pictures but also the raw frames they were made from, including pictures of Curiosity staring at its feet and other such that didn't make the cut.  There are plenty of other goodies as well.  I particularly like the "white-balanced" images, which have been post-processed to show what the same terrain would look like under sunlight on Earth.

And there are, of course, a Twitter hashtag (#MSL) and (obligatory plug) updates on NASA's Google Plus page.

As inspiring as it may have been to watch thrilling news coverage with Walter Cronkite narrating, there's something much more intimate about being able to visit a web site from time to time and watch the story unfold directly from the source.

Dear Reader,

When I was in school (a while ago, admittedly), we learned how to write a letter:
Dear So and So, 
Blah, blah, blah ... 
David Hull
(or, with a close friend or family member, "Love, David" or "Yours, David", or such)

In email these days, if there's any salutation or closing at all, it's something like:
Hi So and So, 
Blah, blah, blah ... 
Some of this, I'm sure, is because times change.  The canonical letter of my youth is miles and miles less formal than one from, say, the 1700s.  Nonetheless, I think the switch from snail mail to email has had an influence.

For one thing, it feels a little funny to add a salutation and closing at all, when the email header includes the names of the sender and recipient.  On the other hand, it can feel a little funny not to have at least something.  So, I think, we tend to revert to what we'd say in a conversation, with both participants standing right there and knowing who the other is.

Beyond the switch to a new medium giving an opportunity for new forms, the medium itself influences the forms.  This has been true with letters as well.  Formal letters with half a page of salutations and closings make more sense when hand-writing and sending a letter is difficult and therefore reserved for occasions like petitioning the royal court, where you are quite possibly addressing an official you only know by name and trying to establish your reputation by means of any titles you may have.

Y'r Humble & Ob'd't S'v't
Etc., etc.

Saturday, August 4, 2012

Answering my random question

I recently asked whether there were more than a Britannica worth of Britannica-quality articles in Wikipedia.  Looking into it a bit, I'd have to generally agree with Earl that no, there aren't.

Britannica has about half a million articles (according to Wikipedia's page on Britannica).  English Wikipedia has about four million.  I would not say that one in eight Wikipedia articles is up to Britannica standards.

Granted, the famous Nature study of 2005 found that Wikipedia science articles are nearly as accurate as Britannica articles -- and that Britannica is far from flawless.  One can dispute the methodology and conclusions of that study, and Britannica did, but the overall conclusion seems at least plausible.

However, apart from science articles only being part of the picture, the writing in Wikipedia is uneven and full of Wikipedia tics.  Britannica, with full-time writers and editors, ought to be a bit better.  I tend to think this is where Wikipedia generally falls short. Factually, the two are comparable.  In style and organization, not so much.

Taking content and writing together, there are probably relatively few Britannica-quality articles in Wikipedia, but there are more than enough that are close enough.

Now CAPTCHA-free

A while ago I turned on "Word Verification", which makes people leaving comments read a hard-to-read word in order to prove they're not a bot.

This seems to have done more harm than good.  I still get the occasional spam comment, and it's a pain for people to leave real comments.  To see what to do about it, I Googled blogger captcha, and up popped this post urging bloggers to "kick Captcha to the curb".  The gist is, no, that extra inconvenience to real readers isn't really worth it.  Spam filters catch spam even if word verification is turned off.

And, of course, "It flags your blog as less professional".  If there's anything this blog stands for, in tone, subject matter and publication schedule, it's iron-clad professionalism.

So I'm turning word verification off.  If it turns out to be a horrible mistake, I can always turn it back on.  Otherwise, no news is good news.

More rumblings in the world of academic publishing

I've written before about the use of online outlets for quick publication of informal (that is, non-peer-reviewed) results, and arXiv in particular.  In The Case for Books, Robert Darnton expresses concern about the state of academic publishing and the power that the major publishers hold over academic researchers and libraries and wonders what will come of it all.

Now it seems things are heating up.  There is a boycott in progress against Elsevier, the academic publishing juggernaut that owns such publications as Lancet.  A number, and evidently a growing number, of academics are simply refusing to publish in or otherwise participate in Elsevier publications, on the grounds that Elsevier's high prices and profit margins and their overall practices are harmful to those who must publish in them, the institutions who must buy the publications, and to the free exchange of ideas itself.

At this writing, 12,558 people have signed up, giving their full names and affiliations in a searchable list.  These are not random people taking potshots from behind pseudonyms.  These people are putting their reputations on the line publicly and, by walking away from one of the major sources of recognition and exposure, potentially hindering their academic careers.  Their names may be found on

The basic issue here is that to have a career in academia, one must produce a steady stream of work.  The universal standard for measuring that stream of work is the number and quality of papers one publishes.  "Publish or perish."

Since anyone at all can print up a paper on a topic of research (and many do), there has to be some mechanism to determine whether a result has any real merit.  In the academic world, that mechanism is peer review.  If you submit a paper to a refereed journal, the editors will select a set of reviewers in your field to go over it.  The reviewers will either reject the article outright or accept it, likely with revisions.

Different journals have different standards for inclusion.  This allows readers to have some idea up front how worthwhile an article is, and provides some means of rating a researcher's output beyond the sheer number of articles published.  In principle, and for the most part in practice, the peer review process ensures that articles in journals are accurate and relevant, at least as far as the reviewers can tell at the time.  Essentially, journals provide brand names.

Peer review is clearly a valuable service, beyond printing and distribution of paper volumes, which is, of course, on the wane.  But there are problems.  In the call to action which started the current boycott,  Timothy Gowers puts forth several complaints:

  • Journals cost too much, particularly since the authors and reviewers are paid by their institutions, not the publisher, and it's largely the same institutions that pay for subscriptions to the journals they're paying to produce.
  • Online access is behind expensive paywalls.
  • Publishers drive the overall cost up by bundling, that is, requiring institutions to buy large numbers of journals, many of which literally go unread, in order to subscribe to the ones they really care about.  An institutional bundle from a given publisher can run into the millions of dollars per year.
  • While many publishers produce expensive journals and require bundling, Gowers calls out Elsevier in particular for several reasons, including supporting legislation that restricts access to published results and playing hardball with institutions that try to resist bundling.
In short, publishers are in serious danger of losing their relevance, and in the view of those joining the boycott, Elsevier is one of the worst offenders.

It's all well and good to object to publishers' behavior and organize a boycott, but the academic world also seems actively engaged in building a more open, web-enabled alternative.  This includes
  • Blogging as a means of informal sharing and discussion.  Indeed, Gowers' call to action appeared on his blog (which, with a mathematician's precision, he calls "Gowers's Weblog")
  • Sites, notably arXiv, for collecting unrefereed preprints.
  • New online refereed journals aiming to take the place of old ones.  Normally establishing a brand can be difficult, but if the editorial board of the new journal is made up of disaffected board members from old journals, their reputations come with them.

While writing this, I was wondering what would be a really webby way to do this.  Here's a sketch:
  • Articles would be published in something more like a wiki form, with a full revision history and editors making changes directly.
  • Since reputation is particularly important here, changes would ideally be digitally signed.
  • Individuals could put their imprimatur on (a particular revision of) an article they thought worthy.
  • The quality of papers could be judged by the reputation of those approving of them, which in turn would be judged by the quality of the papers they'd produced ...
And then it occurred to me that in practice there would probably come to be groups of people whose approval was particularly significant within particular fields.  It would be good to be able to establish groups of, say experts in homology or complex analysis.  It would also be good to have people who were good at steering new works to the appropriate groups of experts.

Hmm ... except for the revision history and digital signatures bit, this sounds an awful lot like a peer-reviewed online journal.

Friday, August 3, 2012

Cookies in the UK (or should that be "biscuits"?)

I haven't tracked down whether Parliament decreed this, though it seems likely, but a number of UK sites I've visited in the past couple of months show you a brief popup or other announcement to the effect that they use cookies (small files that your browser stores on your disk and hands back to the site on later visits so the site can tell it's you).  The announcement is typically a couple of simple sentences with a link for further information.  For example:
This site uses cookies. By continuing to browse the site you are agreeing to our use of cookies.  Find out more here.
The linked page details in clear, precise language what cookies are and what the site uses them for.  It explains how to set your browser to disable cookies for the site, with the understanding that you might not have as nice an experience since the site won't be able to remember who you are.  Once you dismiss the announcement you don't see it again, because -- of course -- it has set a cookie and knows not to come back (unless you disabled cookies or later clear the cookie).

Wow.  They Got It Right.  Well done!

Random question

Are there now more than a Britannica worth of Britannica-quality articles on Wikipedia?

Is there a UX crisis?

Back in the early days of computing, a software crisis was declared.  Projects were being launched with high expectations -- this was back when computers could do absolutely anything -- only to end up late, over budget, disappointingly lacking in features, buggy to the point of uselessness, or not delivered at all.

Many solutions were proposed.  Software should be written in such a way that it could be mechanically proved correct.  Software engineering should become a proper engineering discipline with licenses required to practice.  Methodologies should be developed to control the development process and make it regular and predictable.  There were many others.

None of these things has happened on a significant scale.  A proof of correctness assumes you understand the problem well enough to state the requirements mathematically, which is not necessarily easier than writing the code itself.  For whatever reason, degrees and certificates have not turned out to be particularly important, at least in the places I've worked for the past decades.

Methodologies have come and gone, and while most working engineers can recognize and understand a process problem when they see it ("Why did I not know that API was about to change?" ... "How did we manage to release that without testing feature X??"), there is a high degree of skepticism about methodologies in general.

This isn't to say that there aren't any software methodologies -- there are hundreds -- or that they're not used in practice.  I've personally seen up close a highly-touted methodology that used hundreds of man-years and multiple calendar years to replace an old mainframe system with a new, state-of-the art distributed solution that the customer -- which had changed ownership at least once during the wait -- was clearly unhappy with.  And well they should have been.  Several months in it had been scaled down as it became clear that the original objectives weren't going to be met.

I've also seen "agile" methodologies put in place, with results that were less disastrous but not exactly miraculous either.  Personally I'm not at all convinced that a formal methodology is as helpful as a good development culture (you know it when you see it), frequent launches, good modularity and lots of testing.

Several things have happened instead of a cure, or cures, for the software crisis.  Languages and tools have improved.  Standards, generally de facto, have emerged.  Now that a lot of software is out, both customers and developers have more realistic expectations about what it can and cannot do.  Best practices have emerged (Unit tests are your friend.  Huge monoliths of code aren't.).  Projects get delivered, often late, over budget, lacking features and buggy, but good enough.  And it's just code.  We can always fix it.  I can sense the late Edsger Dijkstra shaking his head in disapproval as I write this, but nonetheless the code is running and a strong case can be made that the world is better for it.

We don't have, nor did we have, a crisis.  What we have is consistent disappointment.  We can see what software could be, and we see what it is, and the gap between the two, particularly in the mistakes we get to make over and over again, is disheartening.

Which leads me back to a persistent complaint: UXen, in general, suck.

Yes, there are plenty of examples of apps and web sites that are easy to use and even beautiful, but there are tons and tons that are annoying, if not downright infuriating, and ugly to boot.  For that matter, there are a fair number of pretty-but-useless interfaces.  Despite decades of UX experience and extensive research, basic flaws keep coming back again and again.  Off the top of my head without trying too hard:
  • Forms that make you re-enter everything if you make a mistake with anything (these actually seem to be getting rarer, and a good browser will bail you out by remembering things for you -- and in many cases that's a perfectly fine solution).
  • Lists of one item that you have to pick from anyway as though there were an actual choice.
  • "Next" buttons that don't go away when you get to the last item (likewise for "Previous")
  • Links to useless pages that just link you to where you wanted to go in the first place.
  • Security theater that pretends to make things safer.  Please make it stop.
  • Forms that require you use a special format for things like phone numbers.  Do I include the dashes or not?
  • Wacky forms for things like dates that throw everything you know about keys like backspace and tab out the window.
  • Error handling that tells you nothing about how to fix the problem.
  • Layouts that only line up right on a particular browser.
  • Pages that tell you to "upgrade" if you're not running a particular browser.
  • General garish design. Text that doesn't contrast with the background, which is too busy anyway.  Text that contrasts too much.  Cutely unreadable fonts.  Animated GIFs that cycle endlessly.
  • Things that pop up in front of what you're trying to look at for no good reason.
  • Editors that assume, a la Heisenberg, that the mere act of opening an edit window on a document causes unspecified "unsaved changes" that you must then decide whether or not to save (yeah, Blogger, you're guilty here).
And so forth.  This is just off the top of my head.  I've ranted about several of these already, though for some reason the industry doesn't seem to have taken heed.

How does this happen?

How does any less-than-satisfactory design ever happen?  One answer is that reality sets in.  Any real project is a compromise between the desire to produce something great and the need to get something out in front of the customer.  Perfect is the enemy of good enough.

In an ideal world, people would be able to describe exactly what they want and designers could just give it to them.  In the real world, people don't always know what they want, or what's reasonably feasible, and designers don't always know how to give it to them.  In the ideal world a designer has at hand all possible solutions and is never swayed by the desire to use some clever new technique whether it really applies or not.  In the real world designers are humans with limited resources.

This isn't unique to software by any means.  Doors have been around for millennia, and people still don't always know how to design them.

I should pause here to acknowledge that UX is difficult.  There are rules and methods, and tons of tools, but putting together a truly excellent UX that's both pleasant and fully functional, that makes easy things easy and hard things possible, takes a lot of thought, effort and back-and-forth with people actually trying to use it.

Again, though, that's not a property of UX.  It's a property of good design.  The question here is why are UX things that seem simple enough -- like avoiding useless buttons and links -- so often wrong in practice.  A few possible answers:
  • Actually, UX designers get it more-or-less right most of the time.  We just notice the failures because they're really, really annoying.
  • It's harder than it looks.  It's not always easy to figure out (in terms even a computer can understand) that a link or button is useless, or how to lay something out consistently on widely different screens.
  • The best tools aren't always available.  Maybe there's a really good widget for handling a changing list of items that allows for both quick and fine-grained scrolling and so forth.  But it's something your competitor wrote, or it's freely available but not on the platform you're using.
  • Dogma.  Occasionally guidelines require foolish consistency and UX is not in a position to bend them.  This may explain some tomfoolery regarding dates, social security numbers and such.
  • Plausible-sounding reasoning that never gets revisited.  It may seem like a great idea to make sure you have a valid social security number by requiring the user to put in the dashes as well.  That way you know they're paying attention.  Well, no.
  • Reinvented wheels.  The person doing the UX hasn't yet developed the "this must already exist somewhere" Spidey sense, or thinks it would be Really Cool to write yet another text editing widget.
  • Software rot.  The page starts out really nicely, but changes are jammed in without regard to an overall plan.  Inconsistencies develop and later changes are built on top of them.
Hmm ... once again, none of these seems particularly unique to UX.  Time to admit it: UX is a branch of software engineering, liable to all the faults of other software engineering endeavors.  Yes, there is an element of human interaction, but if you think about it, designing a library for people to code to is also a kind of UX design, just not one with screens and input devices.  You could just as well say the same things that make UX development error prone make library design error prone as the other way around.

To answer the original question, there is no UX crisis, no more than there was a software crisis.  We just have the same kinds of consistent disappointment.

But who asked?  Well, I did, in the title of this post.  Interestingly enough, no one actually seems to have declared a UX crisis, or at least the idea doesn't seem to have taken off.  Maybe we have learned a bit in the past few decades after all.

Wednesday, July 25, 2012

There's an app for that web site

(To whoever posted the last three comments -- not that there's even the slimmest chance you'll read this -- no, I do not want to buy French sunglasses from you.  Quel dommage.)

Ah, the late 90s.  We knew everything.  We knew that phones, TVs and PCs were going to "converge" until there was no real difference between them.  We knew that the web was exploding and was going to keep exploding.  "Broadband", that is, speeds faster than 56kb, was going to be everywhere.  We knew that mobile computing was going to be big and that the web would necessarily look different on a phone as opposed to a big monitor with a keyboard and mouse.

Honest.  I remember people talking about all this in the hallways and in the restaurants that always seemed to have at least two VCs interfacing animatedly in the booth behind you, before I left the Valley.

And we were right.  Unfortunately, we were wrong about a few of things as well, like whether this was all going to happen "right now, at internet speed" or over the course of decades.  And whether a company had to actually show a profit to be worth a gazillion bucks.  And this idea that in order to have half a chance in this blindingly fast new world, you had to become the "first mover" no matter the cost.  And buildings like this.  I mean, who would want to work there?

Now that web-and-video-enabled phones with decent bandwidth are commonplace, what does the web look like on them?  Well, if you actually try to use your phone's browser, it looks pretty unimpressive.  Pages come up in tiny print.  If you try to zoom in so you can actually read them, they may or may not reformat so as not to spill over the edge of the screen.  Selecting links or navigating to the right text box can be pot luck.  In all, pretty dismal.

Not that there haven't been efforts to make web pages look and feel differently if the browser is running on a phone.  There certainly have been, and again, I recall some of those efforts from back in the day.  But that's often not what happens.  What happens instead, often, is an app.

I can read this blog on a phone browser, if I want to, and it looks OK, because Blogger has machinery in place to present it in "feed" form, without all the formatting of the full web version.  This is exactly in line with the "one web site for all browsers" model, but it takes considerable extra effort.  If I go to a random web site, including major ones, I may or may not arrive at something useful.  At the end of the day, phones are just too different from the big-screen/mouse/keyboard setup.

To deal with the small screen, limited keyboard facilities and other peculiarities, phones have to do things significantly differently:

  • Much less text fits on the screen and typing is often cumbersome, so graphics play a larger visual role.
  • The layout changes, often radically.  Elements appear and disappear depending on where attention is focused.  Buttons are more common than links.  Input elements like buttons and text boxes tend to have reserved chunks of real estate, as opposed to being part of a big page that scrolls.
  • A touch screen favors gestures like swiping, pinch/spread for zooming out or in, long press instead of some altered flavor of click (right-click, shift-click, control-alt-meta-cokebottle-click ...), and so forth.
  • Autocomplete is even more important.
  • A phone is more apt to lose and regain connectivity, so it often makes sense to cache results deliberately, as opposed to counting on some generic caching layer to hold on to whatever happens to be around at the moment.
  • Phones are mobile, so physical location can play a much bigger role.  Not a lot of turn-by-turn GPS web pages out there.
  • Phones are phones.  You might switch from listening to a song to taking a call at any moment.  To some extent different apps on the phone have to cooperate to make this happen smoothly.
Put this all together and it's going to be next to impossible to maintain a web site that can automatically look good on all the major browsers and all the major phone platforms.  A better solution is to separate the information in the web site from its presentation and develop the PC/laptop presentation more or less separately from the phone presentation.

That explains why a good portion of apps are essentially web sites redone for the phone.  As long as the separation is done reasonably cleanly, this is the right call.  A weather web site and a weather phone app ideally share the same raw weather information, and probably a fair bit of common elements like icons for "sunny" and "fair to partly cloudy", but the web designer doesn't need to figure out how to recognize and handle a swipe gesture and the phone designer can dispense with a lot of web markup machinery.

It took me a while to pick up on this, not because it's that hard to notice but because I'm a little slow that way.  "Apps", huh?  Sure are a lot of them, and a lot that sound like web sites.  What's the point?  Must be some sort of marketing gimmick.  But of course apps are not a gimmick at all.

Saturday, July 14, 2012

75 years of Tanglewood online

This has actually been going on for a while, but in keeping with the usual Field Notes standard of cutting-edge reportage I only just now noticed that the Boston Symphony Orchestra, as part of its celebration of the 75th anniversary of its Tanglewood concert series, is bringing out 75 concerts from its vaults throughout the summer.  Many of the concerts had not been previously available and as I understand it some are of programs that were only performed at Tanglewood.

The BSO is making one new concert available each day as a free stream.  After the first day the concert is available for sale, whole or in parts.  You can also subscribe to the whole series at a substantial discount off the cost of buying the concerts individually.

Imagine what a promotion like this would have looked like before the web.  The symphony would have worked out a deal with one or more radio stations to get a regular block of time for broadcasting the day's selection.  Assuming it could swing the deal, you the listener would have to set aside that same block of time to listen to the concert, or at least record it off the air for later listening.

The symphony could make the entire series available for mail order as a set of CDs (or vinyl, if we want to go back in time).  If you didn't want the full set, you might be able to order individual CDs, but you wouldn't get to pick what was on them.  If you liked one piece from each of five concerts, you could end up buying five CDs to get them all.  And then you'd wait for them to show up in the mail.  If you lived outside the listening area of the radio stations involved, you'd have to buy the concerts on spec without a chance to listen, and you'd be more likely not to have heard about them at all.

Put together all the conveniences of the web, I wouldn't quite say you've got a revolution.  The dedicated classical music fan has had access to top-quality performances for quite some time.  Nonetheless, it's enough to make a difference.  Whether it's also enough to keep the symphonies in business in this age of digital entertainment remains to be seen, but it certainly seems like a good approach to try.

Thursday, June 28, 2012

Yet another wacky security scheme

Passwords are easy to get wrong.  Trying to make people come up with "stronger" passwords just makes it worse.  Security questions just provide another avenue of attack, probably an easier one.  So, ladies and gentlemen, may I introduce to you: The security word.

"What is it?", you may later regret asking.

You give the site a "security word".  Later, they will ask you not for the word, but a few randomly selected letters, for example the second, fifth and eighth, and next time it might be the first, fifth and sixth (note to self -- lopado­temacho­selacho­galeo­kranio­leipsano­drim­hypo­trimmato­silphio­parao­melito­katakechy­meno­kichl­epi­kossypho­phatto­perister­alektryon­opte­kephallio­kigklo­peleio­lagoio­siraio­baphe­tragano­pterygon may not be the best choice for this exercise).

If you picked, say, security, and the system asks for the second fifth and eighth letters, you would give 'e', 'r' and 'y'.  If someone's looking over your shoulder, how much information do they have?  Let's fire up the old UNIX shell

$ grep '^.e..r..y.*' /usr/share/dict/words | wc -l

What this means is that there are 84 words in the dictionary on my system that have 'e', 'r' and 'y' in those positions, or about six bits of entropy.  Most of them are words like ventrohysteropexy and dextrogyratory that people are unlikely to pick.  The person who helped me set up the account in question recommended something "easy to remember".  Odds are it's "security".

If not, all an attacker has to do is guess the letters that the site asks for next time.  There's a good chance that at least one will be one the attacker has already seen.  There won't be a lot of choices for the unknown letters.  Without looking at the list, I'd bet that 'q' isn't on it and 'e', 't' and a few others cover most of the possibilities.  Even without having looked over your shoulder, an attacker would know just from the security word being English that certain letters are better to try in certain positions.

So basically you have another hoop to jump through that adds minimal actual security, but tries to create the illusion of strong security, while really just making the system harder to use.  Huzzah.

Wednesday, June 27, 2012

Where's the fire?

It's been a busy fire season in the Southwest, with hot, dry weather and an abundance of fuel.  While I'm generally skeptical about the world-changing potential of social media (ultimately, of the idea that a new technology is necessarily likely to have a great impact), that doesn't mean social media can't play a role.

One example is this fire map from esri (makers of geographic information systems (GIS) software).  Along with official data such as wind information from NOAA and fire perimeters from USGS, it includes layers for Flickr, Twitter, and YouTube activity.  The YouTube feature for some reason got stuck on the same video until I reloaded the page, at which point it worked fine.

The Twitter layer seems to be tagged by the location of the person tweeting.  For example, someone in Fort Collins tweeting a Denver Post story about the Waldo Canyon fire over a hundred miles away is shown in Fort Collins, not where the fire is.  To be fair, it's a lot easier to locate the person tweeting than to figure out from the contents of the tweet that it's really about something somewhere else.

All in all, esri's map seems useful to me mostly for the official information.  The social layer is interesting, but by no means essential.  Searching for "Colorado fire" on Twitter search turns up many more tweets, at least as relevant as those from the map.  Likewise for a YouTube search.  Neither of these searches directly maps the location of the footage, but this doesn't seem like a great obstacle.  Wildfires are quickly given distinctive names ("High Park fire", "Waldo Canyon fire") and you can easily search on those.

And of course wildfires would quickly be given distinctive names.  People need to tell them apart.  If I live in Colorado Springs, I don't care much about the High Park fire, but I care a lot about the Waldo Canyon fire.  As a side effect, it's easy to search for information about a given fire without consulting a map.

And what does such a search find?  Among other things, quite a few links to, and videos from, local newspapers and TV stations.

In short, what does a social-media-enhanced map and search space look like?  A fair bit like one without social media, at least in the context of events with wide interest where there are well-developed traditional media sources.

But broadcasting was never really supposed to be the strong point of social media anyway.

Tuesday, May 1, 2012

Since when are we all on a first-name basis?

Time was, if you had to line a bunch of people up alphabetically, you did it by surname:
  • Ball, Lucille
  • Marx, Harpo
  • Zoolander, Derek
More and more, though, we seem to be arranging people by first name
  • Derek Zoolander
  • Harpo Marx
  • Lucille Ball
Seems odd.  I blame the web.  Gmail does it.  Facebook does it.   Wikipedia can generally autocomplete from a first name ("Ab" gets me Abraham Lincoln at the top) but not the last (spelling out "Lincoln" in full gets me Lincolnshire and a longish list of other names, but nothing on the president).  There are plenty of other examples, I claim.  Indeed, it almost seems to be becoming the norm.

Objectively, there's probably not much to pick between the two schemes.  Without actually measuring, I'd guess that last names tend to be more unique (or "more nearly unique", if you must), but on the other hand we tend to think of people with their first names, well ... first.

Now perhaps this is just a generational thing.  Kids These Days, after all, have no regard for propriety and convention, just like my generation before them.  Perhaps the major software companies have had a hand, spreading their Silicon Valley disregard for regimented old-school thinking.

However, I think Wikipedia's autocomplete is notable.  If we're searching for something about someone, the key to our search is the person's name, as we would say it.  I don't think "What was Marx, Harpo's birth name?" (Adolph, changed to Arthur by 1911).  I think "What was Harpo Marx's birth name?" or just "What was Harpo's birth name?" (or I could just honk and whistle).  Either way, I start with "Harpo", and sure enough, "Harpo Marx" is on the list by the time I've typed that.  Other systems work similarly.

Searching by typing something in and expecting results to come back is quintessentially webby, and the autocomplete box is Web 2.0 in particular (whatever Web 2.0 is).

Friday, April 6, 2012

Old albums

A few weeks ago the Encyclop√¶dia Britannica finally threw in the towel and, after over 200 years, stopped publishing the lovely multi-volume sets that have graced bookshelves the world over.  Naturally, there has been a run on the last edition (2010).  When I heard the story on the radio today there were supposed to have been no more than 800 copies left.  I'd be surprised if there were any left by now.

Clearly the whole point of this run is to get at the physical volumes.  The contents can be had digitally for much less.  The doorstop edition is valuable for the same reason any artifact is valuable apart from any utility it may have: rarity and emotional significance.

Imagine you are rummaging through an attic trying to decide what to keep and what to throw.  You run across a stash of vinyl LPs of popular hits from the 70s.  Odds are most if not all of the songs can be had digitally with better sound, but that's not the point.  As with the 2010 Britannica it's the physical artifact that matters.  Do you like that vintage artwork on the jacket with the circular imprint of the record worn into it?  Do you enjoy the tactile experience of dropping the needle on the platter, the crackle and pop of surface noise, the ritual of cleaning any wayward lint from the grooves?

Then you run across an album of photos, page after plain page of pictures tucked into little white corner-pockets, colors desaturated, edges curling.  Tucked into an envelope with them are the negatives.  Scan them and you probably have images of reasonable quality that you can attach to an email, share on your favorite social site and archive durably.  The physical artifact is less important here.  It's the actual images that matter, images you can't get anywhere else.  With the bits, you could create another album as good as the original one.

That's the common question that determines what's really of interest: what can't you get anywhere else?  It's not a matter of songs versus pictures or LPs versus photos.  If the vinyl in the Greatest Hits album is warped and cracked and the album art is nothing special, you may as well just buy the tunes online.  If the photo album is something your great Aunt put together, with cutouts and notes and decorations, you probably want the physical album as much as the images.

If the content is important, then you'll want to get it into the cloud, or at least into bits on some local disk.  If the artifact is important, then the web will play less of a role.

What's twice as big as the internet?

(Yikes ... I went 0 for March!)

I've mentioned before that telescopes can generate a lot of data.  IBM seems inclined to drive the point home by collaborating with ASTRON (the Netherlands Institute for Radio Astronomy) to put together "exascale" computing horsepower behind the world's largest radio telescope.

The telescope is actually (or rather, will be) an array of millions of antennas spread out over a square kilometer, from which the name SKA, for Square Kilometer Array.  This array is expected to produce on the order of an exabyte of data per day.  This is an absolutely ridiculous amount of data by today's standards.  Think one million terabyte disk drives, or twenty million feature film's worth of Blu-ray, or ... according to IBM, twice the daily volume currently carried on the internet.

I'm a little skeptical as to exactly how one measures that, but hey, you've got to trust a press release, right?

So where do you put an exabyte a day worth of data?  Well, you don't.  You're certainly not going to upload it to the web.  Particle physicists are faced with the same problem of having to figure out what portion of a huge data set to keep for later analysis, and a large part of running an experiment is setting up the "trigger" criteria by which the software collecting the data will decide what to keep and what to throw.  IBM and ASTRON's system will be dealing with the same problem, but on an even larger scale.

Or I suppose you could sign up two million people and somehow stream an equal share of the data to each at Blu-ray resolution all day every day, but somehow I doubt that kind of crowdsourcing will help much.

Friday, February 24, 2012

Is it OK to tweet "fire" in a crowded theater?

Evidently not.

Or at least, it's not a good idea to tweet in jest that you'll blow an airport sky-high if it remains closed for snow, so preventing you from visiting your girlfriend.  Paul Chambers of Doncaster, England found this out the hard way, paying a fine of £1000, gaining a criminal record and losing his job in the bargain.  His appeal will be heard before the high court of the UK and his defense has had at least one high-profile fundraiser, but it's all a bit sobering, to say the least.

This lack of humo(u)r on the part of airport security is not new, by the way, nor limited to the UK.  I remember as a kid -- so, ahem, well before 9/11 -- noticing a sign at the airport we were flying out of saying it was a federal crime even to joke about hijacking, bombs and such, and promptly blanching and making a mental note not to make any smart comments to the nice folks by the metal detector.

With that in mind, the remarkable aspect of the case isn't so much that it involves Twitter, though it is one of the first such cases, but that the authorities chose to prosecute for this particular remark at all.  I don't know how often such cases are prosecuted, but I'd guess it's not too often.  They certainly don't seem to make the press much.  I doubt the story would have been less remarkable had Mr. Chambers been brought in for making the same remark in person at the ticket counter.

In any case, caveat tweetor.

[Paul Chambers' conviction was eventually quashed, two and a half years later on the third appeal, the case having attracted considerable attention and celebrity involvement.  It's not clear if his job was reinstated, but according to Wikipedia he and his girlfriend did eventually marry.]

Saturday, February 11, 2012

Now I've seen everything

Sorry, horrible title.  I couldn't resist.

"Blind photographer" isn't a phrase that would spring to most people's minds readily, but not only are there such, there is -- of course -- a blog dedicated to blind photography.  From what I can tell, the photographers featured here aren't totally blind, but they are legally blind.  For example, I originally stumbled on this blog after reading about Craig Royal, who writes "My peripheral vision is blurred and the central vision is obscured by a white blindspot." and who processes his pictures with the aid of Photoshop and a telescope.

In other words, the blindness in question, while not complete, is very real and has a real effect on the images produced.  Indeed, the photographs on the site have a character all their own and, in my personal estimation, are just plain good art.

Clever copy and paste

Generally, if you select some part of a web page, copy it and paste it somewhere else, you'd expect to see pretty much what you'd selected, maybe with the formatting munged a bit.  Recently, though, I copied something (small enough for fair use) from one of the major sports outlets and was mildly surprised to see that it pasted with a handy "Read more" link including the URL of the article I'd quoted.  You can do that sort of thing in today's wonderful world of AJAX.

I suppose one could see this as an attempt to control copying of copyrighted material, which was muddled somewhere into my initial reaction, but really it seems like a more or less useful thing to do, and completely legitimate for a commercial publication.  For that matter, even in a non-commercial context attribution matters and an automatic backlink could be a nice feature.

Thursday, January 26, 2012

What, if anything, is a magazine?

A recent New York Times article tells the store of Esquire magazine's troubles in 2008 and 2009, and how it was able to survive them by adapting to the world of online publishing.

I'm not sure I've ever read Esquire in either print or digital form.  For that matter, I don't buy magazines much any more, but I do follow the (free) online content of some, particularly The Economist.

So do I, or does a digital Esquire subscriber, read magazines?  Pretty clearly yes, just as there's pretty clearly more to a magazine than its print edition.  So what's a magazine?  Some thoughts:
  • A classic magazine is almost always periodical, though a few publish irregularly.  On the web, content generally goes up when it's ready, regardless of the print publishing schedule.  Let's say a magazine is an ongoing publication.  There may or may not be a sequel to your favorite book, but part of publishing a magazine is the promise that there will be more.
  • A magazine is not tied to any particular individual.  Even in cases like Forbes or Oprah, where a particular individual's identity is an integral part of the brand, the actual magazine is the work of many people.  It is an institution, that can survive the departure of any particular person (though in some cases better than others).  This is where we can probably best see the tie to the earlier sense of magazine as a storehouse, and it's also a distinguishing feature between an online magazine and a blog.
  • Even though it's a group effort, a magazine does have a personality, or at least a good one does.  Even if its contributors don't always see eye to eye, there will be something about having that particular mix of opinions and styles that makes the magazine what it is.
From this point of view, as long as there are ongoing publications with multiple contributors and a recognizable personality, there will be magazines, regardless of the actual mechanics of publishing.

A corollary to that is that there ought to be just as much of a market for magazines as there ever was.  The puzzle, as always, is reaching that market and making sure everyone still gets paid, which is why I find it interesting that the headline of the Times article is in past tense: "How Esquire Survived ...", not "How will Esquire survive ..."

Wednesday, January 18, 2012

Does there have to be an app for that?

Weddings are generally public affairs, and they always have been.  I doubt it's ever been particularly difficult to find out who's planning a public wedding and when in a given area.  With the advent of online wedding planning it's now perhaps a bit easier yet, and if you're looking for a wedding to crash, well, there's an app for that.

Now, I'm with the author of the article in thinking that the kind of person who would use such a thing has -- how shall we say -- issues, but the flip side of that is, who's actually going to use it, as opposed to just having a laugh looking up one's friends and acquaintances?  Or more precisely, who's going to use it who wouldn't have been willing and able to crash a given wedding anyway?

In general, there's a lot of gray area when it comes to "enabling technologies", not to mention the larger sticky issue of to what extent technology can or should be considered without considering its potential consequences.  On the one hand, it's easy to say "The real problem is the wedding sites' privacy models.  The app just pulls together information that's already available."  But that's a cop-out.  As we've seen, pulling together information that's already available and making it universally accessible (if not useful) can make a significant difference.  Sometimes this is good, sometimes not, and just because something can be done doesn't mean it should be.

Just how much of a difference pulling together existing information and making it easy to get to can make depends on what the information is, how hidden it was, who wants to know and a host of other factors.  In this particular case, I doubt the app will make much difference.  That's not to condone wedding crashing, or the app, or to excuse its creators.  If your wedding is crashed by some tech-savvy boor who would otherwise have missed out, you have my sympathies, for whatever that's worth.