More AI kool-aid

Another in the occasional series of "not directly related to the web" posts -- if only to remind us that such topics still exist:

The latest edition of 60 Minutes features a piece on functional Magnetic Resonance Imaging (fMRI) entitled Reading Your Mind. Back in the Golden Age of Science Fiction (more on that later), it was a given that, since the brain had been discovered to give off electromagnetic signals, it would be possible to interpret those signals to determine what someone was thinking. Fast-forward several decades and throw in a whole lot of technology, and we can now do some interesting things. For example:

• Determine which concrete noun, drawn from a limited set, a subject is thinking of (there's some nice stagecraft in the video version of that segment, but never mind).
• Determine whether a subject is thinking of adding or of subtracting two given numbers.
• Determine whether a subject has seen a particular view of a virtual environment before.
• (They don't give as much detail on this one): Distinguish signatures for kindness, hypocrisy and love.

To go with that, a few grains of salt. All of the above is done under very artificial conditions. For starters, the subject is lying in an MRI machine.

The thoughts being identified are carefully and narrowly defined. In the first two cases, the set of possible responses is taken from a small, specific set. This seems very much like the case of speech recognition. Currently it's not too hard to recognize words and phrases spoken by a variety of speakers so long as they're drawn from a small list ("technical support" or "order status", say).

In the third case, the system is really trying to spot a marker for recognition of a familiar scene. In the last case, who knows what they're doing? All in all, the subtitle, Incredible Research Lets Scientists Get A Glimpse At Your Thoughts, hits it about right. Great research, but the end result is a "glimpse" not a "reading".

There are also some significant ethical questions, many of which are raised in the piece by Paul Root Wolpe, director of the Center for Ethics at Emory University in Atlanta. Not the least of which is, to what extent should we believe any of this?

Granted, we are looking at objectively observable effects, reproducible under laboratory conditions. No one's doubting the basic data. But the further you get from controlled, concrete results like "when the subject saw a picture of a hammer, these sites lit up", the more room there is for interpretation. In one disturbing case in India, a woman was convicted of murder based at least in part on fMRI evidence that indicated she was familiar with the circumstances of her husband's death by poisoning. This seems like several more levels of extrapolation than the current state of the art warrants, particularly when lives are at stake.

I would say that the star of the show, CMU neuroscientist Marcel Just, is also guilty of unwarranted extrapolation, though in a mostly harmless way, in this exchange at the end:

"Do you think one day, who knows how far into the future, there'll be a machine that'll be able to read very complex thought like 'I hate so-and-so'? Or you know, 'I love the ballet because…'?" [Lesley] Stahl asked.

"Definitely. Definitely," Just said. "And not in 20 years. I think in three, five years."

"In three years?" Stahl asked.

"Well, five," Just replied with a smile.

Really? I'd like to try that. Five years from now (so January 2014) I'll gladly step into whatever test apparatus Just and CMU have set up at the time provided:

• Everyone else involved in the experiment is willing to go through the same steps first (hey, I know people do MRIs all the time, but still ...)
• I will have never seen or been seen/scanned/whatever by the equipment before the test.
• The equipment will render its interpretations without human aid (naturally, there will be people setting up equipment, pushing buttons and so forth, but the machine's opinions must be its own).
• In no case will I be shown images or words or receive other cues to respond to beyond those listed below (no cold reading techniques, please, however well-intentioned).
• Directly before the test, I will submit a strongly encrypted message containing my answers to the questions. I will make every good faith effort to think of the same answers during the test (perhaps the CMU team will have some way of verifying this?). My pre-test answers will be decrypted directly after the test for comparison (perhaps the team can glean the relevant passphrase for bonus points?).
• I assume any remaining details, such as "What's a major news outlet?" can be resolved in due time, and that in general everyone assumes good faith.

Before I go on: there is, or at least ought to be, a definite James Randi cast to the conditions I'm describing. That's not an accident, but it's not because I doubt for a minute that there's real science involved here. Rather, I'm convinced there is real science going on and I'd like to see a nice rigorous test of it, to the best of my limited abilities to concoct one. So without further ado, here are some tasks I would like to see put to the machine:

• On hearing the cue "person and feeling" I will think of a person whose name has appeared in the New York Times in the previous year, and consider my feelings about that person. The machine should identify the person and whether I regard the person favorably, unfavorably or neutrally.
• On hearing the cue "art form and reasons", I will think of an art form and three reasons one might like it. The machine should identify the art form and the reasons. [There may need to be some negotiation over how specific the art-form could be. Ballet should be fine, origami is probably OK, wrought-iron sculpture or Joycean wordplay might be a bit too specific, and odd-meter blues-based oboe improvisation in f-sharp minor, however artful, would be cutting way too thin -- though I'd be interested to hear one]
• On hearing the cue "U.S. county", I will think of a county (or parish, or borough as the case may be) somewhere in the United States. The machine should identify the county (including the state it is in).
• On hearing the cue "celebrity", I will think of an image of a celebrity mentioned in a news item from a major outlet no more than three days before the test. The machine should identify the celebrity.
• On hearing the cue "musical selection" I will think of a musical selection released as a track (or whatever we're calling them then) by a major label (or whatever is publishing music then). The machine should identify the selection.
• On hearing the cue "Euclid", I will think of the proof of a proposition from Euclid's Elements (books I - XII). The machine should identify the proposition in question.
• On hearing the cue "Shakespeare", I will think of a line from the 1609 printing of Shakespeare's Sonnets. The machine should identify the line.
• On hearing the cue "hoops" I will think of an NCAA Division I basketball team. The machine should identify the team.
• On hearing the cue "field note", I will think of a post from this blog. The machine should identify the post.
• [While proofreading, I thought of one more. It would make an even ten, though I wanted an odd number to ensure that "majority" is well-defined: On hearing the cue "legal principle", I will think of a legal principle defined in Black's Law Dictionary. The machine should identify that principle.]

I will gladly publish the results, whomever they favor, to this blog or, if that's not available, to the web in some other form (ah .. so this is about the web after all).

At this writing, I'm confident that no machine will exist in five years, in CMU's lab or anyone else's, capable of identifying the majority of those thoughts. I would be impressed by a machine that could reliably identify any of them.

I suspect that the best shot would be to pick up on the short-term "phonological loop" by which we are thought to store audio "images", as when silently repeating a phone number in order to remember it. A couple of the items are aimed at thwarting that, and I would certainly feel free to think of, say, the sights and sounds unique to a particular county rather than the name of that county, particularly if it were one I'd visited.

I also suspect that broadening the range of possible answers significantly raises the bar and the next five years' research, however diligent, will not be enough to clear it.

But I'd be happy to be proved wrong.

P.S. Wolpe (the ethicist) says "I always tell my students that there is no science fiction anymore. All the science fiction I read in high school, we're doing." While I appreciate that many of the hypothetical ethical problems raised by science fiction are no longer hypothetical, it's not hard to come up with a list of science fiction staples that are nowhere near reality. [Maybe he just didn't read that much science fiction?] Since he's an ethicist and not, say, a physicist, I won't trot out anywhere near the full list that came to mind, but one could start with faster-than-light travel, or even interstellar travel at half light-speed, or even accelerating anything macroscopic to half light-speed, or a permanent moon base, or a Star Trek-style transporter, or ...

[Stunningly, no one contacted me to take me up on my carefully-crafted challenge.  And, of course, five years down the line it doesn't look like anyone is claiming anything close to Marcel Just's prediction.  A deeper question: What prompts people who, one would think, ought to know better than to make such predictions to make them anyway?  --D.H. May 2015]