Crowds and wisdom

Suppose you're measuring, say, the four walls of a room. Have twenty people each do the measurement with off-the-shelf tools. Then do the measurement very carefully with a laser interferometer or whatever. While we're in gedanken mode, assume that the room itself is very precisely joined, so that measuring to a few decimal places actually means something.

The central limit theorem tells us that the measurements will tend to fit a normal (i.e., "bell curve") distribution peaking very near the precise length. If you take the average of the imprecise measurements for each wall, the result will generally be quite close to the precise measurement. If you consider all four walls, the combined result -- the four averages -- will generally be closer to the precise measurement than the best individual set of four measurements, assuming the errors in the imprecise measurements are random.

Now take a typical "wisdom of crowds" example: the Oscar ™ party where each guest guesses who will win. A ballot consisting of the most popular choices almost always does better than the best individual ballot. Clearly this is not quite the same as the measurement problem above. At the very least you'd need a different metric. On the other hand, is this "wisdom", or just statistics at work? Two possible answers, not necessarily incompatible:

Wisdom of crowds is about more than Oscar ™ parties. Surowiecki's original book talks about situations like pedestrians optimizing traffic flows or markets setting prices. The key ingredients for crowd wisdom, he argues, are diversity of opinion, independence, decentralization and aggregation.

The party example is fairly low-powered. It might be explicable in terms of statistics, but something like cars not hitting each other or customers distributing themselves among popular restaurants may not.

Wisdom is more about statistics than we might like to think. The words "wise" and "wizened" come from the same root, having to do with age [In a comment to a post seemingly chosen at random, Earl points out that this etymology is hogwash. Dictionary.com confirms this. Nonetheless, the point still seems valid]. Wisdom is the judgment we (ideally) gain with life experience. It's a matter of gut feel or intuition, not of sequential reasoning. In other words, it seems more likely based on statistics than logic. Do we grow wiser mainly by accumulating more data points?

A stock example is chess mastership. Chess masters typically look at many fewer moves than beginners or even experts, but the moves they look at are better. A master will also typically be better at remembering positions from actual games, as opposed to random placements of pieces, while the rest of us will do about equally well at each. A master is drawing on a large store of game experience and using this to structure the analysis of the position at hand. Clearly there is more involved than a simple "this position looks like that one", but just as clearly that's part of the picture.

Whatever statistical analysis a master is subconsciously doing isn't simple enough to have been captured algorithmically. Computers can beat masters at chess, but they do it by bashing through vast numbers of hypothetical moves. Programs that try to "understand" positions beyond simple rules like "I have more material and my pieces are more centrally located" tend to fail.

If that doesn't muddy the waters enough, you might consider this viewpoint.