The Wisdom of Crowds

The book “The Wisdom of Crowds” by James Surowiecki explains how the aggregation of information in groups can result in outcomes better than if the decision was made by any single person in the group. He argues that there are 3 types of Crowd Wisdom:

• Cognition: Market judgment, which he argues can be much faster, more reliable, and less subject to political forces than the deliberations of experts, or expert committees.

• Coordination of behavior, such as optimizing the utilization of a popular restaurant, or not colliding in moving traffic flows. The book is replete with examples from experimental economics, but this section relies more on naturally occurring experiments such as pedestrians optimizing the pavement flow, or the extent of crowding in popular restaurants. He examines how common understanding within a culture allows remarkably accurate judgments about specific reactions of other members of the culture.

• Cooperation: How groups of people can form networks of trust without a central system controlling their behavior or directly enforcing their compliance. This section supports the Chicago school of free markets.

In Operations Research, I spent a lot of time modeling Coordination and Cooperation. In my studies, the societies of the Ant and the Bee display incredible Coordination and Cooperation.  What makes them work, however, is arguably their biology and that they function with goals via a rank-and-file anthropology. There’s no vying for spots or rank, each member knows what their job is and goes about doing it.  There’s no evidence of confusion or even much of a learning curve.  They know their place, their job, and they do it.

Failures of Crowd Intelligence

James Surowiecki argues that there are also failures to crowd intelligence:

• Too centralized: The Columbia shuttle disaster, which he blames on a hierarchical NASA management bureaucracy that was totally closed to the wisdom of low-level engineers.

• Too divided: The U.S. Intelligence community failed to prevent the September 11, 2001 attacks partly because information held by one subdivision was not accessible by another. Surowiecki’s argument is that crowds (of intelligence analysts in this case) work best when they choose for themselves what to work on and what information they need. (He cites the SARS-virus isolation as an example in which the free flow of data enabled laboratories around the world to coordinate research without a central point of control.)

• Too imitative: Where choices are visible and made in sequence, an “information cascade” can form in which only the first few decision makers gain anything by contemplating the choices available: once this has happened it is more efficient for everyone else to simply copy those around them.

The Democratization of the Web

I agree with James Surowiecki on his points on the failures of crowd intelligence.  I see this as evident in what has been called the ‘Democratization of the Web.’  The proliferation of blogs, which potentially allows everybody a voice, actually results in a few voices and less intelligence.  Let me explain: the web is incredibly imitative.  Look at Digg or it’s counterparts; Digg aims to display what is hot.  From a reader’s perspective, it is interesting to see what the pulse of the web is at any point in time.  But, what results from this is that readers become writers of what is hot — the readers who then write about what is hot at the moment perceive that they are adding to the conversation, their $0.02, if you will.  But, in reality, it is really imitative behavior.  In aggregate, the social web no longer makes independent decisions, but looks to others to inform their decision making and opinions.

This phenomena is what social scientist call an “Information Cascade“.  Once on this slippery slope, it is difficult to make independent decisions.  Contrast this to the open source community.  Open Source works very well and prides itself on being a free-thinking, independent community.  But, ultimately only one developer writes code that gets added to the kernel.  There may be collaboration and coordination via newsgroups and concurrent versioning systems, but ultimately the rolling-up of modifications to the kernel is controlled by a smaller group.  Community — yes; but, it’s also very centralized and it works really well — a balance seems to have been reached between centralization and community and, it works really well.

Digg isn’t De.licio.us

Digg, I believe results in less intelligence.  The crowd is imitative; there are many followers in this space.  De.licio.us, on the other hand, is the result of truly independent choices: it’s simply personal bookmarking, made public.  The assumption is that a human will bookmark what is useful to them; making that bookmark public allows others to benefit from your discovery.  In contrast, Digg is very supportive of crowd psychology, which can be very dumb, mimicky, and in the end, not all that helpful.

Web Governance?

No, I’m not arguing for Web Governance.  A true democracy, like the web has become, would not work in real societies.  The United States is a Republic, not a true democracy. The founding fathers were very nervous of a true democracy because they understood the failures of crowd wisdom.  In their minds, the crowd wasn’t all that intelligent.  In closing, we can benefit from the domocratization and the social web.  It’s entertaining and participation can be useful.  The social web is also very trendy and can result is a less intelligent web (including this blog).

[sources: wiki, roush, surowiecki]