7.7 Prediction Markets and the Wisdom of Crowds

Many social software efforts leverage what has come to be known as the wisdom of crowds. In this concept, a group of individuals (the crowd often consists mostly of untrained amateurs), collectively has more insight than a single or small group of trained professionals. Made popular by author James Surowiecki (whose best-selling book was named after the phenomenon), the idea of crowd wisdom is at the heart of wikis, folksonomy tagging systems, and many other online efforts. An article in the journal Nature positively comparing Wikipedia to Encyclopedia Britannica lent credence to social software’s use in harnessing and distilling crowd wisdom (Giles, 2005).

The crowd isn’t always right, but in many cases where topics are complex, problems are large, and outcomes are uncertain, a large, diverse group may bring collective insight to problem solving that one smart guy or a professional committee lacks. One technique for leveraging the wisdom of crowds is a prediction market, where a diverse crowd is polled and opinions aggregated to form a forecast of an eventual outcome. The concept is not new. The stock market is arguably a prediction market, with a stock price representing collective assessment of the discounted value of a firm’s future earnings. But Internet technologies are allowing companies to set up prediction markets for exploring all sorts of problems.

Consider Best Buy, where employees are encouraged to leverage the firm’s TagTrade prediction market to make forecasts, and are offered small gifts as incentives for participation. The idea behind this incentive program is simple: the “blue shirts” (Best Buy employees) are closest to customers. They see traffic patterns and buying cycles, can witness customer reactions first hand, and often have a degree of field insight not available to senior managers at the company’s Minneapolis headquarters. Harness this collective input and you’ve got a group brain where, as wisdom of crowds proponents often put it, “the we is greater than the me.” When Best Buy asked its employees to predict gift card sales, the “crowd’s” collective average answer was 99.5 percent accurate; experts paid to make the prediction were off by 5 percent. Another experiment predicting holiday sales was off by only 1/10 of 1 percent. The experts? Off by 7 percent (Dvorak, 2008; Dye, 2008)!

In an article in the McKinsey Quarterly, Surowiecki outlined several criteria necessary for a crowd to be “smart” (Dye, 2008). The crowd must

• be diverse, so that participants are bringing different pieces of information to the table,
• be decentralized, so that no one at the top is dictating the crowd’s answer,
• offer a collective verdict that summarizes participant opinions,
• be independent, so that each focuses on information rather than the opinions of others.

Google, which runs several predictive markets, underscored these principles when it found that predictions were less accurate when users were geographically proximate, meaning folks in the same work group who sat near one another typically thought too much alike (Cowgill, et. al., 2009). Poorer predictive outcomes likely resulted because these relatively homogeneous clusters of users brought the same information to the table (yet another reason why organizations should hire and cultivate diverse teams).

Many firms run predictive markets to aid in key forecasts, and with the potential for real financial payoff. But University of Chicago law professor Todd Henderson warns predictive markets may also hold legal and ethical challenges. The Securities and Exchange Commission may look askance at an employee who gets a heads-up in a predictive market that says a certain drug is going to be approved or fail clinical trials. If she trades on this information is she an insider, subject to prosecution for exploiting proprietary data? Disclosure issues are unclear. Gambling laws are also murky, with Henderson uncertain as to whether certain predictive markets will be viewed as an unregulated form of betting (Dye, 2008).

Publicly accessible prediction markets are diverse in their focus. The Iowa Electronic Market attempts to guess the outcome of political campaigns, with mixed results. Farecast (now part of Microsoft’s Bing knowledge engine) claims a 75 percent accuracy rate for forecasting the future price of airline tickets¹. The Hollywood Stock Exchange allows participants to buy and sell prediction shares of movies, actors, directors, and film-related options. The exchange, now owned by investment firm Cantor Fitzgerald, has picked Oscar winners with 90 percent accuracy (Surowiecki, 2007). And at HedgeStreet.com, participants can make microbets, wagering as little as ten dollars on the outcome of economic events, including predictions on the prices of homes, gold, foreign currencies, oil, and even the economic impact of hurricanes and tropical storms. HedgeStreet is considered a market and is subject to oversight by the Commodity Futures Trading Commission (Lambert, 2006).

¹“Audit Reveals Farecast Predictive Accuracy at 74.5 percent,” farecast.live.com, May 18, 2007, http://www.prnewswire.com/news-releases/farecast-launches-new-tools-to -help-savvy-travelers-catch-elusive-airfare-price-drops-this-summer-58165652.html.

References

Cowgill, B., J. Wolfers, and E. Zitzewitz, “Using Prediction Markets to Track Information Flows: Evidence from Google,” working paper accessed November 30, 2009, via http://bocowgill.com/GooglePredictionMarketPaper.pdf.

Dvorak, P., “Best Buy Taps ‘Prediction Market,’” Wall Street Journal, September 16, 2008.

Dye, R., “The Promise of Prediction Markets: A Roundtable,” McKinsey Quarterly (2008): 83–93.

Giles, J., “Special Report: Internet Encyclopedias Go Head to Head,” Nature 438, no. 15 (December 14, 2005): 900–901.

Lambert, E., “Hedging for Dummies,” Forbes, March 13, 2006, 70–72.

Surowiecki, J., “Crowdsourcing the Crystal Ball,” Forbes, October 15, 2007.