In 1906, Francis Galton was at a country fair where attendees had the opportunity to guess the weight of a dead ox. Galton took the guesses of 787 fair-goers and found that the average guess was only one pound off of the correct weight — even when individual guesses were off base.
This concept, known as “the wisdoms of crowds” or “collective intelligence,” has been applied to many situations over the past century, from people estimating the number of jellybeans in a jar to predicting the winners of major sporting events — often with high rates of success. Whatever the problem, the average answer of the crowd seems to be an accurate solution.
But does this also apply to knowledge about systems, such as ecosystems, health care, or cities? Do we always need in-depth scientific inquiries to describe and manage them — or could we leverage crowds?
This question has fascinated Antonie J. Jetter, associate professor of Engineering and Technology Management for many years. Now, there’s an answer. A recent study, which was co-authored by Jetter and published in Nature Sustainability, shows that diverse crowds of local natural resource stakeholders can collectively produce complex environmental models very similar to those of trained experts.
For this study, about 250 anglers, water guards and board members of German fishing clubs were asked to draw connections showing how ecological relationships influence the pike stock from the perspective of the anglers and how factors like nutrients and fishing pressures help determine the number of pike in a freshwater lake ecosystem. The individuals’ drawings — or their so-called mental models — were then mathematically combined into a collective model representing their averaged understanding of the ecosystem and compared with the best scientific knowledge on the same subject.
The result is astonishing. If you combine the ideas from many individual anglers by averaging their mental models, the final outcomes correspond more or less exactly to the scientific knowledge of pike ecology — local knowledge of stakeholders produces results that are in no way inferior to lengthy and expensive scientific studies.
“The result gets better the more anglers are involved in the collective solution,” Jetter explained, “but it is not a simple vote count.”
The study found that it is important that the opinions of different types of anglers — recreational anglers, fisheries managers or board members of fishing associations — are preserved and taken into account separately. The knowledge base of only a single group of anglers may have group-specific biases that can accumulate to a possibly wrong solution.
“I am excited about the possibilities for other complex systems,” Jetter said. “We now understand how we can investigate problems like improving schools or increasing ridership in public transportation — we ask people who frequently interact with these systems and merge their system descriptions. This has huge potential for making cities smarter.”
The most exciting and rewarding aspect of this study, however, is how it came together. The study was lead by Payam Aminpour, who is presently a Ph.D. student at Michigan State University. “It is truly a story of interdisciplinarity, collaboration, persistence, and serendipity that stretches over years and continents,” Jetter said.
For PSU, it started when Jetter met an ecologist, Steven Gray, at a conference. They collaborated on two grants under Dr. Jetter’s leadership and frequently discussed how to obtain system descriptions to scale. In 2016, it finally clicked: connecting ideas from two earlier papers, including one by Jetter, they proposed leveraging the wisdom of crowds and drafted initial ideas for how to make it work. However, they still needed a data set — and an expert in quantitative analysis.
Data came in the form of an almost forgotten 2010 study by Gray’s collaborator, fishery scientist Robert Arlinghaus at the Leibniz Institute for Freshwater Ecology in Germany. And the expert analyst and scientific lead of the study turned out to be Ph.D. student Payam Aminpour, a civil engineer by training, who had joined Gray’s team at the Department of Community Sustainability at MSU and closely collaborated with Jetter on PSU-led grants that made the collaboration possible. Additional contributors were Joshua Introne and Alison Singer, former MSU affiliated scholars.
“About a hundred years after the introduction of the wisdom of crowds phenomenon by Francis Galton, we expand this theory and show empirically that averaging judgments from large crowds can also work for the cognitively more demanding task of describing a complex system,” Jetter said. “Maybe this will make our cities smarter, one day. If it does, it is a testament to interdisciplinary research.”
