Scientists Get Meta, Asking Study Subjects to Design a Study

Rather than just ask the crowd to help answer math problems or scientific ponderings, why not challenge the crowd to design the questions themselves?

Gleaning a bit of insight from “the wisdom of the crowd” is all the rage these days. Crowds help with problems as wide ranging as predicting Olympic victories to analyzing gene regulatory networks, and handfulls of TED talks dote on the topic.

A group of researchers recently took this idea a step further: Rather than just ask the crowd to help answer math problems or solve scientific ponderings, why not appeal to the crowd to design the questions themselves? In other words, would it be useful to crowdsource an entire scientific study, from question to hypothesis to answer?

To find out, University of Vermont researchers set out to discover if volunteers who visited two different websites could pose, refine, and answer questions of each other. Specifically, they wanted the volunteers to figure out a way to predict each other’s body weight and home electricity use. Sure enough, the researcher’s computer models soon caught on to the crowdsourced questions and answers and successfully started to predict a user’s monthly electricity consumption and BMI.

Some questions the crowd came up with were obvious, like “Do you think of yourself as overweight?” And no surprise, that question proved to be the the most accurate at predicting a person’s body weight.

Other questions, however, were a bit more outside of the box. “How often do you masturbate a month?” took the researchers by surprise, for example, and they were even more titillated to find that the question proved to be the second-most predictive for projecting volunteers’ weights – even more so than “How often do you eat in a day?”

“Sometimes the general public has intuition about stuff that experts miss,” the researchers say.

The work shows that crowds can generate hypotheses, though  they point out that their study does not examine the causes of those variables – BMI and electric use – under scrutiny.

“Going forward, this approach may allow us to involve the public in deciding what it is that is interesting to study,” they say. “It’s potentially a new way to do science.”

More from

Can Crowdsourcing Really Spark Innovation? 

Gandhi’s Wisdom Falls Short 


Get the latest stories in your inbox every weekday.