Urenio Watch Watch: Collaborative Innovation

Evaluation of Different Approaches to Brainstorming

Researchers from the INSEAD and Wharton business schools published a study, Idea Generation and the Quality of the Best Idea, that analyzes the process of generating and selecting ideas.

In a wide variety of settings, organizations generate a number of possible solutions to a problem’”ideas’” and then select a few for further development. The researchers examine the effectiveness of two group structures for such tasks’”the team structure, in which the group works together in time and space, and the hybrid structure, in which individuals first work independently and then work together. They define the performance of a group as the quality of the best ideas identified. Prior research has defined performance as the average quality of ideas or the number of ideas generated, ignoring what most organizations seek, a few great ideas.

The researchers build a theory that relates organizational phenomena to four different variables that govern the quality of the best ideas identified: (1) the average quality of ideas generated, (2) the number of ideas generated, (3) the variance in the quality of ideas generated, and (4) the ability of the group to discern the quality of the ideas. They test this theory with an experiment. They find that groups organized in the hybrid structure are able to generate more ideas, to generate better ideas, and to better discern the quality of the ideas they generate. Moreover, they find that the frequently recommended brainstorming technique of building on others’ ideas is counterproductive; teams exhibiting such build-up neither create more ideas, nor are the ideas that build on previous ideas better.

The report based on a laboratory experiment, which compares the two group structures with respect to each of these four variables individually, and which measures their collective impact on the quality of the best idea. An accurate measurement of idea quality is central to the study. While most prior research has relied on the subjective evaluation of idea quality by one or two research assistants, the INSEAD and Wharton researchers use two alternative approaches: a web-based quality evaluation tool which collects about 20 ratings per idea and a purchase-intent survey which captures about 40 consumer opinions about their intent to purchase a product based on the idea. Their framework, with its emphasis on the importance of the best idea, and their novel experimental set-up let them make the following three contributions:

  1. They find evidence that the best idea generated in the hybrid structure is better than the best idea generated by a team. This result is driven by the fact that the hybrid structure results in about three times as many ideas per unit of time and that these ideas have significantly higher average quality.
  2. They find that the hybrid structure is better at identifying the best ideas from the set of ideas it previously generated. However, They also find that both team and hybrid structures are, in absolute terms, weak in their ability to assess the quality of ideas.
  3. They show that idea generation in teams is more likely to lead to ideas that build on each other. However, in contrast to the common wisdom articulated by many proponents of team brainstorming, they show that such build-up does not lead to better idea quality. In fact, they find that ideas that build on a previous idea are worse not better, on average.