Popularity and Consensus Scores

Popularity and Consensus Scores

Every response to an open-ended question has a popularity score and a consensus score.

These scores are calculated based on the how the group voted on the response during the binary comparison voting exercise, and the scores tell you how well a particular answer represents the group. The higher the scores are, the better the response represents the group.

What is Popularity?

Popularity represents the average opinion of the group. The score is based on an estimate of the percentage of time the given idea was chosen over its competing ideas during the head-to-head voting activity

What is Consensus?

Consensus represents group agreement or what is driving the Popularity score. For example, if 10% of respondents chose an idea 100% of the time, that signifies low consensus (as opposed to whether 50% of participants chose the idea 80% of the time or 100% of participants chose an idea 10% of the time).

What makes Popularity different from Consensus?

Popularity can be viewed as an estimate on how often a response was selected - driven solely by votes; whereas, consensus represents the distribution of how often an item was selected across the entire sample. You will also see color coding based on Favorability. The rankings that you see when analyzing the data on the Rank screen are based solely on the popularity scores. In contrast, the favorability classification is based on the intersection of popularity and consensus. So if Answer A has a higher popularity score but a lower Favorability rating than Answer B, Answer A will appear higher in the rankings. Please refer to the chart below to see a breakdown in Favorability classifications!


How did we do?