Open-ended questions: How the voting process works

Open-ended questions: How the voting process works

What is the voting process?

Participants will have a specific amount of time to respond to open-ended questions, which is set by the moderator. The baseline that we recommend is 2 minutes.

After a participant submits their response to an open-ended question, they are prompted to vote on pairs of other participants' responses until the time for the question runs out. This voting exercise determines the popularity and consensus score shown on each response in the rank tab.

How the voting process works

The voting process is used to understand a group efficiently and at scale. Because it isn't efficient for every participant to vote on every response submitted, the voting process enables us to infer and calculate a score representing the group's opinion of every response. This is done using Bayesian inference and statistical modeling. The more a participant votes, the more the Remesh platform learns about them and can more accurately make predictions about their behaviors and opinions.

Q: Does every participant see the same pairs of response?

A: No. Each participant see responses that are selected strategically to understand their opinion.

Q: How does the algorithm decide which pairs of responses to show to each participant?

A: The algorithm is optimized for learning as much new information from the choice made between each new pair shown. In order to optimize for that, it takes the following into consideration:

  • Responses submitted after the voting exercise has begun and have not yet been voted on
  • Responses that the algorithm is still uncertain on how the participant would vote
  • Responses that seem to have a similar level of popularity and more voting is needed to determine if one response is truly liked more than the other

Q: I never was given the option to vote on the responses that were ranked most popular, why is that?

A: Your voting pattern is used by the algorithm to infer how you would have voted on a pair of responses that you did not vote on. This means that you won't see all of the responses and you may not see the most popular.

For example, imagine a Remesh conversation about politics. If a participant always votes conservative for every election choice and then when given a choice between a Somewhat Liberal, Somewhat Conservative, and Very Conservative candidate for the current election, the participant chooses the Very Conservative candidate. Based on the history of votes and the current vote, there is a strong prediction that given the choice between a Very Liberal (a not shown candidate) and a Very Conservative candidate, the person would still select the Very Conservative candidate. So while the system cannot show all responses, it makes sure to show the pairs of the responses that would provide the most information gain to understand the opinion of each individual person.The end result is a ranked list of responses that have statistical scores associated with them, called Popularity and Consensus.


How did we do?