Etiqueta: analysis

The crowdsourcing dilemma: the idea with the most votes isn’t always the best idea

You might wonder, with great anti-biasing technology, why wouldn’t the idea with the most votes always be the best idea? There are all sorts of reasons: a great idea might have been entered relatively late in the crowdsourcing process or the submitter might have given it a non-compelling title, for example. But by looking at implicit data, as well as explicit data, (that is looking at how the crowd interacts with ideas and not just at the hard data like votes), you can identify other indicators for ideas that are truly merit worthy, despite not getting the most votes, or even a lot of votes. You may not be able to immediately tell if the “underdog” idea is in fact a better idea, but you can provide it with more visibility within the crowd so that you can do an apples-to-apples comparison with the big vote getting ideas.

Here are some of the things we do, and suggest others do, to ensure a reliable, accurate outcome, and avoid the “popularity contest” syndrome:

* Multiple idea order display – Display ideas in a variety of ways, such as most recent, most discussed and most active for example, and don’t just default to listing the top voted ideas.
* Zero-start finalist round – Use a finalist round to allow the entire crowd to focus on just a few ideas which all show signs of being superior ideas, and start all finalists at zero votes.
* Weighted voting – give insider experts, your panel, or more long-time active members more vote weight… you’ll find these people are highly motivated to filter the best not just the popular to the top


Randy Corke| http://www.chaordix.com/blog

One of the common complaints about crowdsourcing is that it can become a popularity contest: the idea that gets the most early votes rises to the top of the list, therefore gets more views, and therefore more votes and becomes the winner. And, unfortunately, for many so-called “crowdsourcing” sites, this is true. You see it on sites like Digg – get enough early “diggs” for your submission to get on the “top news” list and your submission can get visibility for a long time.

We work hard to surface the best quality results for our clients from their crowdsourcing projects, so as you would expect, we have developed ways to avoid this “early vote” bias and other forms of bias. But even with great design and planning, the best technology and the right methodology, you can’t completely eliminate the possibility of a less-worthy idea getting the most votes. However, it IS possible to use analysis and crowd management techniques to ensure that other highly worthy ideas can be identified, so that the chances of truly finding the best idea are maximized.

Leer más “The crowdsourcing dilemma: the idea with the most votes isn’t always the best idea”

Making sense of crowdsourced data

Turns out crowdsourcing has a lot in common with information governance.

This week at a beautiful desert oasis in New Mexico, Chaordix participated on IBM’s Data Governance Council Forum to work on strengthening and modernizing the Maturity Model that sets out best practices for information governance that distinguish leading organizations. Among an elite of IT, governance and finance professionals for some of the world’s largest organizations in health, banking, utility sector and the US Army, I began feeling an outsider to the scope and scale of information management challenges that these organizations face.

But when asked what they wanted to talk about, it turned out their top 3 interests were the same as top concerns enterprises have about data that is…crowdsourced!


Claudia Moore | http://www.chaordix.com

Is it true? Do I understand it? Is it data I can apply to perform better, right now?

Turns out crowdsourcing has a lot in common with information governance.

This week at a beautiful desert oasis in New Mexico, Chaordix participated on IBM’s Data Governance Council Forum to work on strengthening and modernizing the Maturity Model that sets out best practices for information governance that distinguish leading organizations. Among an elite of IT, governance and finance professionals for some of the world’s largest organizations in health, banking, utility sector and the US Army, I began feeling an outsider to the scope and scale of information management challenges that these organizations face.

But when asked what they wanted to talk about, it turned out their top 3 interests were the same as top concerns enterprises have about data that is…crowdsourced! Leer más “Making sense of crowdsourced data”