Read our latest articles for tips, tricks, advice and news about research techniques and solutions.
About a month ago, we published After the Bot Scare blog on workers providing bad quality data on Amazon's Mechanical Turk. This month, we followed up with our “farmers” to assess the effectiveness of the tools we created to deal with the problem. In this blog, we present data from our follow-up study and evidence to suggest our tools are working.
Read More
A Case Study From a Recent JESP Article A new study appearing in the Journal of Experimental Social Psychology suggests Americans strongly believe in economic mobility because they fail to appreciate how vast wealth inequality really is. In this blog,...
Read More
By now, even casual users of MTurk have heard about recent concerns of “bots” or low quality data. We've written about the topic here and laid out evidence that suggests “bots” are actually foreign workers using tools to obscure their true location (here). Perhaps most importantly, we've created two tools to help keep these workers out of your studies. In this blog, we introduce a third tool: the Universal Exclude List.
Read More
In this blog, we review recent concerns about bots, along with general data quality issues, on Mechanical Turk and report the results of a study we conducted to investigate the problem.
Read More
Last week, the research community was struck with concern that “bots” were contaminating data collection on Amazon's Mechanical Turk (MTurk). We wrote about the issue and conducted our own preliminary investigation into the problem using the CloudResearch database. In this blog, we introduce two new tools CloudResearch is launching to help researchers combat suspicious activity on MTurk and reiterate some of the important takeaways from this conversation so far.
Read More
When researchers collect data online, it's natural to be concerned about data quality. Participants aren't in the lab, so researchers can't see who is taking their survey, what those participants are doing while answering questions, or whether participants are who they say they are. Not knowing is unsettling.
Read More
CloudResearch is announcing a change in our pricing for the MicroBatch feature. MicroBatch is now included as a Pro feature, with a fee of 2 cents + 5% per complete. This will also provide users with access to all other...
Read More
When researchers learn about conducting research online, it can sometimes be difficult to understand quite how all the tools that are available can actually be applied to a project. This post is about how specific research ideas can be carried...
Read More Some workers on MTurk are extremely active, and take the majority of posted HITs. This can lead to many issues, some of which are outlined in our previous post. Although MTurk has over 100,000 workers who take surveys each year,...
Read More It is important to consider how many highly experienced workers there are on Mechanical Turk. As discussed in previous posts, there is a population pool of active workers in the thousands, but this is far from exhaustible. A small group...
Read More
