Blog

Read our latest articles for tips, tricks, advice and news about research techniques and solutions.

Exclude Workers With One Click
Jonathan Robinson, PhD

Exclude Workers With One Click

Problem: Suppose you’re running a Mechanical Turk survey and need to exclude workers who took a previous survey. How can you quickly set this up. Some of the currently used solutions require following multiple steps to set things up and...

Read More >
Longitudinal and Follow Up Surveys on Mechanical Turk
Jonathan Robinson, PhD

Longitudinal and Follow Up Surveys on Mechanical Turk

Problem: Suppose you need to run a survey on Mechanical Turk and follow up with the same workers a week, month or year later. A few issues come up: How can I limit the follow up surveys to survey takers...

Read More >
Finding More Mechanical Turk Workers Faster with CloudResearch Restart HIT
Jonathan Robinson, PhD

Finding More Mechanical Turk Workers Faster with CloudResearch Restart HIT

Problem: Suppose you need to run a HIT with 1000 Workers. Or a HIT that is only open to Workers who have an approval Rating of 95% or more and have completed 500 HITs or more. Although when you launched...

Read More >
Creating Mechanical Turk Custom Panels With CloudResearch Worker Groups
Jonathan Robinson, PhD

Creating Mechanical Turk Custom Panels With CloudResearch Worker Groups

Problem: Suppose you need to run a group of HITs open only to participants who are women under 50. You previously ran a HIT and know the Worker IDs that you want to reach, but have no way to email...

Read More >
IRB Template for Mechanical Turk and TurkPrime
Leib Litman, PhD

IRB Template for Mechanical Turk and TurkPrime

An IRB will generally request a description of how participants will be recruited, reimbursed and interacted with. Additionally IRBs always request information about how anonymity of the participants is protected. Members of the IRB board may not be familiar with Amazon Turk, and it may be helpful to include a brief description of MTurk in your IRB application.

Read More >
The New New Demographics on Mechanical Turk: Is there Still a Gender Gap?
Leib Litman, PhD

The New New Demographics on Mechanical Turk: Is there Still a Gender Gap?

In this blog, we report on the demographics of Mechanical Turk workers by reviewing 75 studies conducted with US-based Workers in 2013 and 2014. From a total of 32,595 Workers, we found 15,324 were female (47%).

Read More >
A Simple Formula for Predicting the Time to Complete a Study on Mechanical Turk
CloudResearch

A Simple Formula for Predicting the Time to Complete a Study on Mechanical Turk

We describe a general formula for predicting the time it takes Workers to complete survey studies on MTurk. The average Worker takes 10.3 seconds to answer a single question. This means that a study with 60 questions should take approximately 10 minutes.

Read More >
How to Minimize Dropout Rate on Mechanical Turk
Leib Litman, PhD

How to Minimize Dropout Rate on Mechanical Turk

It is generally thought that pay rate does not affect data quality on Mechanical Turk. For example (Buhrmester, Kwang, & Gosling, 2011) showed that whether Workers are paid 5 cents or one dollar for a survey study, the internal reliability of the surveys does not change. They did show however that fewer Workers will take the surveys that pay less. We recently replicated these findings for both US and India-based Workers (Litman et al, 2014).

Read More >
Determining Completion Rate and Dropout Rate on Mechanical Turk
Leib Litman, PhD

Determining Completion Rate and Dropout Rate on Mechanical Turk

What is the completion rate and dropout rate? Dropout rate is defined as the percentage of participants who start taking a study but do not complete it. Dropout rate is sometimes referred to as attrition rate, and is the opposite...

Read More >
Why Use CloudResearch Panels?
Leib Litman, PhD

Why Use CloudResearch Panels?

MTurk Requesters are often interested in studying specific groups of people. For example, a researcher may be interested in men over 40, Republicans, people who are concerned about the cleanliness of sponges, or cancer survivors. CloudResearch Panels utilizes various techniques...

Read More >

SUBSCRIBE TO RECEIVE UPDATES

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.