Many researchers set up their studies to use the TurkPrime AutoApprove feature so that they do not need to manually approve worker assignments based on the secret code that workers enter. On occasion, a researcher may set up his study incorrectly which results in many worker assignments getting automatically rejected. This was a significant cause for distress among the Mechanical Turk workers who received rejections for their work which then went unpaid and also lowered their MTurk approval rating. This was also a sore point for TurkPrimeresearcherswho had to deal with upset workers and correspond with them, and often reverse their rejections.
To alleviate this issue, TurkPrime will give researchers greater control of the rejection process when Auto Approve is used:
- Only worker assignments that have the correct secret codes will be auto-approved while workers with invalid secret codes will require manual rejection. (If a rejection is made in error by a researcher, it may still be reversed.) This manual rejection is made using the same interface as is available for non-AutoApproved workers as shown below.
- Researchers will see the secret codes the workers entered and whether they were correct. they will then have the option to approve, reject or leave their status undecided.
- The manual rejection process must be completed within the Amazon Approval window (default of 7 days). Otherwise, Mechanical Turk will automatically approve the pending assignments that were not resolved buy that time.
