Although the idea may be simple, writing effective screening questions is a lot more complicated. Screening questions need to be short and easy for respondents to answer. They should avoid binary response options but also should not include too many response options. You should order the response options in the right way, and the overall screener shouldn’t contain too many questions. Above all, you absolutely need to ensure you don’t lead people to guess which attributes you’re looking for. To balance all these objectives—and to demonstrate examples of good and bad survey screening questions—we wrote this blog. Let’s dig in.
People say summer is the time to go on vacation, relax, and maybe even read some books for fun. However, I decided to do a full-time internship at CloudResearch during the day, while continuing to work on my PhD research in the evenings and weekends. It’s been hard, but totally worth it! My experience so far has been very different from grad school, and it’s helping me make an informed decision about staying in academia vs. going into industry. There’s a lot I love about academia, and someday I’ll write a blog post about my reasons for wanting to pursue a tenure-track position. But for now, here are my top three reasons for liking industry
At CloudResearch, we’re all about innovation. From tackling the Bot Scare to exploring gender pay gaps to examining the ethics of MTurk, we know what it takes to keep up with best practices in the everchanging field of behavioral research. To further support the work of conducting high-quality, innovative research online, we are thrilled to announce two exciting new opportunities: our inaugural conference, and our CloudResearch Grant Award!
At the heart of good research methods are two concepts known as survey validity and reliability. Digging into these concepts can get a bit wonky. However, understanding validity and reliability is important for both the people who conduct and consume research. Thus, we lay out the details of both constructs in this blog.
Pollsters have known for decades that people are increasingly unwilling to participate in polls and about the problems this can create. In 2016, for example, White voters without a college degree were especially unlikely to participate in polls. The result was that many forecasts seriously underestimated support for President Trump. Even though pollsters thought they fixed this problem heading into 2020, polls in 2020 also underestimated support for President Trump, likely because voters low in social trust avoided the polls. As the last two Presidential cycles show, correcting for one form of non-response bias doesn’t necessarily protect polls from other forms of systematic non-response.
Last month, people in the U.S. marked the one year anniversary of the COVID-19 pandemic. For most of 2020, circumstances on the ground were changing faster than many forms of data could catch up with, and there was fear this may be true with the demographic composition of people on Amazon Mechanical Turk. So, after one year, we decided to look at whether there were any major changes to the demographics of people on MTurk as a result of the pandemic.
