Whether you’re new to the field of online research or looking to improve your skills on new platforms, we’re here to help. Our guide will you walk you through the basic concepts and terminology behind data quality, teach you to effectively evaluate and groom your own data sets and give you a practical introduction to implementing data quality processes at your company.
If you were a researcher studying human behavior 30 years ago, your options for identifying participants for your studies were limited. If you worked at a university, you might be able to rely on a student subject pool, or, with a lot of legwork, identify people in the community to participate in your studies. If you worked in the marketing industry, your company might conduct a focus group or hire an outside firm to conduct a phone survey, a mail survey or an in-person study with your target audience. Either way, the options for finding participants were slow, costly and restricted.
As a researcher, you are aware that planning studies, designing materials and collecting data each take a lot of work. So when you get your hands on a new dataset, the first thing you want to do is start testing your ideas. Were your hypotheses supported? Which marketing campaign should you launch? How do consumers feel about your new product? These are the types of questions you want answered. But before you can draw conclusions from your dataset, you need to inspect and clean it, which entails identifying and removing problem participants.
Technology has transformed behavioral science research. Researchers today can quickly access participants from all over the world and collect data in ways not possible in the past. Key to this transformation has been online participant recruitment platforms like Mechanical Turk (MTurk) and market research panels. Although these panels offer the opportunity to conduct research quickly and efficiently, they also pose unique opportunities and challenges for managing data quality. Not all platforms are built the same way or are equally valid for different types of research. As we will see, finding participants “fit for purpose” for a research study is a big part of managing data quality in online studies.
Collecting quality data begins with selecting the right participants. A phrase common in many research circles is that participant selection should be “fit for purpose.” This means the participants a researcher selects for a study should be well suited for finding answers to the questions that motivated the study.