AI-Survey Tools for Qualitative and Quantitative Research

Introduction

What if you had a close friend or family member who fell into a conspiracy theory? Perhaps they believed the government was hiding evidence of aliens, that a secret cabal controlled world events, or that a popular health practice was harmful. Every time you shared contradicting evidence, you were met with resistance or dismissal.

How could you change this person's mind? Would it be best to ask questions and try to understand their perspective? Should you double down on facts? Should you share personal stories? What might persuade the person you care about to change their mind?

The challenge of persuading someone to change a deeply held but potentially harmful belief is a common experience. For that reason, some researchers have recently wondered: can artificial intelligence offer a new approach to this problem?

In 2023, researchers from MIT and several other universities collaborated on a fascinating study. Their research examined whether personalized conversations with an AI interviewer could reduce people's beliefs in conspiracy theories. The research team investigated this question through an online study involving over 2,100 participants who had strong beliefs in various conspiracy theories (Costello et al., 2025).

In the experiment, participants were asked to describe a conspiracy theory they found compelling and to explain their reasoning. Then, they engaged in a back-and-forth dialogue with an AI interviewer that was instructed to provide counterarguments and evidence challenging the person's beliefs. The results were impressive. The AI conversations reduced conspiracy beliefs by approximately 20% compared to a control group, and these effects persisted even when measured two months later. Even participants with deeply entrenched beliefs showed significant reductions in their conspiracy thinking after these personalized AI interactions.

Not long ago, this study would have been impossible. Having thousands of people engage in personalized conversations would have presented overwhelming logistical challenges. But today, using online participant recruitment platforms combined with AI-based survey tools, researchers can conduct personalized interviews on a massive scale.

In this chapter, we will explore how AI is transforming survey research by facilitating qualitative and mixed methods studies. In Module 8.1, we will learn about the fundamental changes AI is bringing to traditional survey methods. We will see how AI allows a dynamic, conversational approach to survey design that can elicit deeper insights than quantitative research, and how this capability expands the toolkit available to behavioral scientists.

In Module 8.2, we will explore a guided research project using the Heinz Dilemma to demonstrate how AI interviews can effectively combine qualitative and quantitative methods. We will walk through the practical aspects of setting up an AI-interview, providing guidance to AI systems, analyzing the resulting data, and using AI tools for mixed methods data analyses. This module will show how researchers can leverage AI to replicate the kind of in-depth qualitative research that traditionally required extensive human resources. Understanding these emerging technologies will likely be essential for any researcher looking to conduct innovative behavioral science in the coming years.

Chapter Outline

Artificial Intelligence (AI) is transforming many aspects of human life, from how people navigate traffic to how they discover music. In the social and behavioral sciences, AI is opening new possibilities, too. In Chapter 2, we learned how ChatGPT's Data Analyst can conduct the statistical analyses covered in this book. In Chapter 4, we explored AI's ability to help create survey instruments. But among the areas where AI is having a particularly dramatic impact is in expanding the capabilities of survey platforms.

Traditional survey platforms are generally passive. They present participants with questions or statements and provide a fixed set of response choices. Yet with AI the research experience can be more dynamic and interactive (e.g., Xiao et al., 2020; Chopra and Haaland, 2023; Wuttke et al., 2024).

At its core, AI refers to computer systems that perform tasks that typically require human intelligence. These systems can understand human language, engage in conversations, and ask follow-up questions, much like a human would. The ability to have flexible conversations with respondents opens many possibilities to expand research methodology. Let's look at an example.

Imagine we are conducting a study to understand why people drink alcohol. A typical quantitative study using the approaches we have learned about in previous chapters might ask respondents: "Why do you drink alcohol?" Participants would then be presented with a predetermined list of answer options (Figure 8.1) and asked to check all the reasons that apply from options like "to relax," "to socialize," and "to cope with stress."

How would the researcher come up with the response options in the first place? They could use previous research on the topic, or they could generate reasons based on experience and logic. This type of research is often called confirmatory because it seeks to confirm and to quantify an existing set of hypotheses about the behavior in question. By design, confirmatory research is constrained to categories defined by the researcher.

What if, however, the goal of the research was to identify novel reasons for drinking alcohol? Or, what if the research on a topic is sparse and there is little existing literature to lean on? In such cases, researchers can adopt an exploratory approach. Instead of presenting predetermined answer options, a researcher can conduct a study that engages respondents in a natural conversation. Respondents can then describe their reasons for drinking alcohol in a setting where they are free to say whatever they want.

As shown in Figure 8.2, the conversation might begin with a simple question about whether the person drinks alcohol. Then, it can explore reasons for drinking or abstaining. When a respondent mentions drinking "when out with friends," the AI can inquire further about how alcohol functions in social situations. This conversation continues, with the AI asking follow-up questions based on the content of each response, allowing respondents to express themselves in their own words.

The example in Figure 8.2 is a simplified case. To illustrate how an actual AI survey can conduct a project of this nature, we gathered responses from 100 people on Connect and asked them about their alcohol consumption. The AI-powered platform we used was Engage. As mentioned in previous chapters, Engage functions like Qualtrics and other survey platforms, but it also has AI interviewing capabilities.

The Engage interviewer had a conversation with each respondent about their reasons for drinking alcohol and encouraged elaboration without leading respondents toward any particular answer. You can explore the study, all one hundred conversations, and the results, by following this link to the Engage platform: https://bit.ly/43dyOVt. Once there, login with the CloudResearch credentials you have used previously or create a new account.

After completing the interviews, the AI analyzed the text data and listed the reasons participants provided for drinking. Figure 8.3 shows the eleven reasons Engage identified from the transcripts.

The analysis revealed a variety of reasons for drinking. These included social facilitation, where people described alcohol as a "social lubricant" that helps them interact with others (51%); drinking simply to enjoy the experience (47%), coping with stress (43%), and drinking to relax (40%).

Some, perhaps, less obvious motivations emerged as well. Thirteen percent of people mentioned drinking to enhance experiences like food and music, while 11% noted drinking to enhance the flavor of meals. The least common motivation, mentioned by 4% of people, was feeling obligated to drink despite having no desire to do so.

Of course, people's reasons for drinking alcohol in this sample are not comprehensive. However, when conducted with thousands of participants and across numerous demographic, cultural, and globally diverse groups, this kind of research can paint a rich tapestry of the reasons people have for drinking alcohol, and it can do so faster and with less effort than perhaps any other method. Such research can have numerous practical applications.

With enough respondents in the sample, it becomes possible to examine how motivations for drinking differ among groups, such as men and women, teenagers and adults, and people in different socio-economic groups. A detailed analysis of people's reasons for drinking could inform public health campaigns, which might tailor their message to different people depending on their motivations. It might also help companies develop and market products that better align with consumers' motivations.

Overall, this example demonstrates how AI-powered conversational surveys can enhance the study of human behavior and how they differ from the traditional quantitative approach (Figure 8.4). By adopting a more flexible and exploratory method, conversational surveys allow participants to express their motivations in their own words through natural dialogue. The AI then identifies patterns and emergent categories from hundreds or even thousands of conversations in a matter of minutes, allowing researchers to uncover the "why" behind people's behavior in ways that are hard to do with more traditional methods.

Qualitative, Mixed Methods, and Quantitative Research

The study above is an example of qualitative research. While it used AI to interview people and analyze the data, traditional qualitative research is done by people and it is a common methodology across many domains of behavioral science. In fields such as anthropology and sociology, for instance, researchers often use qualitative methods to answer a wide range of research questions.

In sociology, many books have been written about various questions in human life based on qualitative interviews. In a classic best-selling book, The Seasons of a Man's Life (1978), Daniel Levinson conducted in-depth qualitative interviews with people of various ages. From the interviews, he discovered patterns in how adults develop and change throughout their lives. By analyzing people's stories, Levinson identified and described distinct periods in adult development, each with its own specific challenges.

Qualitative research has also produced several other fascinating books. In Hooking Up: Sex, Dating, and Relationships on Campus, Kathleen Bogle (2008) described the hookup and dating culture on contemporary college campuses. Elliot Liebow (1993) described the daily struggles of women who lived on the streets in Tell Them Who I Am: The Lives of Homeless Women. And, in The Tenants of East Harlem (2006), Russell Leigh Sharman describes the realities of changing inner city life in a New York City neighborhood.

Beyond interviewing people, qualitative research also has direct applications to quantitative research. For example, using a method called cognitive interviewing, qualitative interviews are used for questionnaire development. In a cognitive interview, a researcher asks people to "think aloud" as they answer specific questions within a questionnaire, sharing everything that goes through their minds. Such question development techniques are invaluable for validating survey questions and making sure people understand the questions as they were meant to be understood. Without such validation, participants can misinterpret survey questions. For example, in one study, people were asked, "How many times have you visited a doctor in the past year?" Many participants were unsure whether to count phone consultations or visits to nurse practitioners (Willis, 2004). Without the opportunity to clarify, the researchers might never have known their questions were being interpreted inconsistently.

The Role of AI in Qualitative and Mixed Methods Research

Despite the usefulness of qualitative research, it has traditionally been difficult to implement for several reasons. The first reason is the time and resources required to conduct open-ended interviews. A skilled researcher might complete three or four interviews in a day. This means a study with 30 participants, a typical sample size for qualitative research, would take two or three weeks just to collect the data.

The second reason involves making sense of the data. Imagine sitting down with 50 interviews, each filled with rich descriptions of people's experiences and thoughts. Researchers need to read through each interview multiple times, first to familiarize themselves with the content, then to identify patterns, and finally to carefully code each response. These analyses can take months.

The time required to conduct qualitative research explains why these studies typically have small sample sizes. Whereas a quantitative study might include hundreds or even thousands of participants, qualitative studies usually work with 20-30 participants (see Yin, 2016). And, even this small number generates an enormous amount of information to analyze.

This is where AI is changing qualitative research in two important ways. First, AI can conduct in-depth interviews with many participants simultaneously (see Austin et al, 2025; Costello et al., 2025). Like a skilled human interviewer, AI can ask follow-up questions when responses are brief or unclear, probe deeper when participants mention something interesting, and maintain a natural flow of conversation. This means researchers can conduct qualitative interviews with hundreds of participants in the time it would traditionally take to interview just one person (see Costello et al, 2025).

Second, AI can analyze large amounts of qualitative data quickly. Researchers can use AI to identify patterns across hundreds of interviews, either by applying existing theories (like coding responses into pre-defined categories) or by discovering new patterns they had not anticipated. While human researchers still need to guide this process and interpret the findings, AI can dramatically speed up the analysis. The few studies that have examined the reliability of AI for qualitative data analysis have generally reported promising results (e.g., Austin et al., 2024).

In the next section, we will conduct a guided research project with the Heinz Dilemma to help illustrate how AI interviews can combine qualitative and quantitative methods.

To understand AI interviews, let's return to the Heinz dilemma used in earlier chapters. While previous chapters used quantitative methods to examine the Heinz dilemma, Kohlberg developed his theory of moral development through intense qualitative research. He spent years conducting in-depth interviews, following adolescents as they grew up and carefully analyzing how their moral reasoning changed over time (Kohlberg, 1984).

In his research, Kohlberg presented moral dilemmas like the story of Heinz. Then, he interviewed participants about what Heinz should do and why. Through careful analysis of these conversations, he noticed patterns in how people justified their moral decisions. Some people focused on avoiding punishment, others on following rules, and still others on abstract principles of justice. These observations led him to identify six stages of moral development. In this section, we will use AI to mimic Kohlberg's qualitative approach.

We will try replicating the results of Kohlberg's interviews by exploring how people think through this moral decision in their own words. What factors do people consider? How do they weigh different principles against each other? How do they justify their conclusions, and what do their reasons reveal about moral judgment?

As with other guided projects, the text will walk you through each step. Unlike other projects, however, this study is intended as more of an overview of AI-driven methods than activities for engagement. We have conducted the study, analyzed the data, and made the results available at this link: https://bit.ly/43dyOVt. After reviewing the study, we encourage you to replicate the project with friends or family, student participants on SONA, or with participants from Connect. You only need about 20 respondents to start exploring AI surveys. For a more comprehensive analysis, we recommend 100 to 200 respondents.

To conduct a qualitative study of Kohlberg's moral stages, we used Engage, which was specifically designed for qualitative and mixed-methods research. Like traditional platforms, Engage allows researchers to create a survey link to share via email or participant recruitment platforms like Connect.

AI-Conducted Interviews: A Conversation About the Heinz Dilemma

To examine the stages of moral reasoning in the Heinz Dilemma, we conducted 100 interviews with participants who were recruited from Connect. The study began with a yes/no question: "Should Heinz have broken into the pharmacy to steal the drug for his wife?" But the real insights come from understanding why people answer the way they do—exactly what Kohlberg was after when developing his theory of moral development.

To receive those answers, we needed to tell the AI how to have the conversation. In Engage, this takes the form of an interview guide, like what a human interviewer would use. The guide is shown in Figure 8.5, following a quantitative question. We instructed the AI to probe deeper with follow-up questions that would encourage participants to share their reasoning process. The interview guide told the AI to ask open-ended questions about specific factors influencing the participant's decision, to explore emotional aspects of their reasoning, and to investigate how they viewed the relationship between the law and morality.

A sample conversation is shown in Figure 8.6. Notice how the AI acknowledges the participant's key points and asks for elaboration on specific aspects of the conversation. Having set up this study on Engage and launched it on Connect, we were able to collect over 100 in-depth interviews in about 1 hour. You can explore all the conversations by following the survey link provided above.

AI-Based Approaches to Developing a Coding System

After conducting interviews about the Heinz dilemma, the next challenge was to analyze how participants reasoned about their decision and to group the responses into categories. In the first analysis, we aimed to categorize responses according to Kohlberg's six stages of moral development.

The first step in this analysis was to provide the AI with a clear definition of each category. We gave detailed descriptions of all six of Kohlberg's stages, including examples of how people at each stage might reason about the Heinz dilemma. For instance, Stage 1 responses were described as focused on avoiding punishment ("Heinz shouldn't steal because he could go to jail"), while Stage 6 responses appealed to fundamental moral principles ("Saving a life is a higher moral obligation than respecting property rights"). These descriptions can be seen in Figure 8.7.

After providing the AI with detailed descriptions, it analyzed each participant's responses and tagged them according to Kohlberg's stages of moral development. Examining a few examples of participant responses reveals how the AI identified and categorized different stages of moral reasoning.

Figure 8.8 shows how the AI tagged a participant's response as Stage 2 (Self-Interest Orientation). When the AI analyzes each response, it provides its reasoning for each decision. This allows researchers to verify whether they agree with the AI's categorization and understand exactly why the AI made each decision. For this respondent, the AI-provided explanation was: "This quote highlights drastic measures taken in a situation of survival, which aligns with self-interest as it focuses on doing what one can to survive. The reasoning is about individual need and survival rather than any social or ethical obligation." This matches our definition of Stage 2 reasoning, which emphasizes practical consequences and reciprocal thinking.

Figure 8.9 shows how another respondents' conversation was categorized as indicating Stage 4 reasoning. The AI's explanation was: "This quote aligns with Stage 4: Authority and Social Order Maintaining Orientation because it emphasizes the importance of following the law and finding legal means to solve problems. The speaker suggests that stealing could set a bad precedent, which underscores a concern for societal rules and order."

This categorization reflects how the participant focuses on maintaining social order ("slippery slope") and following legal procedures rather than considering either individual needs (Stage 2) or universal principles (Stage 6).

AI Tools for Mixed Methods Analyses

While the AI's ability to categorize individual responses uncovers how different people reason about moral decisions, it can also be used to identify broader patterns across responses. This is where AI enables a true mixed-methods approach. Broadly defined, mixed-methods is any technique that combines qualitative research with quantitative analysis (see Curry and Nunez-Smith, 2015). One way to bring a mixed methods approach to the current study is to simply count the number of times each moral reasoning stage was detected.

After counting how often each stage is observed, we can see the distribution of different types of moral reasoning in the sample, as shown in Figure 8.10. Stage 5 (Social Contract Orientation) appeared most frequently, showing up in 62% of responses. This was followed by Stage 2 (Self-Interest Orientation) at 45% and Stage 4 (Authority and Social Order) at 38%. Because participants often used multiple types of moral reasoning in their responses, the percentages sum to more than 100%.

This kind of quantitative summary can reveal patterns that might not appear when looking at individual responses. For instance, we can see that while participants commonly appealed to social contracts (Stage 5) and practical consequences (Stage 2), relatively few people (11%) reasoned at the highest level of moral reasoning, universal ethical principles (Stage 6).

The Researcher's Role in AI-Driven Research

As transformative as AI is, it is important to emphasize the complementary relationship between human and AI when using these tools. While AI can interview people and process thousands of responses in minutes, the researcher's expertise remains central. Researchers must determine which patterns are meaningful, craft queries to explore these patterns, interpret the significance of what was found, and connect the findings to broader theoretical frameworks. AI accelerates the process, but it does not replace the need for human judgment or expertise.

The key advantage of using AI is that exploratory analyses can be done almost instantly. What once might have taken months can now be accomplished with a few clicks, allowing researchers to conduct qualitative analyses at the speed of quantitative research. Indeed, by enabling natural conversations to be conducted at scale, AI-based methods help bridge the gap between quantitative and qualitative approaches. As AI systems become more sophisticated in their conversational abilities, researchers may find it increasingly useful to conduct studies that combine the best aspects of structured surveys, in-depth interviews, and large-sample statistical analysis to expand our understanding of human behavior across countless domains (see Austin et al., 2025).

Summary

In this chapter, we explored how artificial intelligence is transforming research in the behavioral sciences. From enhancing traditional surveys with dynamic, conversational approaches to enabling qualitative research at unprecedented scales, AI is expanding the toolkit available to researchers in ways that were unimaginable just a few years ago. The MIT study on changing conspiracy beliefs that opened the chapter highlighted how AI can personalize interactions with thousands of participants simultaneously, while our exploration of the Heinz dilemma demonstrated how AI can both conduct in-depth interviews and analyze the volume of data these conversations produce.

The capabilities of AI-driven survey platforms bridge the traditional divide between qualitative and quantitative research. Using platforms like Engage, researchers can now conduct hundreds of in-depth interviews in the time it would traditionally take to complete just one. Moreover, the ability to analyze these conversations both through existing theoretical frameworks (like Kohlberg's stages) and through exploratory analyses offers a powerful combination of confirmatory and discovery-oriented research. The result is a more nuanced and comprehensive understanding of human behavior, motivation, and decision-making. And these tools are only going to improve over time, which brings us to a good place to end Part I of this book: at the intersection of traditional research methods and the newest tools for behavioral research.

In Part I of this book, you have built a strong foundation of knowledge about behavioral research. Together, we learned about descriptive, correlational, experimental, and mixed methods designs; we learned about the tools researchers use to facilitate modern research; we learned how to find, develop, and validate measurement instruments; we explored several techniques of statistical analysis; and we saw how AI can enhance traditional approaches to research. This knowledge forms the bedrock of behavioral science.

In addition to gathering knowledge, Part I gave you several opportunities to conduct your own research. By walking through guided projects before testing your own ideas, you have developed one of the most important qualities in learning to conduct behavioral research: experience. This experience should help you tackle future projects with confidence.

In Part II, you will build upon the fundamentals and learn to apply them in the context of online research. In particular, you will discover how to design studies that take advantage of the unique capabilities of digital platforms while navigating their potential challenges. Whether you are conducting a simple survey, a complex experiment, or a mixed-methods study combining qualitative and quantitative approaches, the principles and practices covered in Part II will equip you to implement your research effectively in the online environment.

In many ways, the evolution of research methodologies parallels broader technological transformations. Just as AI has moved from science fiction to everyday reality, research methods have progressed from paper-and-pencil surveys to sophisticated online interactions. The future of behavioral science lies at the intersection of classic research methods and modern technological innovations. By mastering both the fundamentals covered in Part I and the strategies for implementation covered in Part II, you will be equipped to conduct research that is not only scientifically sound but also leverages the full potential of contemporary tools. This combination will allow you to address nearly any question that sparks your curiosity.

Appendix: Instructions for Writing Up the Results of a Study

After completing a research project, it's time to communicate what you found. Use the report guidelines below to write up the results of each independent project you conduct in your course. Each report should be approximately 2-3 pages long and include the following sections.

Abstract - Do this last!

Begin your report with a brief abstract of 150-200 words that provides a complete overview of the study. The abstract should include the purpose of your research, how you conducted it, your key findings, and the main conclusion. Think of this as a condensed version of your entire paper that someone could quickly read to understand what you did and what you found.

Your abstract should touch on your research question, the variables you examined, your sample, the method of data collection, the statistical approach, key results including correlation coefficients, and the significance of your findings. Write this section last, after you've completed the rest of your report.

Here is an example abstract: "This study examined the relationship between mindfulness practice and perceived stress levels among college students. Participants (N = 124) completed online measures assessing their frequency of mindfulness practice and perceived stress. Results revealed a significant negative correlation between mindfulness practice and stress (r = -.41, p < .001). These findings suggest that regular mindfulness practice may be associated with lower stress levels in college students, though the cross-sectional nature of this study prevents causal conclusions. Future research should examine these variables longitudinally to better understand how mindfulness practice affects stress over time."

Introduction

Begin with a clear statement of your research question and hypothesis. What relationship were you investigating? What did you predict you would find, and why? This section should be brief but specific, focusing on the variables you measured and the relationship you expected to find between them.

Literature review

Indicate whether your study is:

• A replication of a previously established effect
• A modification of a known relationship
• An investigation of a novel relationship

To determine this, conduct a literature search using Google Scholar, as you learned in Chapter 2. This will help you contextualize your study within existing research. Make sure to reference any key studies that inform your hypothesis or relate closely to your research question.

Method

Describe how you conducted your study in straightforward terms. Include information about the participants: How many people participated in your study? What were their key demographics (average age range, gender distribution)? Then, describe the measures you used: Which variables did you measure and how? Describe each scale or measurement tool, including well-formatted tables that list the items in each measurement scale you used. Finally, describe the procedure. How did you collect your data? Was it through Connect, student participants, or another source? What software was used for stimulus presentation (Qualtrics)?

This section should be factual and concise. Provide enough information that someone could understand what you did without overwhelming them with details.

Results

This is where you present what you found. Focus on the key statistical findings that address your research question. If your study is descriptive, present statistics that describe people's responses. If the study is correlational, report the correlation coefficient(s) between your main variables, noting both the strength and direction of the relationship. And, if the study is experimental, report the tests that allow you to determine if there was an effect of your independent variable(s).

Mention whether your results were statistically significant. Describe any interesting patterns in your data, such as unexpected relationships between variables. Include a figure that visually represents your main finding (a scatter plot or bar graph), and then include all relevant SPSS syntax and output in an Appendix. This is like showing proof of your work.

Remember to present your results in plain language alongside the statistical information. For example: "A Pearson correlation analysis was conducted to examine the relationship between anxiety (as measured by the GAD-7) and depression (as measured by the PHQ-9), r(524) = .82, p < .001. These results suggest that anxiety and depression are strongly related. Specifically, the results suggest that people who experience higher levels of anxiety also tend to experience higher levels of depression."

Discussion

Interpret what your findings mean in about a paragraph or two. Did your results support your hypothesis? Why or why not? How do your findings connect to previous research or theories? What might explain the patterns you observed?

Also, briefly mention at least one limitation of your study and one idea for future research that could build on your findings.

Conclusion

End with a brief statement (2-3 sentences) summarizing the main takeaway from your study and why it matters.

Formatting Guidelines

• Write in clear, non-technical language that anyone with basic knowledge of behavioral science could understand.
• Use proper APA-style formatting for reporting statistics.
• Include one figure that visually represents your main finding.
• Double-space your document with 1-inch margins.
• Include a title that captures the essence of your study.

Reference section

Include a reference section that lists all the sources you cited in your report. Format your references according to APA style guidelines. At minimum, you should include references for:

1. Any previous research you mentioned when contextualizing your study
2. The sources of any established measures or scales you used
3. Any theories or frameworks you referenced in your discussion

For example:

Burke, M., & Kraut, R. E. (2016). The relationship between Facebook use and well-being depends on communication type and tie strength. Journal of Computer-Mediated Communication, 21(4), 265-281.

Cohen, S., Kamarck, T., & Mermelstein, R. (1983). A global measure of perceived stress. Journal of Health and Social Behavior, 24(4), 385-396.

Gosling, S. D., Rentfrow, P. J., & Swann, W. B. (2003). A very brief measure of the Big-Five personality domains. Journal of Research in Personality, 37(6), 504-528.