Implications of recent advancements in AI for qualitative research

The term ‘artificial intelligence’ or ‘AI’ was coined by computer scientist John McCarthy in 1956, but in 2023 this acronym seems to be on everyone’s lips — what’s changed?

In brief, ‘generative’ AI systems – which enable computers to generate original content — have evolved rapidly over the last decade. Enter the much-talked-about ChatGPT, Dall-E, and so on. 

More to the point, these tools have become widely accessible to ordinary people, including us humble market researchers. So, it’s high time — and hopefully not too late — to consider AI technology’s usefulness to our industry.

Critically, we need the cause of some anxiety: the prospect that AI systems may be able to perform our market research jobs better than we can.

This concern points to a fundamental question: is human intelligence inimitable? Recently, there has been some insightful analysis on human intelligence and how it fundamentally differs from the sort of derivative ‘intelligence’ a machine can exhibit. These arguments (which we won’t summarise here) give us cause for optimism.

AI for qualitative market research

Given the accelerating development of the AI sector, we’re interested in a relatively specific and dry question: what will AI mean for market research? This general question, as with most regarding AI and its impact on jobs in the knowledge economy, we’ve often heard formulated negatively —that is: how much market research work will be lost to AI systems? Such concern is probably justified, and the natural defensive response is to point out activities only a market researcher, and not an AI bot, could perform. Indeed, one of the most central aspects of qualitative market research — conducting interviews and focus groups — appears to be just the kind of task humans are uniquely equipped to do. In this regard, an analogy with therapy strikes us as productive.

Recently, the question of whether AI systems should provide therapy via a chatbot has seen a fair bit of media attention. But the mainstream view appears to be that while an AI therapist is better than nothing, it is no substitute for a human therapist; AI systems might be useful in batch mental illness diagnostics or for reminding a person to meditate when anxious, however algorithms should not be used for psychotherapy because of the practice’s complex discursive demands.

Qualitative market research interviews and therapeutic sessions share some important similarities with reference to the conversational (and subtly investigative) nature of the interactions: both involve asking open-ended questions, actively listening to responses, and attempting to uncover underlying motivations and experiences. We think that — like therapy —successful focus groups and in-depth interviews are greatly enhanced by the rapport developed between the two parties which, for instance, encourages participants to fully commit to the process and give genuine answers. Qualitative market researchers also, like therapists, need to interpret non-literal or non-verbal communication; and sometimes we need to display sensitivity towards interviewees or have regard to the most appropriate way to behave when discussing delicate topics. For purposes such as these, people like professor of mental health and philosophy Şerife Tekin argue that conversational AI cannot adequately emulate or decode the complexities of human emotion.

But the very fact that AI systems probably couldn’t do everything a market researcher needs to do points to the fact that these machines could accomplish tasks we would never be able to. (The obvious example here is analysing terabytes of data in the form of fieldwork videos and transcripts.) So: if artificial intelligence isn’t identical to human intelligence, then these systems doubtless have non-human capabilities. Such capabilities, we believe, could supplement and improve human-led market research. Additionally, if an AI bot can also perform some of the more robotic tasks we are required to do as researchers, using it may allow us more time to focus on the especially human undertakings our work entails. So, more positively, what will AI add to market research?

Some companies have already designed and released relevant products. Over the last week we’ve trialled CoLoop—an ‘AI co-pilot for qualitative research’—launched in early 2023 by London-based software engineer Jack Bowen. From our perspective, CoLoop works like this:

1.    Upload all the video or audio content from your fieldwork;

2.   The application transcribes all discussion content into text;

3.   Then, start chatting with the software. For example ask it to ‘summarise the barriers to buying an electric car’;

4.    Wait a couple of seconds, and the chatbot will produce a perfectly worded response to your question.

The AI-generated analysis doesn’t tell us the full story (e.g., it will leave out the tone in which someone described their feelings about EVs) but so far, we’ve found CoLoop useful in two ways.

First, we’ve turned to this AI program to check our implicit biases. Many qualitative researchers will be familiar with the problem of ‘availability bias’: the tendency to mainly refer to the information that comes to mind most easily and quickly. For example, we might focus on the interviews we led ourselves, rather than those done by a co-worker, or we might inadvertently emphasise the contributions of the ‘loudest person in the room’ from a focus group. So we’ve used CoLoop as an ‘availability heuristic’ to make sure the person writing the client report isn’t unconsciously biasing the findings in some way.  

Second, it’s saved us time, allowing us to crunch the transcript data much more efficiently than we otherwise would. We can neatly organise our reporting themes into grids, and quickly summon relevant quotes.

In summary... 

At the time of writing, we see AI technologies like CoLoop as helpful, time-saving tools that may improve the integrity of research findings.

Will AI replace our jobs as qualitative researchers?

Safe for now. But we’ll be looking in particular to see how multimodal AI develops, and whether these kinds of machine learning systems start to accurately recognise body language, tone of voice, pregnant pauses, and all those subtle little cues that we rely on to get the full story.