AUTHOR

JUSTIN BATROUNEY
Technology Director

Overview

In quantitative market research, the humble survey is the most powerful tool in our arsenal. Our surveys primarily consist of the typical fixed code-frame questions with a list of options for a respondent to choose from – be it single or multi-select. Sure, we add in complexity such as randomisation, cells, quotas and logic but at the end of the day, respondents are still choosing answers from fixed code frames. They are constrained.

Fixed code frame questions are easy to evaluate and make sense of at scale with nothing more than Excel or your favourite stats packages such as SPSS or Q. But as market researchers, we like to dig a little deeper, without the constraints of a fixed code frame. This is where the open-ender comes into play. We ask the respondent to type a few words or sentences in response to a question where they are free to express their opinion in their own words without the constraints of options to choose from.

The open-ender is a powerful tool where great insight can be gleaned from not just what is said but how it is said and even what is not said. As market researchers, we have a love-hate relationship with the open-ender. We know they are powerful, but we also know they are really time consuming to analyse and make sense of.

The open-ender is a powerful tool where great insight can be gleaned from not just what is said but how it is said and even what is not said.

As market researchers, we have a love-hate relationship with the open-ender. We know they are powerful, but we also know they are really time consuming to analyse and make sense of.

Whilst it is easy to cherry-pick a few open-ended responses that help craft a story, to really unleash their power, every open-ended question needs to be read and coded against a code-frame. We can then get a sense of how many respondents were talking about colour, texture, brand fit, sentiment or whatever it is you might be interested in.

The code-frame for open-enders is usually agreed on either up-front or after analysing a smallish sample of responses. This code-frame is then applied to all responses manually by a human. This is time consuming, laborious and expensive.

BUT THEN CAME AI

With AI, we can ingest all the responses to an open-ended question and have it code each response against the code-frame instantly. It can replicate hours of human effort in seconds. In fact, not just replicate but go several steps further.

A human coder will have an inbuilt bias towards the code-frame.

This bias is introduced either upfront if a code-frame is pre-determined, or by the sample responses they are exposed to initially when formulating their code- frame. If the initial sample responses are not truly representative of the whole sample, the code-frame can easily be skewed.

AI interprets the entire sample before determining a code-frame and it is therefore truly representative of all responses. No bias, and no constraints. The AI can also be improved, by the researcher reviewing its initial findings and asking it to have another go, but this time combining certain items or ignoring irrelevant ones. This process can be repeated until a succinct picture is created.

As good as a human coder may be, they have completed their analysis at a point in time – usually when the survey is no longer in field. What if we want to get a read on the data mid-way through a study? Or we have a study that is always listening. This is where AI shines. In seconds, AI can give you an analysis of the responses at any point in a study. You can get a feel for what is being said and pivot accordingly while the study is still in field.

Coding open-enders is just one way to utilise their power. How nice would it be to interrogate the data? You might have a hypothesis that you know the open-enders can answer, but examining the code- frames is not getting you the answer you need. With AI you can simply ask it a question and it will analyse all the responses and give you an answer in seconds. “Of all the respondents that mentioned shopping online, what were some of the barriers to purchase?” or “did respondents mention the serving size was too small?”.

These are examples of questions the AI can respond to. And the responses are extremely powerful and insightful. “Many respondents mentioned it is difficult to understand if the sizes listed would fit them, and some said the shipping information was confusing.” is an example of a reply.

AI has allowed us to truly unlock the power of the open-ended question. It is designed to be unconstrained in the data it collects, but until now it has been constrained by our analysis. AI has thrown these constraints away.

INSIGHT IQ

InsightIQ has extensive AI capabilities. It can analyse, interrogate and code not just open-ended questions, but video responses, in-depth interviews and forum discussions too.