There is no doubt that Check all that Apply
(CATA) questionnaires are increasing in popularity in
consumer research. At Sensory Dimensions, more and more of our clients are
using CATA data, and we're not alone – the topic was the subject of a whole
workshop at last year’s 10th Pangborn Sensory Science
Symposium held in Rio de Janeiro.
So, what are the benefits of CATA, and what are
the pitfalls? Read on to find out.
The Benefits of CATA
CATA (which is also known as Tick all that Apply
or TATA) offers the researcher a route to recovering the elusive ‘why’ people
like or do not like a product. The alternatives so far have been open-ended
questions, which often return a lack of consensus and limited insight, or a long
series of pre-determined intensity and ‘Just about Right’ scales. Requiring a
response, even when not relevant to the respondent, scales can generate noise
in the data; and the long questionnaire makes for tired and bored consumers.
CATA provides a compromise – respondents are
presented with a pre-determined list of terms but only have to check, or tick,
those that they consider applicable to the test product. No measure of
intensity is required, and they can ignore terms they consider irrelevant. Furthermore,
by offering a list of terms, CATA also helps in those situations where consumers
find it hard to encapsulate their perception.
Terms used in CATA
lists can be purely sensory – such as sweet,
bitter, crunchy; or emotional – fresh,
energising; or functional – good for
breakfast. With sensory terms there is also the possibility of exploring
the ideal, such as too sweet or not sour enough. The technique has the
advantage that it is suitable for any product, is quick and simple for
consumers to apply, and the data is amenable to statistical interpretation,
even Preference Mapping.
This all sounds fantastic…but what are the
pitfalls? At the Pangborn 2013 workshop a few key messages came out. The number one consideration
is how you list the terms. Research using eye tracking glasses has shown that
respondents read CATA word lists from top left to bottom right, in other words,
they follow normal reading behaviour. Attention drops as they read through the
list, but changing the order of the words leads to consumers looking for longer
and taking longer to choose. Eye tracking has also shown that less time is
spent on later samples. So it is essential to change the order of the word
lists between samples and between assessors.
Secondly, we need to consider the number of terms
used and the way we group them on the questionnaire. Ares and colleagues (2013)
found that splitting terms by modality (flavour, texture, appearance) led to use
of more terms and to increased discrimination between products. They also found
that asking people to check the list whilst eating the product gave better
discrimination than post-eating checking. The question of how many terms still
needs further research. In conclusion, shorter CATA lists split by modality are
better, but you still need to randomise across samples and assessors.
All supplementary questions
that we ask in a consumer study have the potential to bias the overall liking
response. CATA has the same potential as a series of attribute intensity or JAR
scales, but some researchers argue that the potential for bias is lessened with
CATA as the technique requires less engagement by the consumer. Certainly
Jaegar and Ares (2014) have found no impact of CATA on overall liking response
but then again, found no effect of asking other types of question either.
Analysing the Data
So what about analysing CATA data? The
simplest way to deal with these data is to present simple frequency plots
showing the relative usage rates for each term across samples. Statistical
analysis of these frequencies can be carried out using a sign or McNemar’s
test, or Cochran’s Q Test for three or more samples. This means that we
can use CATA to know if products are different. To understand what makes the
products different, correspondence analysis allows us to map products based on
CATA data, effectively summarising our frequency plots into a two- or
three-dimensional solution. Multi-dimensional scaling will determine
correlation between attributes. Penalty analysis is also being used with CATA
data to understand the impact of attributes on liking and, by using benchmark
products, we can determine which attributes need modifying to improve a
product. These techniques are further
Overall CATA provides an exciting and powerful
approach to understanding consumer response to products and, where appropriate,
is worthy of serious consideration as part of a well-designed consumer study.
Ares, G., Jaegar, S.R., Bava, C.M., Chheang, S.
L., Jin, D., Gimenez, A., Vidal, L., Fiszman, S.M. and Varela, P. (2013) Food
Quality & Preference Volume 30, Pages 114-127
Jaegar, S. R. and Ares, G. (2014) Food Quality
& Preference Volume 35, Pages 1-5, In press
Details of the IFST Student Prize for 2014 have been released. Contact us to find out more.
Get in touch with us to discuss how we