Why is it important to have both qualitative and quantitative measures for feedback?

User research is a non-negotiable component of experience design.

It’s what gives us consumer insight which in turn helps us make informed decisions about design. There are two main types of user research: qualitative and quantitative and of course, there are pros and cons associated with both. When to deploy either-or depends very much on the design decision you need to make.

About qualitative research

Qualitative research methods are used to gather feedback and insights from a relatively small group of users. In general, I would describe qualitative research as answering the; How, When, Where and Why, of user behaviour. Examples of qualitative methods include: unstructured or semi-structured interviews, ethnography, diary studies, focus groups and co-creation sessions.

Capabilities

• Explore or validate concepts/designs with a small sample (approx. 12-50 participants)
• Observe immediate reactions and emotional responses to an experience, including tone of voice, facial expressions and body language
• Uncover unexpected pain-points or opportunities to add value in realtime

Limitations

• Sample size is limited, therefore lacking the confidence of numbers
• Sample obtained is not (statistically) representative of the entire user population
• Unstructured methods can lead to inconsistent data capture

About quantitative research

Quantitative research methods are also used to gather insights but the size and range of the user group can be expanded. This type of research focuses more on collecting user “data” as opposed to just insight. In general, I would describe quantitative research as answering the “who” and “what” of user behaviour. Examples of quantitative methods include: structured, unmoderated testing, surveys, and questionnaires.

Capabilities

• Explore or validate concepts and designs with a large sample (approx. 50-1000+ participants)
• Capture behavioural data for task-based research, including task completion times, clickpaths, heatmaps, and success or failure scores
• Measure satisfaction and usability with standard measures (i.e. System Usability Scale, Net Promoter Score)

Limitations

• Unable to probe participants in the moment on their responses or observed behavior
• Lacks moderation; therefore the stimulus presented must be functional for use without assistance from a moderator

Choosing the right method to inform design decisions

Research methods are not interchangeable and have specific objectives which they are most suitable for. Choosing the most applicable method will depend on what design decisions need to be made for the creation or refinement of a design.

As a rule of thumb, use qualitative research methods for exploring ideas, designs, or processes which do not require concrete hypotheses or structured, statistically valid feedback. I have found that qualitative methods are particularly useful for serendipitous discovery, and often provide more in-depth insights.

You can use quantitative research methods such as A/B testing for validating or choosing a design based on user satisfaction scores, perceived usability measures, and/or task performance. You will find the insight will be more superficial, but the data is statistically valid and can be generalised to the entire user population.

A combination of qualitative and quantitative research is typically best for most design projects if budget allows. By using both methods you can achieve a deeper level of insight through the exploratory nature of the research in addition to statistical evidence to support your design decisions. Using qualitative and quantitative research together answers to the; Who, What, When, Where, Why, and How, of user behaviours and experiences.

Using both qualitative and quantitative approaches in your research

Both qualitative and quantitative research are indispensable tools in a researcher’s toolbox. Cint provides a platform for conducting quantitative research via online surveys – a solution that can be quite useful for augmenting qualitative research.

First, it’s important to understand the difference between qualitative and quantitative research. Qualitative research is primarily exploratory in nature, and helps a researcher better understand motivations, needs, processes, and rationale for behaviors (among other things). It provides deep insights into a situation, and helps form ideas or hypotheses for potential quantitative research.

Quantitative research, like online surveys, tends to be more numbers-focused and can help to assess hypotheses from qualitative research. Adding a quantitative approach to qualitative research can provide a more holistic (and quantifiable) view of the subject matter you are researching.

Conversely, qualitative research can be beneficial when designing a quantitative research project, helping in the following ways:

• Learning about the consideration process – that is, how a customer thinks about or uses a product or service, and the context for a purchase.
• Hearing a customer’s language – this helps when writing questions. Speaking in a customer’s language (rather than the internal language of a corporation) helps elicit the best answers leading to the best insights.
• Understanding the holistic customer experience – hearing descriptions of the logical sequence of events helps to frame the quant survey in a way that will make sense to the consumer. This also helps hone-in on the right customer targets, and the timeframes to be studied.

Considerations when moving from qual to quant research

Once qualitative research has been completed, you can use these findings to inform your quantitative research. Some considerations for migrating to quantitative research include the following:

Which targets should be interviewed, and how does that impact my sample design?

The fundamentals of any quant survey involve identifying the right sample design to fit the research objectives. The key question when migrating from qualitative to quantitative research is: “Did the qual interviews uncover insights that change the incoming assumptions?” For example, did we learn we should expand the population under study, possibly including purchase influencers rather than only interviewing decision makers. If so, expanding the scope of the sample design to include new targets can prove beneficial.

How can I use the quant research to obtain the deepest insights, maybe going beyond the original research objectives?

If the scope of the sample design has been expanded, creating quotas with statistically readable base sizes for any new targets is desirable – which can lead to insights above and beyond the original objectives. Regardless of the target audience(s) to be studied, researchers should balance statistical reliability with sample budget to obtain the best outcome for all constituents of the research.

If further qualitative findings are needed, how can I obtain those through the quant survey?

While traditional open-ends provide opportunities to obtain qualitative data, they do lack the richness gained through one-on-one dialogue. By embedding an interactive feature into a quant survey to further query respondents, open-ends can become more valuable. There are a number of companies today offering capabilities to engage an interviewer in real-time from a quant survey, or to probe through a chat-bot. In both cases, these interactions lead to the best of both worlds – deeper qualitative responses at quantitative research sample sizes.

Are there hybrid qual/quant platforms that lend themselves to exploration similar to traditional qual?

In a similar manner, researchers today have created a variety of methods and services that merge the depth of qualitative interviewing with the breadth of a quant survey. Some of these services engage hundreds of people to vote up or down the answers of others to achieve a consensus, while others are moderated more like a focus group but with a more readable sample size. In all cases, researchers should explore and consider some of these options for merging qual with quant, or creatively fusing the best of both into your own methodology.

Can quant interviewing replace any future qualitative research?

Historically quantitative research has been more expensive than qualitative. But today the rapid deployment of online surveys with reasonably priced sample can deliver insights fast and cheaper with greater statistical reliability than some qualitative projects. If the intent of some research was a qualitative deep-dive into certain subject matter (for example, the different uses of a household cleaning product), it could make more sense to use a qual/quant hybrid approach or to integrate a real-time chat tool into an online survey. In that way, qualitative depth is obtained along with the prevalence of the different product uses along with statistical reliability.

Best uses of qualitative and quantitative research methods

Quantitative and qualitative research are complementary methods that work well together to provide insights that are both deep and wide. Regardless of the research objectives, now more than ever researchers have options and countless qual/quant tools to design projects that deliver more actionable insight.

Contact our teamto learn more about Cint’s survey solutions for quantitative research!

If you are interested in adding a quantitative component to your qualitative research, Cint’s team of experts can help guide you through the process. Contact the Cint team for more information.

Why do we need both qualitative and quantitative data?

What is qualitative and quantitative feedback?

Why do we need to couple quantitative assessments results with qualitative?

Why we use qualitative and quantitative research?