10 alternative qualitative methods

alternative qualitative methods


At the National Council for Research Methods ‘Research Methods Festival’ last month, Steve Wright (from the University of Lancaster) mentioned in his talk the frustrations he has with students that do the bog-standard ’12 semi-structured interviews’ methodology for their qualitative research projects. This prompted a lot of discussion and empathy over lunch, with many tutors lamenting how students weren’t choosing some of the more creative methods for qualitative research.


Even a lot of the popular textbooks on qualitative research only mention the basic methods, or some variants on textual data collection (eg Braun and Clarke 2013). Even if it’s not interviews of some kind, transcribed focus groups and other textual methods definitely dominate the literature. Helen Kara has a textbook specifically on Creative Methods, which is well worth a read if you are looking for inspiration. But the value of qualitative research can be magnified by choosing the right imaginative methodology, and thinking outside the box a little to redefine what we can collect and analyse as ‘data’.


This is a huge world, but I wanted to give a taster (with lots of examples) of 10 qualitative methods that can go a lot beyond the default ’12 semi-structured interviews’ and engage with participants in new and exciting ways.

 


Diaries


OK, we’ve talked about diaries before. But there is much more to diaries than just hand written journals. You can also have audio diaries (Williamson et al 2015) and video diaries (Bates 2013). There are even diary apps for phones (Garcia et al. 2015), which can notify partipants at reguar intervals to find out what they are doing or feeling. Laura Radcliffe and Leighann Spencer gave a great talk on the challenges and advantages of diary apps at RMF 2018. Each have their own benefits and give you a different level of insight into participants lives, but for certain research, especially where you want to minimise recall issues, regular recording in one of these ways can be really useful.

 

Participant Photography


Although sometimes connected with diaries, getting participants to record their life through Photo Elicitation can get them to reflect on important issues, and provides a good basis for discussion. Usually you give your participants a camera (although with the ubiquity of smartphones this is rarely necessary these days) and ask them to take pictures of things that have meaning to them about your research question. This is the concept of Photo Voice, where you give your paricipants a way to express their lives and experiences pictorially. There’s a nice overview here by Harper (2002).

 


Art


Many of the ‘creative methods’ focus on different ways to integrate art into research. You can basically use any medium, but the idea is often to get participants to reflect on their life experiences and create something (a drawing, clay sculpture, collage) that expresses something connected to the research. Examples include ‘Target drawings’ Tracy, et al. (2006), clay sculptures, (Or 2015), self-portraits (Esteban-Guitart et al 2016), drama and theatre (Norris 2010) or even quilting (Bacic et al. ND). There are many more listed in this presentation by Mannay (2016). This is a huge field, and always fun to see different ways people have been innovative here. However, a key part of the method is getting participants to either label and explain, or discuss with the researchers and other participants the meaning and different interpretations of their creations.

 


Walking methods


If your research is connected to a place, or how people experience an area, there are many interesting approaches you can do with participants while walking with them through a place and getting them to explain their world. These have various names and variations such as the ‘walking interview’ Jones et al. (2008), transecting or walking fieldwork (Goschel 2015). You can record these visually, aurally or with notes and pictures, or get participants to reflect on them afterwards.

 


Mapping / network diagrams


Another good tool for getting people to explore and explain their geographical area with researchers, but mapping tools can also be used to demonstrate other things, such as connections between organisations people use, social networks, or how they see connections between concepts as in mind mapping (Burgess-Allen and Owen-Smith 2010) . There is pictorial narrative mapping Lapum et al. 2015 (which is more like some of the artistic reflection techniques above), body mapping which can be used to show pain (Mukherjee 2002), or getting local people to create and label a map of their area.

 

Secondary Analysis


To some, this may seem even more boring than just doing qualitative interviews, but secondary analysis of other sources of data can be really interesting and insightful, and avoids a lot of practical and ethical issues. You can do document, media or social media analysis or even re-analyse someone’s existing dataset to see if it can reveal something about a different research question. There’s some more advice on our post here.

 

Games and activities


When you do focus groups, don’t just facilitate dry discussion: use games and fun activities to get your participants engaged and sharing. You can use sorting and ranking exercises with cards you make with each card representing a part of the research. You can get people to discuss photos, newspaper articles, made up stories about a controversial issues or flip-charts where you get people to come up with ideas or answer difficult questions. Get people to move: show how strongly they agree with a statement by standing at different positions along a line. In each of these situations, the data can be either the outcome (where people stand / what people share) or the discussion that ensures. There’s a whole book of tips and tricks for making focus groups more interesting (and successful): Participatory Workshop (Chambers 2002).

 

Participatory research


This isn’t always a method in itself, but in some situations it can be really valuable to include participants in the data collection or analysis. In some paradigms they can be seen as the real experts of their own lived experiences, or an ‘insider’ can be a useful co-researcher. Often they are able to make sure that the most relevant questions are being asked, can act as gatekeepers to other participants that might be difficult to reach, or will have their own interpretations of the data that can challenge researchers. It also can shift the power dynamic away from binary researcher and researched. Much more on our blog post on participatory research.

 

Observation / Ethnography


If you have the time to deeply engage with an organisation or a group of people, researchers can become embedded in their research subject with ethnography or participant observation. Usually a researcher will spend weeks, months or even years watching and learning a research context first hand, and it can give very detailed data and understanding. However, there are shorter variations of observation or ‘rapid ethnographies’ (Vindrola-Padros and Vindrola-Padros 2017) which can be a great complement to other qualitative research methods: verifying and expanding on other sources of data.

 

Surveys


Now, this again might seem a bit boring, but I think surveys are often overlooked as a qualitative research method. There are a good way to reach out to lots of people, online, in person or by post, and you can be a lot more creative with questions. Get people to explain what they see in a picture. Use one word to express how you feel about something.  Use emoji’s or get people to rate or rank statements. Ask questions about identity in different ways: which Disney princess do you most associate with, and why? Leave space for lots of open ended answers, but choose creative and engaging questions to get people to think and reflect.

 

Hopefully this post has inspired you to consider or even try out some different qualitatve methods that differ from the normal boring ones. The key with all these is to consider what exactly will constitute the data you collect, and then how you will analyse it. For data that comes back to text or transcripts, Quirkos can be a fun and engaging way to help you analyse differently as well. Give the free trial a go, and see how it makes qualitative analysis a visual method!

 

 

 

What is qualitative observation?

qualitative observation

 

Essentially, observation is a type of, or more likely, a part of ethnography. In ethnography, anthropologists (people who study people) turn their observations of people, cultures and organisations into written field notes (a bit like a research diary). While some of this may be reflexive (the participants own thoughts and feelings) most focuses on the activities and interactions of the people being studied.

 

There are broadly two types of observation. The first is participant observation in which the researcher becomes part of and gets involved in the context, area or group they are studying. The second is direct observation, where the researcher does not take part in the activity or setting, but is more of a fly on the wall – passively watching and recording what is happening.

 

There are advantages to both approaches: for example it’s easier to observe and take notes with direct observation, while during participant observation you may be actively taking part in the meeting / surfboarding session (Kawulich 2005). However, participant observation can allow for a deeper level of understanding, embedding and acceptance from the study group, allowing for more significant insights. Taking part in the culture/activity can also provide a ‘Walk two moons in their moccasins’ revelation, allowing the researcher to fully understand and empathise with the decisions and actions of participants.

 

Typically, a participant observer would offer to get involved by volunteering, doing some useful task like taking minutes or driving people around – essentially doing favours that let them help out while being able to see what is going on. It does not need to involve the actual task or skill being researched – for example in an ethnography of two tattoo parlours the author “helped maintain files of tattoo designs, working behind the front desk” although eventually got tattoos herself (Velliquette 1998).

 

One specific field of observational research is ‘Organisational Ethnography’, where researchers look at organisations, management or work places. (Ybema et al. 2009). Here ethnographers may look at a wide range of organisations from parliament (Crewe 2018) to a steel mill in Sheffield (Ahrens and Mollona 2007).

 

However there are also methodological limitations to observation. Even with direct observation, there can be an effect from having the researcher in the room – people’s behaviour may not be normal, and maybe modified if participants have a sense of being watched or judged (see the Hawthorne Effect). With time and acceptance of the research, the effect may become less, but it is still difficult to claim pure objectivity in observational research, especially when the researcher is talking part directly in the culture of the researched.

 


This is why reflexivity is so important in ethnography and participant observation, because the prejudices and interpretations of the researcher need to be untangled (or at least made explicit) from the data.

 

Observational data


Any method of observation has a myriad of practical and theoretical challenges. The first is to consider what kinds of data will be produced during the observation. Usually these will be field notes, but may also include documents (minutes from meetings, policy), audio, video, music and direct comments from people in the field of study. Many ethnographers use a dictaphone to record either the whole session live, or more likely their own thoughts and reflections afterwards. This creates audio data which probably will need to be at least partially transcribed.


Researchers need to have a loose plan before they start their fieldwork of what kinds of data will be collected and how, so that they can make sure the data can be effectively analysed. However, there will often be unexpected sources and type of data in a long and embedded fieldwork project like this, so prepare for some flexibility. Also, consider the volume of data that participant observation will generate (like most qualitative methods). For one study 40 hours of observation generated 28,000 words when transcribed (Conway 2017).


It’s also worth thinking about triangulation, and paring with other qualitative methods. For example, semi-structured interviews can be a good compliment to observation, as interviews allow you to ask questions one-on-one with people who have been part of the ethnography. These can be used to check assumptions, and ask for questions and clarifications on aspects of culture that are not obvious (e.g. Why do you all wear these hats?). Just remember that these direct questions are generating a different type of data to the observation: the participant during an interview is conscious of being questioned about their culture, and is giving an expressed opinion (These hats are stylish) which may not match the researchers interpretations based solely on observation (People wear hats to emulate the cool kids).


In fact, there is usually a little informal observation going on in most qualitative research projects. It’s hard just to meet a series of people for interviews without watching the culture around them and how they act with others (as Katz (2002) says – social researchers are always in ‘the field’). And often finding the right people to interview (if this is your designated research method) involves some participant observation to identify the most interesting respondents.

 

 

Gaining access, consent and trust


Yet for any type of observation, there will be significant issues around access and consent. The first hurdle is to persuade a group of people that it is a good idea to have a nosey researcher hanging out with them for months at a time. It can sometimes be tempting to claim that the research will be useful to them; in getting their situation better understood, or identifying issues and problems in their culture. However, this is a difficult thing to promise. While all good researchers should provide feedback and share findings with participants, the things that an academic researcher is investigating may not match with the immediate problems of participants. Qualitative observation of this type is usually based around fairly speculative exploration, with a sort of grounded theory approach, so there is little guarantee from the start exactly what the area of focus will be.


Usually, gaining access will be done through ‘gatekeepers’ (see more on gatekeepers in this article on recruitment  – https://www.quirkos.com/blog/post/designing-a-qualitative-recruitment-strategy). This may be a senior leader (mayor, tribal leader) in a cultural setting, or manager in an organisation. However, it is worth considering wider issues of consent with the many people a researcher will encounter. Although a senior manager may have given permission for the research, this does not automatically mean that their subordinates also give consent. There may be situations where this is explicit ‘Everyone must take part’ but individuals may not be freely giving consent if they are scared of going against the wishes of their boss.


Getting access for this type of in-depth observation can be a lengthy and fractional process, where researchers are only given access to certain areas at first, and as trust grows they are invited to more closed-off activities (such as weddings or management meetings). Building trust and rapport is an important skill that ethnographers must develop, and to which there are rarely shortcuts – long periods of time are usually required to negotiate access. Indeed, some researchers have come to see the difficulty of negotiating access as an important part of the ethnography itself (Frandsen 2015).

 

 

We are going to look more at ethnography in a future blog post, but what ever type of observation you are making, you might consider qualitative analysis software like Quirkos to help analyse and find themes in your qualitative text data. Download a free trial today, and see why people describe Quirkos as ‘intuitive’, ‘colourful’, and even ‘fun’!

 

 

Quantitative vs. qualitative research

quantitative vs qualitative research


So this much is obvious: quantitative research uses numbers and statistics to draw conclusions about large populations. You count something that is countable, and process results across the sample.

 

Qualitative methods are more elusive: however in general they revolve around collecting data from people about an experience. This could be how they used a service, how they felt about something, and could be verbal or written. But it is generally speech or talk, albeit with a variety of levels inferred above and below this (if they are truthful, or if what they say has deeper or hidden meaning). Rather than applying a statistical test to the data, a qualitative researcher must read/listen to the data and make an interpretation of what is being discussed, often hoping to discover patterns or contradictions.

 

Interpretation is done in both approaches: quantitative results are still examined in context (often compared with other numbers and data), and given a metric of significance such as a p-value or r-squared to assess if the results, or a part of them, are meaningful.

 

Finally, in general it is seen that a quantitative approach is a positivist paradigm, while qualitative methods fit better with constructionist or pragmatic paradigms (Savin-Baden and Major 2013). However, both are essentially attempting to model and sample something about the world in which we live so that we can simplify and understand it. And it’s not a case of one is better than the other: just like a hammer can’t be used to turn screws, or a screwdriver to hammer in nails, the different methods have different uses. Researchers should always make sure that the question comes first, and that is used to choose the methodology.

 

But you should also ask, is there a quantitative way to measure what your question is asking? If it’s something as simple as numbers of people, or a quantitative aspect like salary. While there are also quantitative measures of things like anxiety or pain that can be used as a proxy to make inferences across a large population. However, for detailed understanding of these issues and how they affect people, these metrics can be crude and don’t get to the detail of the lived experience.

 

However, choosing the right approach also depends on the how much is known about the research question and topic area. If you don’t know what the problems are in a field, you don’t know what questions to ask, or how to record the the answers.

 

I would argue that even in the physical sciences, qualitative research comes first, and sets questions to answer with quantitative methods. Quantitative research projects usually grow from qualitative observations of the physical world, such as 'I can see that ice seems to melt when it gets warm. At what temperature does ice melt?' or qualitative exploration of the existing literature to find things from other research that is surprising or unexplained.

 

In the classic high-school science experiment above, you would quantitatively measure the melting point of water by taking a sample. You don't try and melt all the ice in the world: you take one piece, and assume that other ice behaves in the same way. In both quantitative and qualitative research, sampling correctly is important. Just as only taking one small piece of impure ice will give you skewed results, so will only sampling one small part of a human population.

 

In quantitative research, because you are usually only sampling for one question at a time (i.e. temperature) it's best to have a large sample size. Especially when dealing with naturally variable, unrestricted variables (for example like a person's height) the data will tend to form a bell curve with a large majority of the answers in the middle, and a small number of outliers at either end. If we were sampling ice to melt, we might find that most ice melts around the same temperature, but very pure or dirty ice will have a slight difference. We would take the answer to be the statistical average, for the mean by adding up all the results and dividing by the sample size.

 

You could argue that the same is true for qualitative research. If you are asking people about their favourite ice cream, you'll get a better answer by asking a large number of people, right? Well this might not always be true. Firstly, just as with the ice melting experiment, sampling every piece of ice in the world will not add much more accuracy to your work but will be a lot more work. And with qualitative research, you are generally asking a much more complicated question for each person sampled, so the work increases exponentially as your sample size grows.

 


As your qualitative data grows, Quirkos can help you manage and make sense of it...

 

Remember, it's rare that qualitative research aims to give one definitive answer: it's more exploratory, and interested in the outlier cases just as much as the common ones. So in our qualitative research question 'What is your favourite ice cream' people may talk about gelato, sorbet or iced coffee. Are these really ice cream? One could argue that technically they are not, but if people consider them to be ice cream, and we want to know what to sell for desert at our restaurant, this becomes relevant. As a result of qualitative research, we usually learn to ask better questions 'What is your favourite frozen dessert?' might be a better question.

 

Now our qualitative research has helped us create a good piece of quantitative research. We can do a survey with a large sample size, and ask the question 'What is your favourite frozen dessert?' and give a list of options which are the most common answers from our qualitative research.

 

However, there can still be flaws with this approach. When answering a survey people don't always say what they mean, and you lose the context of their answers. In surveys there is primacy effect which means that people are lazy, and much more likely to tick the first answer in a list. In this case, the richness of our qualitative answers are lost. We don't know what context people are talking about (while walking along a beach, or in a restaurant or at home?) and we also loose the direct contact with the respondent so we can tell if they are lying or being sarcastic, and we can't ask follow on questions.

 

That's why qualitative research can still be useful as part of, or following quantitative research, for discovering ‘Why’ – understanding the results in the richness of lived experience. Often research projects will have a qualitative component – taking a subset of the the larger quantitative study and getting an in-depth qualitative insight with them.

 

There’s no shame in using a mixed methods approach if it is the most appropriate for the subject area. While there is often criticism over studies that ‘tack-on’ a small qualitative component, and don’t properly integrate or triangulate the types of results, this is a implementation rather than paradigm problem. But remember, it’s not a case of one approach vs another, there are no inheriently good or bad approaches. Methods should be appropriate to each task/question and should be servants to the researcher, not ruling them (Silverman 2013).

 

Quirkos is about as close as a pure qualitative software package as you can find. It's quick to learn, visual and keeps you close to the data. Our focus is on just doing qualitative coding and analysis well, and not to attempt  statistical analysis of qualitative data. We believe that for most qualitative researchers that's the right methodological approach. However, there is capacity for allow some mixed method analysis, so that you can filter results by demographic or other data.

 

The best way to see if Quirkos works for you is to give it a go! Download our one month free trial of the full version with no restrictions, and see if Quirkos works for your research paradigm.

 

 

Qualitative methods blog posts

qualitative methods

Articles on qualitative methods

 

 

This series aims to introduce qualitative methods and some of the main approaches in collecting qualitative data.

 

 

Why qualitative research?
There are lies, damn lies, and statistics It's easy to knock statistics for being misleading, or even misused to support spurious findings. In fact, there seems to be a growing backlash at the...

What is a Qualitative approach
The benefit of having tastier satsumas is difficult to quantify: to turn into a numerical, comparable value. This is essentially what qualitative work does: measure the unquantifiable quality of...

An overview of qualitative methods
There are a lot of different ways to collect qualitative data, and this article just provides a brief summary of some of the main methods used in qualitative research. Each one is an art in its own...

Thinking About Me: Reflexivity in science and qualitative research
Reflexivity is a process (and it should be a continuing process) of reflecting on how the researcher could be influencing a research project. In a traditional positivist research paradigm...

 

 

Qualitative Interviews

 

10 tips for semi-structured qualitative interviewing
Many qualitative researchers spend a lot of time interviewing participants, so here are some quick tips to make interviews go as smooth as possible: before, during and after! 1. Let your...

Designing a semi-structured interview guide for qualitative interviews
Interviews are a frequently used research method in qualitative studies. You will see dozens of papers that state something like We conducted n in-depth semi-structured interviews with...

 

 

Focus Groups

Considering and planning for qualitative focus groups
This is the first in a two-part series on focus groups. This week, we are looking at some of the why you might consider using them in a research project...

Tips for running effective focus groups
In the last blog article I looked at some of the justifications for choosing focus groups as a method in qualitative research. This week, we will focus on some practical tips to make sure that focus groups run smoothly...

 

 

Participatory Methods

Participatory Qualitative Analysis
Engaging participants in the research process can be a valuable and insightful endeavour, leading to researchers addressing the right issues, and asking the right questions. Many funding...

Participant diaries for qualitative research
I've written a little about this before, but I really love participant diaries! In qualitative research, you are often trying to understand the lives, experiences and motivations of...

 

 

Qualitative and mixed method surveys

Bringing survey data and mixed-method research into Quirkos
Later today we are releasing a small update for Quirkos, which adds an important feature users have been requesting: the ability to quickly bring in quantitative and qualitative data from any...

The importance of keeping open-ended qualitative responses in surveys
I once had a very interesting conversation at a MRS event with a market researcher from a major media company. He told me that they were increasingly ‘costing-out’ the qualitative open-ended questions from customer surveys...
 

How to set up a free online mixed methods survey
It's quick and easy to set up an on-line survey to collect feedback or research data in a digital format that means you can quickly get straight to analysing the data. Unfortunately, most...


 

Qualitative evaluations

Qualitative evaluations: methods, data and analysis
Evaluating programmes and projects are an essential part of the feedback loop that should lead to better services. In fact, programmes should be designed with evaluations in mind, to make sure that...

Using Quirkos for Systematic Reviews and Evidence Synthesis
Most of the examples the blog has covered so far have been about using Quirkos for research, especially with interview and participant text sources. However, Quirkos can take any text source you can...
 

Qualitative evidence for evaluations and impact assessments
For the last few months we have been working with SANDS Lothians, a local charity offering help and support for families who have lost a baby in miscarriage, stillbirth or soon after birth. They...


 

Sampling and sample sizes

Sampling considerations in qualitative research
Two weeks ago I talked about the importance of developing a recruitment strategy when designing a research project. This week we will do a brief overview of sampling for qualitative research...
 

Reaching saturation point in qualitative research
A common question from newcomers to qualitative research is, what's the right sample size How many people do I need to have in my project to get a good answer for my research...
 

Triangulation in qualitative research
Most qualitative research will be designed to integrate insights from a variety of data sources, methods and interpretations to build a deep picture. Triangulation is the term used to describe this comparison and meshing of different data...

 

 

Recording and Transcribing

Recording good audio for qualitative interviews and focus groups
Last week's blog post looked at the transcription process, and what's involved in getting qualitative interview or focus-group data transcribed. This week, we are going to step...
 

Transcribing your own qualitative data
In a previous blog article I talked about some of the practicalities and costs involved in using a professional transcribing service to turn your beautifully recorded qualitative interviews and...

Transcription for qualitative interviews and focus-groups
Audio and video give you a level of depth into your data that can't be conveyed by words alone, letting you hear hesitations, sarcasm, and nuances in delivery that can change your...


 

 

Tips for running effective focus groups

In the last blog article I looked at some of the justifications for choosing focus groups as a method in qualitative research. This week, we will focus on some practical tips to make sure that focus groups run smoothly, and to ensure you get good engagement from your participants.

 


1. Make sure you have a helper!

It’s very difficult to run focus groups on your own. If you are wanting to layout the room, greet people, deal with refreshment requests, check recording equipment is working, start video cameras, take notes, ask questions, let in late-comers and facilitate discussion it’s much easier with two or even three people for larger groups. You will probably want to focus on listening to the discussion, not have to take notes and problem solve at the same time. Having another facilitator or helper around can make a lot of difference to how well the session runs, as well as how much good data is recorded from it.

 


2. Check your recording strategy

Most people will record audio and transcribe their focus groups later. You need to make sure that your recording equipment will pick up everyone in the room, and also that you have a backup dictaphone and batteries! Many more tips in this blog post article. If you are planning to video the session, think this through carefully.

 

Do you have the right equipment? A phone camera might seem OK, but they usually struggle to record long sessions, and are difficult to position in a way that will show everyone clearly. Special cameras designed for gig and band practice are actually really good for focus groups, they tend to have wide-angle lenses and good microphones so you don’t need to record separate audio. You might also want to have more than one camera (in a round-table discussion, someone will always have their back to the camera. Then you will want to think about using qualitative analysis software like Transana that will support multiple video feeds.

 

You also need to make sure that video is culturally appropriate for your group (some religions and cultures don’t approve of taking images) and that it won’t make people nervous and clam up in discussion. Usually I find a dictaphone less imposing than a camera lens, but you then loose the ability to record the body language of the group. Video also makes it much easier to identify different speakers!

 


3. Consent and introductions

I always prefer to do the consent forms and participant information before the session. Faffing around with forms to sign at the start or end of the workshop takes up a lot of time best used for discussion, and makes people hurried to read the project information. E-mail this to people ahead of time, so at least they can just sign on the day, or bring a completed form with them. I really feel that participants should get the option to see what they are signing up for before they agree to come to a session, so they are not made uncomfortable on the day if it doesn't sound right for them. However, make sure there is an opportunity for people to ask any questions, and state any additional preferences, privately or in public.

 


4. Food and drink

You may decide not to have refreshments at all (your venue might dictate that) but I really love having a good spread of food and drink at a focus group. It makes it feel more like a party or family occasion than an interrogation procedure, and really helps people open up.

 

While tea, coffee and biscuits/cookies might be enough for most people, I love baking and always bring something home-baked like a cake or cookies. Getting to talk about and offer  food is a great icebreaker, and also makes people feel valued when you have spent the time to make something. A key part of getting good data from a good focus group is to set a congenial atmosphere, and an interesting choice of drinks or fruit can really help this. Don’t forget to get dietary preferences ahead of time, and consider the need for vegetarian, diabetic and gluten-free options.

 


5. The venue and layout

A lot has already been said about the best way to set out a focus group discussion (see Chambers 2002), but there are a few basic things to consider. First, a round or rectangle table arrangement works best, not lecture hall-type rows. Everyone should be able to see the face of everyone else. It’s also important not to have the researcher/facilitator at the head or even centre of the table. You are not the boss of the session, merely there to guide the debate. There is already a power dynamic because you have invited people, and are running the session. Try and sit yourself on the side as an observer, not director of the session.

 

In terms of the venue, try and make sure it is as quiet as possible, and good natural light and even high ceilings can help spark creative discussion (Meyers-Levy and Zhu 2007).

 


6. Set and state the norms

A common problem in qualitative focus group discussions is that some people dominate the debate, while others are shy and contribute little. Chambers (2002) just suggests to say at the beginning of the session this tends to happen, to make people conscious of sharing too much or too little. You can also try and actively manage this during the session by prompting other people to speak, go round the room person by person, or have more formal systems where people raise their hands to talk or have to be holding a stone. These methods are more time consuming for the facilitator and can stifle open discussion, so it's best to use them only when necessary.

 

You should also set out ground rules, attempting to create an open space for uncritical discussion. It's not usually the aim for people to criticise the view of others, nor for the facilitator to be seen as the leader and boss. Make these things explicit at the start to make sure there is the right atmosphere for sharing: one where there is no right or wrong answer, and everyone has something valuable to contribute.

 


7. Exercises and energisers

To prompt better discussion when people are tired or not forthcoming, you can use exercises such as card ranking exercises, role play exercises and prompts for discussion such as stories or newspaper articles. Chambers (2002) suggests dozens of these, as well as some some off-the-wall 'energizer' exercises: fun games to get people to wake up and encourage discussion. More on this in the last blog post article. It can really help to go round the room and have people introduce themselves with a fun fact, not just to get the names and voices on tape for later identification, but as a warm up.

 

Also, the first question, exercise or discussion point should be easy. If the first topic is 'How did you feel when you had cancer?' that can be pretty intimidating to start with. Something much simpler, such as 'What was hospital food like?' or even 'How was your trip here?' are topics everyone can easily contribute to and safely argue over, gaining confidence to share something deeper later on.

 


8. Step back, and step out

In focus groups, the aim is usually to get participants to discuss with each-other, not a series of dialogues with the facilitator. The power dynamics of the group need to reflect this, and as soon as things are set in motion, the researcher should try and intervene as little as possible – occasionally asking for clarification or to set things back on track. Thus it's also their role to help participants understand this, and allow the group discussion to be as co-interactive as possible.

 

“When group dynamics worked well the co-participants acted as co-
researchers taking the research into new and often unexpected directions and engaging in interaction which were both complementary (such as sharing common experiences) and argumentative” 
- Kitzinger 1994

 


9. Anticipate depth

Focus groups usually last a long time, rarely less than 2 hours, but even a half or whole day of discussion can be appropriate if there are lots of complex topics to discuss. It's OK to consider having participants do multiple focus groups if there is lots to cover, just consider what will best fit around the lives of your participants.

 

At the end of these you should find there is a lot of detailed and deep qualitative data for analysis. It can really help digesting this to make lots of notes during the session, as a summary of key issues, your own reflexive comments on the process, and the unspoken subtext (who wasn't sharing on what topics, what people mean when they say, 'you know, that lady with the big hair').


You may also find that qualitative analysis software like Quirkos can help pull together all the complex themes and discussions from your focus groups, and break down the mass of transcribed data you will end up with! We designed Quirkos to be very simple and easy to use, so do download and try for yourself...

 

 

 

Qualitative evaluations: methods, data and analysis

reports on a shelf

Evaluating programmes and projects are an essential part of the feedback loop that should lead to better services. In fact, programmes should be designed with evaluations in mind, to make sure that there are defined and measurable outcomes.

 

While most evaluations generally include numerical analysis, qualitative data is often used along-side the quantitative, to show richness of project impact, and put a human voice in the process. Especially when a project doesn’t meet targets, or have the desired level of impact, comments from project managers and service users usually give the most information into what went wrong (or right) and why.

 

For smaller pilot and feasibility projects, qualitative data is often the mainstay of the evaluation data, when numerical data wouldn’t reach statistical analysis, or when it is too early in a programme to measure intended impact. For example, a programme looking at obesity reductions might not be able to demonstrate a lower number of diabetes referrals at first, but qualitative insight in the first year or few months of the project might show how well messages from the project are being received, or if targeted groups are talking about changing their behaviour. When goals like this are long term (and in public health and community interventions they often are) it’s important to continuously assess the precursors to impact: namely engagement, and this is usually best done in a qualitative way.

 

So, what is best practice for qualitative evaluations? Fortunately, there are some really good guides and overviews that can help teams choose the right qualitative approach. Vaterlaus and Higgenbotham give a great overview of qualitative evaluation methods, while Professor Frank Vanclay talks at a wider level about qualitative evaluations, and innovative ways to capture stories. However, there was also a nice ‘tick-box’ style guide produced by the old Public Health Resource Unit which can still be found at this link. Essentially, the tool suggests 10 questions that can be used to assess the quality of a qualitative based-evaluation – really useful when looking at evaluations that come from other fields or departments.

 

But my contention is that the appraisal tool above is best implemented as a guide for producing qualitative evaluations. If you start by considering the best approach, how you are going to demonstrate rigour, choosing appropriate methods and recruitment, you’ll get a better report at the end of it. I’d like to discuss and expand on some of the questions used to assess the rigour of the qualitative work, because this is something that often worries people about qualitative research, and these steps help demystify good practice.

 

  1. The process: Start by planning the whole evaluation from the outset: What do you plan to do? All the rest will then fall into place.
     
  2. The research questions: what are they and why were these chosen? Are the questions going to give the evaluation the data it needs, and will the methods capture that correctly?
     
  3. Recruitment: who did you choose, and why? Who didn’t take part, and how did you find people? What gaps are there likely to be in representing the target group, and how can you compensate for this? Were there any ethical considerations, how was consent gained, and what was the relationship between the participants and the researcher(s)? Did they have any reason to be biased or not truthful?
     
  4. The data: how did you know that enough had been collected? (Usually when you are starting to hear the same things over and over – saturation) How was it recorded, transcribed, and was it of good quality? Were people willing to give detailed answers?
     
  5. Analysis: make sure you describe how it was done, and what techniques were used (such as discourse or thematic analysis). How does the report choose which quotes to reproduce, and are there contradictions reported in the data? What was the role of the researcher – should they declare a bias, and were multiple views sought in the interpretation of the data?
     
  6. Findings: do they meet the aims and research questions? If not, what needs to be done next time? Are there clear findings and action points, appropriate to improving the project?

 

Then the final step for me is the most important of all: SHARE! Don't let it end up on a dusty shelf! Evaluations are usually seen as a tedious but necessary internal process, but they can be so useful to people as case-studies and learning tools in organisations and groups you might never have thought of. This is especially true if there are things that went wrong, help someone in another local authority not make the same mistakes!

 

At the moment the best UK repositories of evaluations are based around health and economic benefits but that doesn’t stop you putting the report on your organisation’s website – if someone is looking for a similar project, search engines will do the leg work for you. That evaluation might save someone a lot of time and money, and it goes without saying, look for any similar work before you start a project, you might get some good ideas, and stop yourself falling into the same pit-holes!

 

Participatory analysis: closing the loop

In participatory research, we try to get away from the idea of researchers doing research on people, and move to a model where they are conducting research with people.

 

The movement comes partly from feminist critiques of epistemology, attacking the pervasive notion that knowledge can only be created by experienced academics, The traditional way of doing research generally disempowers people, as the researchers get to decide what questions to ask, how to interpret and present them, and even what topics are worthy of study in the first place. In participatory research the people who are the focus of the research are seen as the experts, rather than the researchers. At face value, this seems to make sense. After all, who knows more about life on a council estate: someone who has lived there for 20 years, or a middle-class outside researcher?

 

In participatory research, the people who are the subject of the study are often encouraged to be a much greater part of the process, active participants rather than aliens observed from afar. They know they are taking part in the research process, and the research is designed to give them input into what the study should be focusing on. The project can also use research methods that allow people to have more power over what they share, for example by taking photos of their environment, having open group discussions in the community, or using diaries and narratives in lieu of short questionnaires. Groups focused on developing and championing this work include the Participatory Geographies working group of the RGS/IBG, and the Institute of Development Studies at the University of Sussex.

 

This approach is becoming increasingly accepted in mainstream academia, and many funding bodies, including the NIHR, now require all proposals for research projects to have had patient or 'lay-person' involvement in the planning process, to ensure the design of the project is asking the right questions in an appropriate way. Most government funded projects will also stipulate that a summary of findings should be written in a non-technical, freely available format so that everyone involved and affected by the research can access it.

 

Engaging with analysis

Sounds great, right? In a transparent way, non-academics are now involved in everything: choosing which studies are the most important, deciding the focus, choosing the methods and collecting and contributing to the data.

 

But then what? There seems to be a step missing there, what about the analysis?

 

It could be argued that this is the most critical part of the whole process, where researchers summarise, piece together and extrapolate answers from the large mass of data that was collectively gathered. But far too often, this process is a 'black-box' conducted by the researchers themselves, with little if any input from the research participants. It can be a mystery to outsiders, how did researchers come to the particular findings and conclusions from all the different issues that the research revealed? What was discarded? Why was the data interpreted in this way?

 

This process is usually glossed over even in journal articles and final reports, and explaining the process to participants is difficult. Often this is a technical limitation: if you are conducting a muli-factor longitudinal study, the calculation of the statistical analysis is usually beyond all but the most mathematically minded academics, let alone the average Jo.

 

Yet this is also a problem in qualitative research, where participatory methods are often used. Between grounded theory, framework analysis and emergent coding, the approach is complicated and contested even within academia. Furthermore, qualitative analysis is a very lengthy process, with researchers reading and re-reading hundreds or thousands of pages of text: a prospect unappealing to often unpaid research participants.

 

Finally, the existing technical solutions don't seem to help. Software like Nvivo, often used for this type of analysis, is daunting for many researchers without training, and encouraging people from outside the field to try and use it, with all the training and licensing implications of this, makes for an effective brick wall. There are ways to make analysis engaging for everyone, but many research projects don't attempt participation at the analysis stage.

 

Intuitive software to the rescue?

By making qualitative analysis visual and engaging, Quirkos hopes to make participatory analysis a bit more feasible. Users don't require lengthy training, and everyone can have a go. They can make their own topics, analyse their own transcripts (or other people's), and individuals in a large community group can go away and do as little or as much as they like, and the results can be combined, with the team knowing who did what (if desired).

 

It can also become a dynamic group exercise, where with a tablet, large touch surface or projector, everyone can be 'hands on' at once. Rather than doing analysis on flip-charts that someone has to take away and process after the event, the real coding and analysis is done live, on the fly. Everyone can see how the analysis is building, and how the findings are emerging as the bubbles grow. Finally, when it comes to share the findings, rather than long spreadsheets of results, you get a picture – the bubbles tell the story and the issues.

 

Quirkos offers a way to practically and affordably facilitate proper end-to-end participatory research, and finally close the loop to make participation part of every stage in the research process.

 

 

10 tips for semi-structured qualitative interviewing

Many qualitative researchers spend a lot of time interviewing participants, so here are some quick tips to make interviews go as smooth as possible: before, during and after!

 

1. Let your participants choose the location

If you want your interviewees to be comfortable in sharing sometimes personal or sensitive information, make sure they can do it in a comfortable location. For some people, this might be their own house, or a neutral territory like a local cafe. Giving them the choice can help build trust, and gives the right impression: that you are accomodating them. However, make sure you make it clear that you need a relatively quiet location free from interruptions: a pub that plays loud music will not only stop you hearing each other, but usually makes recordings unusable!

 

2. Remember that they are helping you

Be polite and curtious, and be grateful to them for sharing their time and experiences. This always gets interviews off on the right foot. Also, try and think about participants motivations for taking part. Do they want the research to help others? Are they looking for a theraputic discussion? Do they just like a chat? Understanding this will help you guide the interview, and make sure you meet their expectations.

 

3. A conversation, not an interregation!

Interviews work best when they are a friendly dialogue: don't be afraid to start with some small talk, even when the tape is running. It turns a weird situation into a much more normal human experience, and starting with some easy 'starter for 10' questions helps people open up. Even a chatty "How did you hear about the project?" can gives you useful information.

 

4. Memorise the topic guide, but keep it to hand

Knowing all the questions in the topic guide can really help, so group them thematically, and memorise them as much as you can. It will really help the flow of information if you can segue seamlessly from one question to another relevant one. However, it's always useful to keep a print-out in front of you, not just for if you forget something, but also to make you seem more human, with a specific role. Joking about remembering all the questions is a great icebreaker, and it gives you something to look at other than the participant, to stop the session turning into a staring match!

 

5. Use open body language and encouraging cues

Face the participant in a friendly way, and nod or look sympathetic at the right times. Sometimes it's tempting for the interviewer to keep quiet during the responses, and not put in any normal encouraging noises like "Yeah", "Hmm" or "Right" knowing how odd these read in a transcript. But these are important cues that people use to know when to keep talking, so if you are going to drop them, make sure you make positive eye contact, and nod at the right times instead!

 

Quirkos - simple qualitative analysis software

 

6. Write notes, even if you don't use them

It always helps me to scribble down some one-word notes on the topic guide when you are doing an interview: first of all it helps focus my thoughts, and remind me about interesting things that the participant mentioned that I want to go back to. But it also helps show you are listening, and makes sure if the recording goes wrong, there is something to fall back on.

 

7. Write-up the interivew as soon as you finish

Just take 15 minutes after each interview to reflect: the main points that came up, how open the respondent was, any context or distractions that might have impared the flow. This helps you think about things to do better in the next interview, and will help you later to remember each interview.

 

8. Return to difficult issues

If a particular topic is clearly a difficult question (either emotionally, or just because someone can't remember) don't be afraid to leave the topic and come back to it later, asking in a different way. It can really help recall to have a break talking about something easier, and then approach the issue sideways later on.

 

9. Ask stupid questions

Don't assume you know anything. In these kinds of interviews, it's usually not about getting the right answer, but getting the respondent's view or opinion. Asking 'What do you mean by family?' is really useful if you discover someone has adopted children, step-sisters and a beloved family dog that all share the house. Don't make any assumptions, let people tell you what they mean. Even if you have to ask something that makes you sound ignorant on a specialist subject, you could discover that someone didn't know the difference between their chemotherapy and radiotherapy.

 

10. Say thank you

And follow up: send a nice card after the interview, don't be like a date they never hear from again! Also, try and make sure they get a summary of the findings of the study they took part in. It's not just about being nice, but to make sure people have a good experience as a research subject, and will want to be involved in the next project that comes along, which might be yours or mine!

 

I hope these tips have been hopeful, don't forget Qurikos makes your transcribed interviews easy to analyse, as well as a visual and engaging process. Find out more and download a free trial from our website. Our blog is updated with articles like this every week, and you can hear about it first by following our Twitter feed @quirkossoftware.

 

 

An overview of qualitative methods

There are a lot of different ways to collect qualitative data, and this article just provides a brief summary of some of the main methods used in qualitative research. Each one is an art in its own right, with various different techniques, definitions, approaches and proponents.

More on each one will follow in later articles, and it’s worth remembering that these need to be paired with the right questions, sampling, and analysis to get good results.

Interviews

Possibly the richest, and most powerful tool: talking to someone directly. The classic definition is “conversations with a purpose“, the idea being that there is something you are interested in, you ask questions about it, and someone gives useful responses.

There are many different styles for example how structured your questions are (this paper has a wonderful and succinct overview in the introduction). These can range from a rigid script where you ask the same questions every time, or completely open discussion, where the researcher and respondent have freedom to shape the conversation. A common middle ground are semi-structured interviews, which often have a topic guide, listing particualar issues to discuss, but will allow questions for clarification, or to follow up on an interesting tangent.

Participant Observation

Often the remit of ethnography or sociology, participant observation usually involves watching, living or even participating in the daily life of research subjects. However, it can also involve just watching people in a certain setting, such as a work meeting, or using a supermarket.

This is probably the most time intensive and potentially problematic method, as it can involve weeks or even years of placement for a researcher, often on their own. However, it does produce some of the richest data, as well as a level of depth that can really help explain complex issues. This chapter is a fine starting point.

Focus groups

A common method used in market research, where a researcher leads a group discussion on a particular topic. However, it is also a powerful tool for social researchers, especially when looking at group dynamics, or the reactions of particular groups of people. It’s obviously important to consider who is chosen for the group, and how the interactions of people in the group affect the outcome (although this might be what you are looking for).

It’s usually a quicker and cheaper way of gauging many reactions and opinions, but requires some skill in the facilitator to make sure everyone’s voice is being heard, and that people stay on track. Also a headache for any transcribers who have to identify different voices from muffled audio recordings!

Participant Diaries

Getting people to write a diary for a research project is a very useful tool, and is commonly used in looking at taboo behaviours such as drug use or sexuality, not just because researchers don’t have to ask difficult questions face-to-face, but that data can be collected over a long period of time. If you are trying to find out how often a particular behaviour occurs, a daily or weekly record is likely to be more accurate than asking someone in a single interview (as in the studies above).

There are other benefits to the diary method: not least that the participant is in control. They can share as much or as little as they like, and only on topics they wish to. It can also be theraputic for some people, and is more time flexible. Diaries can be paper based, electronic, or even on a voice recorder if there are literacy concerns. However, researchers will probably need to talk to people at the beginning and end of the process, and give regular reminders.

Surveys

Probably one of the most common qualitative methods are the open ended questions on surveys, usually by post, on-line, or ‘guided’ by someone with a clipboard. Common challenges here are

  • Encouraging people to write more than one word, but less than an essay
  • Setting questions carefully so they are clear, but not leading
  • Getting a good response rate and
  • Knowing who has and hasn’t responded

The final challenge is to make sure the responses are useful, and integrating them with the rest of the project, especially quantitative data.

Field notes

Sometimes the most overlooked, but most vaulable source of information can be the notes and field diaries of researchers themselves. These can include not just where and when people did interviews or observations, but crucial context, like the people who refused to take part, and whether a interviewee was nervous. It need not just be for ethnographers doing long field work, it can be very helpful in organising thoughts and work in smaller projects with multiple researchers.

As part of a reflexive method, it might contain comments and thoughts from the researcher, so there can be a risk of autobiographical overindulgence. It is also not easy to integrate ‘data’ from a research diary with other sources of information when writing up a project for a particular output.

 

This is just a whistle-stop introduction, but more on each of these to follow…