Sampling considerations in qualitative research

sampling crowd image by https://www.flickr.com/photos/jamescridland/613445810/in/photostream/

 

Two weeks ago I talked about the importance of developing a recruitment strategy when designing a research project. This week we will do a brief overview of sampling for qualitative research, but it is a huge and complicated issue. There’s a great chapter ‘Designing and Selecting Samples’ in the book Qualitative Research Practice (Ritchie et al 2013) which goes over many of these methods in detail.

 

Your research questions and methodological approach (ie grounded theory) will guide you to the right sampling methods for your study – there is never a one-size-fits-all approach in qualitative research! For more detail on this, especially on the importance of culturally embedded sampling, there is a well cited article by Luborsky and Rubinstein (1995). But it’s also worth talking to colleagues, supervisors and peers to get advice and feedback on your proposals.

 

Marshall (1996) briefly describes three different approaches to qualitative sampling: judgement/purposeful sampling, theoretical sampling and convenience sampling.

 

But before you choose any approach, you need to decide what you are trying to achieve with your sampling. Do you have a specific group of people that you need to have in your study, or should it be representative of the general population? Are you trying to discover something about a niche, or something that is generalizable to everyone? A lot of qualitative research is about a specific group of people, and Marshall notes:
“This is a more intellectual strategy than the simple demographic stratification of epidemiological studies, though age, gender and social class might be important variables. If the subjects are known to the research, they may be stratified according to known public attitudes or beliefs.”

 

Broadly speaking, convenience, judgement and theoretical sampling can be seen as purposeful – deliberately selecting people of interest in some way. However, randomly selecting people from a large population is still a desirable approach in some qualitative research. Because qualitative studies tend to have a small sample size due to the in-depth nature of engagement with each participant, this can have an impact if you want a representative sample. If you randomly select 15 people, you might by chance end up with more women than men, or a younger than desired sample. That is why qualitative studies may use a little bit of purposeful sampling, finding people to make sure the final profile matches the desired sampling frame. For much more on this, check out the last blog post article on recruitment.

 

Sample size will often also depend on conceptual approach: if you are testing a prior hypothesis, you may be able to get away with a smaller sample size, while a grounded theory approach to develop new insights might need a larger group of respondents to test that the findings are applicable. Here, you are likely to take a ‘theoretical sampling’ approach (Glaser and Strauss 1967) where you specifically choose people who have experiences that would contribute to a theoretical construct. This is often iterative, in that after reviewing the data (for theoretical insights) the researcher goes out again to find other participants the model suggests might be of interest.

 

The convenience sampling approach which Marshal mentions as being the ‘least rigorous technique’ is where researchers target the most ‘easily accessible’ respondents. This could even be friends, family or faculty. This approach can rarely be methodologically justified, and is unlikely to provide a representative sample. However, it is endemic in many fields, especially psychology, where researchers tend to turn to easily accessible psychology students for experiments: skewing the results towards white, rich, well-educated Western students.

 

Now we turn to snowball sampling (Goodman 1961). This is different from purposeful sampling in that new respondents are suggested by others. In general, this is most suited to work with ‘marginalised or hard-to-reach’ populations, where responders are not often forthcoming (Sadler et al 2010). For example, people may not be open about their drug use, political views or living with stigmatising conditions, yet often form closely connected networks. Thus, by gaining trust with one person in the group, others can be recommended to the researcher. However, it is important to note the limitations with this approach. Here, there is the risk of systemic bias: if the first person you recruit is not representive in some way, their referrals may not be either. So you may be looking at people living with HIV/AIDS, and recruit through a support group that is formed entirely of men: they are unlikely to suggest women for the study.

 

For these reasons there are limits to the generalisability and appropriateness of snowball sampling for most subjects of inquiry, and it should not be taken as an easy fix. Yet while many practitioners explain the limitations with snowball research, it can be very well suited for certain kinds of social and action research, this article by Noy (2008) outlines some of the potential benefits to power relations and studying social networks.

 

Finally, there is the issue of sample size and ‘saturation’. This is when there is enough data collected to confidently answer the research questions. For a lot of qualitative research this means collected and coded data as well, especially if using some variant of grounded theory. However, saturation is often a source of anxiety for researchers: see for example the amusingly titled article “Are We There Yet?” by Fusch and Ness (2015). Unlike quantitative studies where a sample size can be determined by the desired effect size and confidence interval in a chosen statistical test, it is more difficult to put an exact number on the right number of participant responses. This is especially because responses are themselves qualitative, not just numbers in a list: so one response may be more data rich than another.

 

While a general rule of thumb would indicate there is no harm in collecting more data than is strictly necessary, there is always a practical limitation, especially in resource and time constrained post-graduate studies. It can also be more difficult to recruit than anticipated, and many projects working with very specific or hard-to-reach groups can struggle to find a large enough sample size. This is not always a disaster, but may require a re-examination of the research questions, to see what insights and conclusions are still obtainable.

 

Generally, researchers should have a target sample size and definition of what data saturation will look like for their project before they begin sampling and recruitment. Don’t forget that qualitative case studies may only include one respondent or data point, and in some situations that can be appropriate. However, getting the sampling approach and sample size right is something that comes with experience, advice and practice.

 

As I always seem to be saying in this blog, it’s also worth considering the intended audience for your research outputs. If you want to publish in a certain journal or academic discipline, it may not be responsive to research based on qualitative methods with small or ‘non-representative’ samples. Silverman (2013 p424) mentions this explicitly with examples of students who had publications rejected for these reasons.

 

So as ever, plan ahead for what you want to achieve for your research project, the questions you want to answer, and work backwards to choose the appropriate methodology, methods and sample for your work. Also, check the companion article about recruitment, most of these issues need to be considered in tandem.

 

Once you have your data, Quirkos can be a great way to analyse it, whether your sample size has one or dozens of respondents! There is a free trial and example data sets to see for yourself if it suits your way of working, and much more information in these pages. We also have a newly relaunched forum, with specific sections on qualitative methodology if you wanted to ask questions, or comment on anything raised in this blog series.

 

 

Recruitment for qualitative research

Recuriting qualitative participants

 

You’ll find a lot of information and debate about sampling issues in qualitative research: discussions over ‘random’ or ‘purposeful’ sampling, the merits and pitfalls of ubiquitous ‘snowball’ sampling, and unending questions about sample size and saturation. I’m actually going to address most of these in the next blog post, but wanted to paradoxically start by looking at recruitment. What’s the difference, and why think about recruitment strategies before sampling?

 

Well, I’d argue that the two have to be considered together, but recruitment tends to be a bit of an afterthought and is so rarely detailed in journal articles (Arcury and Quandt 1999) I feel it merits its own post. In fact, there is a great ONS document about sampling, but it only has one sentence on advice for respondent recruitment: “The method of respondent recruitment and its effectiveness is also an important part of the sampling strategy”. Indeed!

 

When we talk about recruitment, we are considering the way we actually go out and ask people to take part in a research study. The sample frame is how we choose what groups of people and how many to approach, but there are huge practical problems in implementing our chosen sampling method that can be dealt with by writing a comprehensive recruitment strategy.

 

This might sound a bit dull, but it’s actually kind of fun – and the creation of such a strategy for your qualitative research project is a really good thought exercise, helping you plan and later acknowledge shortcomings in what actually happened. Essentially, think of this process as how you will market and advertise your research project to potential participants.

 

Sometimes there is a shifting dynamic between sampling and recruitment. Say we are doing random sampling from numbers in a phone book, a classic ‘random’ technique. The sampling process is the selection of x number of phone numbers to call. The recruitment is the actually calling and asking someone to take part in the research. Now, obviously not everyone is going to answer the phone, or want to answer any questions. So you then have a list of recruited people, which you might actually want to sample from again to make a representative sample. If you found out everyone that answered the phone was retired and over 60, but you wanted a wider age profile, you will need to refactor from your recruited sample.

 

But let’s think about this again. Why could it be that everyone who consented to take part in our study was retired? Well, we used numbers from the phone book, and called during the day. What effect might this have? Numbers in the phone book tend to be people who have been resident in one place for a long time, many students and young people just have mobiles, and if we call during the day, we will not get answers from most people who work. This illustrates the importance of carefully considering the recruitment strategy: although we chose a good random sampling technique, our strategy of making phone calls during the day has already scuppered our plans.

 

How about another example: recruitment through a poster advertising the study. Many qualitative studies aren’t looking for very large number of respondents, but are targeting a very specific sample. In this example, maybe it’s people who have visited their doctor in the last 6 months. Sounds like a poster in the waiting room of the local GP surgery would work well. What are the obvious limitations here?

 

simple qualitative analysis software from quirkos

 

First of all, people who see the poster will probably have visited the GP (since they are in that location), however, it actually only would recruit people who are currently receiving treatment. People who had been in the previous 6 months but didn’t need to go back again, or had such a horrible experience they never returned, will not see our poster and don’t have a chance to be recruited. Both of these will skew the sample of respondents in different ways.

 

In some ways this is inevitable. Whichever sampling technique and recruitment strategy we adopt, some people will not hear about the study or want to take part. However, it is important to be conscious of not just who is being sampled, but who is left out, and the likely effect this has on our sample and consequently our findings. For example our approach here probably means we oversample people who have chronic conditions requiring frequent treatment, and undersample people who hate their doctor. It’s not necessarily a disaster, but just like making a reflexivity statement about our own biases, we must be forthright about the sampling limitations and consider them when analysing and writing conclusions.

 

For these reasons, it’s often desirable to have multiple and complementary recruitment strategies, so that one makes up for deficiencies in the other. So a poster in the waiting room is great, but maybe we can get a list of everyone registered at the surgery, so we can also contact people not currently seeking treatment. This would be wonderful, but in the real world, we might hit problems with the surgery not being interested in the study, not able to release that information for confidentiality reasons, and the huge extra time such a process would require.

 

That’s why I see a recruitment strategy as a practical battle plan that tries to consider the limitations and realities of engaging with the real world. You can also start considering seemingly small things that can have a huge impact on successful recruitment:


• The design of the poster
• The wording of invitation letters
• The time of day you make contact (not just by phone, but don’t e-mail first thing on a Monday morning!)
• Any incentives, and how appropriate they are
• Data protection issues
• Winning the support of ‘gatekeepers’ who control access to your sample
• Timescales
• Cost (especially if you are printing hundreds of letters of flyers)
• Time and effort required to find each respondent
• And many more…


For a more detailed discussion, there’s a great article by Newington and Metcalfe (2014) specifically on influencing factors for recruitment in qualitative research.

 

Finally, I want to reiterate the importance of trying to record who has not been recruited and why. If you are directly contacting a few dozen respondents by phone or e-mail, this is easy to keep track of: you know exactly who has declined or not responded, likely reasons why and probably some demographic details.

 

However, think about the poster example. Here, we will be lucky if 1% of people that come through the surgery contact us to take part in the study. Think through these classic marketing stages: they have to see the poster, think it’s relevant to them, want to engage, and then reach out to contact you. There will be huge losses at each of those stages, and you don’t know who these people are or why they didn’t take part. This makes it very difficult in this kind of study to know the bias of your final sample: we can guess (busy people, those who aren’t interested in research) but we don’t know for sure.

 

Response rates vary greatly by method: by post 25% is really good, direct contact much higher, posters and flyers below 10%. However, you can improve these rates with careful planning, by considering carefully who will engage and why, and making it a good prospect to take part: describe the aims of the research, compensate time, and explain the proposed benefits. But you also need to take an ethical approach, don’t coerce, and make promises you can’t keep. Check out the recruitment guidelines drawn up by the Association for Qualitative Research.

 

My personal experience tells me that most people who engage with qualitative research are lovely! They want to help if they can, and love an opportunity to talk about themselves and have their voice heard. Just be aware of what kinds of people end up being your respondents, and make sure you acknowledge the possibility of hidden voices from people who don’t engage for their own reasons.

 

Once you get to your analysis, don't forget to try Quirkos for free, and see how our easy-to-use software can make a real qualitative difference to your research project! To keep up to date with new blog articles on this, and other qualitative research topics, follow our Twitter feed: twitter.com/quirkossoftware.