Recruitment for qualitative research

Recuriting qualitative participants


You’ll find a lot of information and debate about sampling issues in qualitative research: discussions over ‘random’ or ‘purposeful’ sampling, the merits and pitfalls of ubiquitous ‘snowball’ sampling, and unending questions about sample size and saturation. I’m actually going to address most of these in the next blog post, but wanted to paradoxically start by looking at recruitment. What’s the difference, and why think about recruitment strategies before sampling?


Well, I’d argue that the two have to be considered together, but recruitment tends to be a bit of an afterthought and is so rarely detailed in journal articles (Arcury and Quandt 1999) I feel it merits its own post. In fact, there is a great ONS document about sampling, but it only has one sentence on advice for respondent recruitment: “The method of respondent recruitment and its effectiveness is also an important part of the sampling strategy”. Indeed!


When we talk about recruitment, we are considering the way we actually go out and ask people to take part in a research study. The sample frame is how we choose what groups of people and how many to approach, but there are huge practical problems in implementing our chosen sampling method that can be dealt with by writing a comprehensive recruitment strategy.


This might sound a bit dull, but it’s actually kind of fun – and the creation of such a strategy for your qualitative research project is a really good thought exercise, helping you plan and later acknowledge shortcomings in what actually happened. Essentially, think of this process as how you will market and advertise your research project to potential participants.


Sometimes there is a shifting dynamic between sampling and recruitment. Say we are doing random sampling from numbers in a phone book, a classic ‘random’ technique. The sampling process is the selection of x number of phone numbers to call. The recruitment is the actually calling and asking someone to take part in the research. Now, obviously not everyone is going to answer the phone, or want to answer any questions. So you then have a list of recruited people, which you might actually want to sample from again to make a representative sample. If you found out everyone that answered the phone was retired and over 60, but you wanted a wider age profile, you will need to refactor from your recruited sample.


But let’s think about this again. Why could it be that everyone who consented to take part in our study was retired? Well, we used numbers from the phone book, and called during the day. What effect might this have? Numbers in the phone book tend to be people who have been resident in one place for a long time, many students and young people just have mobiles, and if we call during the day, we will not get answers from most people who work. This illustrates the importance of carefully considering the recruitment strategy: although we chose a good random sampling technique, our strategy of making phone calls during the day has already scuppered our plans.


How about another example: recruitment through a poster advertising the study. Many qualitative studies aren’t looking for very large number of respondents, but are targeting a very specific sample. In this example, maybe it’s people who have visited their doctor in the last 6 months. Sounds like a poster in the waiting room of the local GP surgery would work well. What are the obvious limitations here?


simple qualitative analysis software from quirkos


First of all, people who see the poster will probably have visited the GP (since they are in that location), however, it actually only would recruit people who are currently receiving treatment. People who had been in the previous 6 months but didn’t need to go back again, or had such a horrible experience they never returned, will not see our poster and don’t have a chance to be recruited. Both of these will skew the sample of respondents in different ways.


In some ways this is inevitable. Whichever sampling technique and recruitment strategy we adopt, some people will not hear about the study or want to take part. However, it is important to be conscious of not just who is being sampled, but who is left out, and the likely effect this has on our sample and consequently our findings. For example our approach here probably means we oversample people who have chronic conditions requiring frequent treatment, and undersample people who hate their doctor. It’s not necessarily a disaster, but just like making a reflexivity statement about our own biases, we must be forthright about the sampling limitations and consider them when analysing and writing conclusions.


For these reasons, it’s often desirable to have multiple and complementary recruitment strategies, so that one makes up for deficiencies in the other. So a poster in the waiting room is great, but maybe we can get a list of everyone registered at the surgery, so we can also contact people not currently seeking treatment. This would be wonderful, but in the real world, we might hit problems with the surgery not being interested in the study, not able to release that information for confidentiality reasons, and the huge extra time such a process would require.


That’s why I see a recruitment strategy as a practical battle plan that tries to consider the limitations and realities of engaging with the real world. You can also start considering seemingly small things that can have a huge impact on successful recruitment:

• The design of the poster
• The wording of invitation letters
• The time of day you make contact (not just by phone, but don’t e-mail first thing on a Monday morning!)
• Any incentives, and how appropriate they are
• Data protection issues
• Winning the support of ‘gatekeepers’ who control access to your sample
• Timescales
• Cost (especially if you are printing hundreds of letters of flyers)
• Time and effort required to find each respondent
• And many more…

For a more detailed discussion, there’s a great article by Newington and Metcalfe (2014) specifically on influencing factors for recruitment in qualitative research.


Finally, I want to reiterate the importance of trying to record who has not been recruited and why. If you are directly contacting a few dozen respondents by phone or e-mail, this is easy to keep track of: you know exactly who has declined or not responded, likely reasons why and probably some demographic details.


However, think about the poster example. Here, we will be lucky if 1% of people that come through the surgery contact us to take part in the study. Think through these classic marketing stages: they have to see the poster, think it’s relevant to them, want to engage, and then reach out to contact you. There will be huge losses at each of those stages, and you don’t know who these people are or why they didn’t take part. This makes it very difficult in this kind of study to know the bias of your final sample: we can guess (busy people, those who aren’t interested in research) but we don’t know for sure.


Response rates vary greatly by method: by post 25% is really good, direct contact much higher, posters and flyers below 10%. However, you can improve these rates with careful planning, by considering carefully who will engage and why, and making it a good prospect to take part: describe the aims of the research, compensate time, and explain the proposed benefits. But you also need to take an ethical approach, don’t coerce, and make promises you can’t keep. Check out the recruitment guidelines drawn up by the Association for Qualitative Research.


My personal experience tells me that most people who engage with qualitative research are lovely! They want to help if they can, and love an opportunity to talk about themselves and have their voice heard. Just be aware of what kinds of people end up being your respondents, and make sure you acknowledge the possibility of hidden voices from people who don’t engage for their own reasons.


Once you get to your analysis, don't forget to try Quirkos for free, and see how our easy-to-use software can make a real qualitative difference to your research project! To keep up to date with new blog articles on this, and other qualitative research topics, follow our Twitter feed:



Designing a semi-structured interview guide for qualitative interviews

clipboard by wikidave


Interviews are a frequently used research method in qualitative studies. You will see dozens of papers that state something like “We conducted n in-depth semi-structured interviews with key informants”. But what exactly does this mean? What exactly counts as in-depth? How structured are semi-structured interviews?


The term “in-depth” is defined fairly vaguely in the literature: it generally means a one-to-one interview on one general topic, which is covered in detail. Usually these qualitative interviews last about an hour, although sometimes much longer. It sounds like two people having a discussion, but there are differences in the power dynamics, and end goal: for the classic sociologist Burgess (2002) these are “conversations with a purpose”.


Qualitative interviews generally differ from quantitative survey based questions in that they are looking for a more detailed and nuanced response. They also acknowledge there is no ‘one-size fits all’, especially when asking someone to recall a personal narrative about their experiences. Instead of a fixed “research protocol” that asks the same question to each respondent, most interviewees adopt a more flexible approach. However there is still a need “ ensure that the same general areas of information are collected from each interviewee; this provides more focus than the conversational approach, but still allows a degree of freedom and adaptability in getting information from the interviewee” –MacNamara (2009).


Turner (2010) (who coincidentally shares the same name as me) describes three different types of qualitative interview; Informal Conversation, General Interview Guide, and Standardised Open-Ended. These can be seen as a scale from least to most structured, and we are going to focus on the ‘interview guide’ approach, which takes a middle ground.


An interview guide is like a cheat-sheet for the interviewer – it contains a list of questions and topic areas that should be covered in the interview. However, these are not to be read verbatim and in order, in fact they are more like an aide-mémoire. “Usually the interviewer will have a prepared set of questions but these are only used as a guide, and departures from the guidelines are not seen as a problem but are often encouraged” – Silverman (2013). That way, the interviewer can add extra questions about an unexpected but relevant area that emerges, and sections that don’t apply to the participant can be negated.


So what do these look like, and how does one go about writing a suitable semi-structured interview guide? Unfortunately, it is rare in journal articles for researchers to share the interview guide, and it’s difficult to find good examples on the internet. Basically they look like a list of short questions and follow-on prompts, grouped by topic. There will generally be about a dozen. I’ve written my fair share of interview guides for qualitative research projects over the years, either on my own or with the collaboration of colleagues, so I’m happy to share some tips.


Questions should answer your research questions
Your research project should have one or several main research questions, and these should be used to guide the topics covered in the interviews, and hopefully answer the research questions. However, you can’t just ask your respondents “Can the experience of male My Little Pony fans be described through the lens of Derridean deconstruction?”. You will need to break down your research into questions that have meaning for the participant and that they can engage with. The questions should be fairly informal and jargon free (unless that person is an expert in that field of jargon), open ended - so they can’t be easily answered with a yes or no, and non-leading so that respondents aren’t pushed down a certain interpretation.



Link to your proposed analytical approach
The questions on your guide should also be constructed in such a way that they will work well for your proposed method of analysis – which again you should already have decided. If you are doing narrative analysis, questions should be encouraging respondents to tell their story and history. In Interpretative Phenomenological Analysis you may want to ask more detail about people’s interpretations of their experiences. Think how you will want to analyse, compare and write up your research, and make sure that the questioning style fits your own approach.



Specific ‘Why’ and prompt questions
It is very rare in semi-structured interviews that you will ask one question, get a response, and then move on to the next topic. Firstly you will need to provide some structure for the participant, so they are not expected (or encouraged) to recite their whole life story. But on the other level, you will usually want to probe more about specific issues or conditions. That is where the flexible approach comes in. Someone might reveal something that you are interested in, and is relevant to the research project. So ask more! It’s often useful in the guide to list a series of prompt words that remind you of more areas of detail that might be covered. For example, the question “When did you first visit the doctor?” might be annotated with optional prompts such as “Why did you go then?”, “Were you afraid?” or “Did anyone go with you?”. Prompt words might reduce this to ‘Why THEN / afraid / with someone’.



Be flexible with order
Generally, an interview guide will be grouped into several topics, each with a few questions. One of the most difficult skills is how to segue from one topic or question to the next, while still seeming like a normal conversation. The best way to manage this is to make sure that you are always listening to the interviewee, and thinking at the same time about how what they are saying links to other discussion topics. If someone starts talking about how they felt isolated visiting the doctor, and one of your topics is about their experience with their doctor, you can ask ‘Did you doctor make you feel less isolated?’. You might then be asking about topic 4, when you are only on topic 1, but you now have a logical link to ask the more general written question ‘Did you feel the doctor supported you?’. The ability to flow from topic to topic as the conversation evolves (while still covering everything on the interview guide) is tricky, and requires you to:



Know your guide backwards - literally
I almost never went into an interview without a printed copy of the interview guide in front of me, but it was kind of like Dumbo’s magic feather: it made me feel safe, but I didn’t really need it. You should know everything on your interview guide off by heart, and in any sequence. Since things will crop up in unpredictable ways, you should be comfortable asking questions in different orders to help the conversational flow. Still, it’s always good to have the interview guide in front of you; it lets you tick off questions as they are asked (so you can see what hasn’t been covered), is space to write notes, and also can be less intimidating for the interviewee, as you can look at your notes occasionally rather than staring them in the eye all the time.


Try for natural conversation
Legard, Keegan and Ward (2003) note that “Although a good in-depth interview will appear naturalistic, it will bear little resemblance to an everyday conversation”. You will usually find that the most honest and rich responses come from relaxed, non-combative discussions. Make the first question easy, to ease the participant into the interview, and get them used to the question-answer format. But don’t let it feel like a tennis match, where you are always asking the questions. If they ask something of you, reply! Don’t sit in silence: nod, say ‘Yes’, or ‘Of course’ every now and then, to show you are listening and empathising like a normal human being. Yet do be careful about sharing your own potentially leading opinions, and making the discussion about yourself.



Discuss with your research team / supervisors
You should take the time to get feedback and suggestions from peers, be they other people on your research project, or your PhD supervisors. This means preparing the interview guide well in advance of your first interview, leaving time for discussion and revisions. Seasoned interviewers will have tips about wording and structuring questions, and even the most experienced researcher can benefit from a second opinion. Getting it right at this stage is very important, it’s no good discovering after you’ve done all your interviews that you didn’t ask about something important.



Adapting the guide
While these are semi-structured interviews, in general you will usually want to cover the same general areas every time you do an interview, no least so that there is some point of comparison. It’s also common to do a first few interviews and realise that you are not asking about a critical area, or that some new potential insight is emerging (especially if you are taking a grounded theory approach). In qualitative research, this need not be a disaster (if this flexibility is methodologically appropriate), and it is possible to revise your interview guide. However, if you do end up making significant revisions, make sure you keep both versions, and a note of which respondents were interviewed with each version of the guide.



Test the timing
Inevitably, you will not have exactly the same amount of time for each interview, and respondents will differ in how fast they talk and how often they go off-topic! Make sure you have enough questions to get the detail you need, but also have ‘lower priority’ questions you can drop if things are taking too long. Test the timing of your interview guide with a few participants, or even friends before you settle on it, and revise as necessary. Try and get your interview guide down to one side of paper at the most: it is a prompt, not an encyclopaedia!


Hopefully these points will help demystify qualitative interview guides, and help you craft a useful tool to shape your semi-structured interviews. I’d also caution that semi-structured interviewing is a very difficult process, and benefits majorly from practice. I have been with many new researchers who tend to fall back on the interview guide too much, and read it verbatim. This generally leads to closed-off responses, and missed opportunities to further explore interesting revelations. Treat your interview guide as a guide, not a gospel, and be flexible. It’s extra hard, because you have to juggle asking questions, listening, choosing the next question, keeping the research topic in your head and making sure everything is covered – but when you do it right, you’ll get rich research data that you will actually be excited to go home and analyse.



Don’t forget to check out some of the references above, as well as the myriad of excellent articles and textbooks on qualitative interviews. There’s also Quirkos itself, software to help you make the research process engaging and visual, with a free trial to download of this innovative tool. We also have a rapidly growing series of blog post articles on qualitative interviews. These now include 10 tips for qualitative interviewing, transcribing qualitative interviews and focus groups, and how to make sure you get good recordings. Our blog is updated with articles like this every week, and you can hear about it first by following our Twitter feed @quirkossoftware.



An early spring update on Quirkos for 2016

spring snowdrops


About this time last year, I posted an update on Quirkos development for the next year. Even though February continues to be cold and largely snow-drop free in Scotland, why not make it a tradition?!


It’s really amazing how much Quirkos has grown over the last 18 months since our first release. We now have hundreds of users in more than 50 universities across the world. The best part of this is that we now get much more feedback and suggestions from qualitative researchers who are using Quirkos for different projects. Although we have always had a ‘road-map’ for developing new features for Quirkos, it’s been an aim to keep that flexible so we adapt to people’s needs.


We are planning a new update for Quirkos (free of course) for the end of March 2016. This version (1.4) will be a fairly major upgrade, but as ever will be released at the same time for Windows, Mac and Linux, with identical features and compatibility across all three.


The most significant improvement will be speed. Although v1.3 did improve this a little, it was not enough. The underlying ‘engine’ for coding and highlights was laggy and slow with large projects, and required complete rewriting from scratch. It has justifiably been the biggest source of criticism so far about Quirkos, but we hope this will now remove the last thing holding many users back. This has taken months, which is why this release is a little later than our typical quarterly updates. However, the difference so far is amazing: a near 10 fold increase in speed when loading, coding and editing sources. Although the interface will still look the same, everyone will notice the under-the-hood difference in small and large projects alike.


There will also be a few minor bug fixes in this release. We had reports that when moving encrypted projects between Windows and Mac, passwords were not accepted. We’ve fixed this issue, and a few others that people have reported. There are also several small improvements suggested by users that should make exploring the data easier. So please always e-mail us with bugs or suggestions, everything reported gets investigated, and we try and fix issues as soon as we can!


We will be sending the new version out to an international group of beta-testers at the end of February, so we are confident that everything works as intended before we make it publicly available. The best way to keep abreast of updates is to follow our Twitter feed: which is usually updated every day.


Looking forward, the next release (v1.5) is due for the summer, and will add some exciting new features, probably including the second most frequently requested addition: memos! Proper note taking functionality is top of many people’s request lists, and will make it much easier to record researcher’s ponderings during the analysis process. For the meantime, check out our blog post article on how to record and code your notes in Quirkos. We also hope to add a lot more tools to help look at word-frequency in their qualitative data sets, including the ever popular word clouds!


In addition to all this, we will have a major new collaboration to announce in the next few months. This is going to represent a major leap forward in functionality for Quirkos, bringing some top minds into the fray to work on the next generation of qualitative analysis software.


So far, we have reinvested all our sales income into development, to make sure that we keep making the software better, and keep current and future users happy. Since all our updates are free, the best way to support further development is to buy a licence, and you will always benefit from work we do in the future to add new capabilities, and be able to suggest the features that will make your qualitative research easier and more fun.