Thinking About Me: Reflexivity in science and qualitative research

self rembrandt reflexivity

Reflexivity is a process (and it should be a continuing process) of reflecting on how the researcher could be influencing a research project.


In a traditional positivist research paradigm, the researcher attempts to be a neutral influence on  research. They make rational and logical interpretations, and assume a ‘null hypothesis’, in which they expect all experiments to have no effect, and have no pre-defined concept of what the research will show.


However, this is a lofty aspiration and difficult to achieve in practice. Humans are fallible and emotional beings, with conflicting pressures on jobs, publication records and their own hunches. There are countless stories of renowned academics having to retract papers, or their whole research careers because of faked results, flawed interpretations or biased coding procedures.


Many consider it to be impossible to fully remove the influence of the researcher from the process, and so all research would be ‘tainted’ in some way by the prejudices of those in the project. This links into the concept of “implicit bias” where even well-meaning individuals are influenced by subconscious prejudices. These have been shown to have a significant discriminatory impact on pay, treatment in hospitals and recruitment along lines of gender and ethnicity.


So does this mean that we should abandon research, and the pursuit of truly understanding the world around us? No! Although we might reject the notion of attaining an absolute truth, that doesn’t mean we can’t learn something. Instead of pretending that the researcher is an invisible and neutral piece of the puzzle, a positionality and reflexivity approach argues that the background of the researcher should be detailed in the same way as the data collection methods and analytical techniques.


But how is this done in practice? Does a researcher have to bare their soul to the world, and submit their complete tax history? Not quite, but many in feminist and post-positivist methodologies will create a ‘positionality statement’ or ‘reflexivity statement’. This is a little like a CV or self-portrait of potential experiences and bias, in which the researcher is honest about personal factors that might influence their decisions and interpretations. These might include the age, gender, ethnicity and class of the researcher, social and research issues they consider important, their country and culture, political leanings, life experiences and education. In many cases a researcher will include such a statement with their research publications and outputs, just Googling ‘positionality statements’ will provide dozens of links to examples.

 

However, I feel that this is a minimum level of engagement with the issue, and it’s actually important to keep a reflexive stance throughout the research process. Just like how a one-off interview is not as accurate a record as a daily diary, keeping reflexivity notes as an ongoing part of a research journal is much more powerful. Here a researcher can log changes in their situation, assumptions and decisions made throughout the research process that might be affected by their personal stance. It’s important that the researcher is constantly aware of when they are making decisions, because each is a potential source of influence. This includes deciding what to study, who to sample, what questions to ask, and which sections of text to code and present in findings.


Why this is especially pertinent to qualitative research? It’s often raised in social science, especially ethnography and close case study work with disadvantaged or hard-to-reach populations where researchers have a much closer engagement with their subjects and data. It could be considered that there are more opportunities for personal stance to have an impact here, and that many qualitative methods, especially the analysis process using grounded theory, are open to multiple interpretations that vary by researcher. Many make the claim that qualitative research and data analysis is more subjective than quantitative methods, but as we’ve argued above, it might be better to say that they are both subjective. Many qualitative epistemological approaches are not afraid of this subjectivity, but will argue it is better made forthright and thus challenged, rather than trying to keep it in the dark.


Now, this may sound a little crazy, especially to those in traditionally positivist fields like STEM subjects (Science, Technology Engineering, Mathematics). Here there is generally a different move: to use process and peer review to remove as many aspects of the research that are open to subjective interpretation as possible. This direction is fine too!


However, I would argue that researchers already have to make a type of reflexivity document: a conflict of interest statement. Here academics are supposed to declare any financial or personal interest in the research area that might influence their neutrality. This is just like a positionality statement! An admission that researchers can be influenced by prejudices and external factors, and that readers should be aware of such conflicts of interest when doing their own interpretation of the results.


If it can be the case that money can influence science (and it totally can) it’s also been shown that gender and other aspects of an academic's background can too. All reflexivity asks us to do is be open and honest with our readers about who we are, so they can better understand and challenge the decisions we make.

 

 

Like all our blog articles, this is intended to be a primer on some very complex issues. You’ll find a list of references and further reading below (in addition to the links included above). Don’t forget to try Quirkos for all your qualitative data analysis needs! It can help you keep, manage and code a reflexive journal throughout your analysis procedure. See this blog article for more!

 

 

References

 

Bourke, B., 2014, Positionality: Reflecting on the Research Process, The Qualitative Report 19, http://www.nova.edu/ssss/QR/QR19/bourke18.pdf


Day, E., 2002, Me, My*self and I: Personal and Professional Re-Constructions in Ethnographic Research, FQS 3(3) http://www.qualitative-research.net/index.php/fqs/article/view/824/1790


Greenwald, A., Krieger, L., 2006, Implicit Bias: Scientific Foundations, California Law Review, 94(4). http://www.jstor.org/stable/20439056


Lynch, M., 2000, Against Reflexivity as an Academic Virtue and Source of Privileged Knowledge, Theory, Culture & Society 17(3), http://tcs.sagepub.com/content/17/3/26.short


Savin-Baden, M., Major C., 2013, Personal stance, positionality and reflexivity, in Qualitative Research: The essential guide to theory and practice. Routledge, London.


Soros, G., 2013, Fallibility, reflexivity and the human uncertainty principle, Journal of Economic Methodology, 20(4) https://www.georgesoros.com/essays/fallibility-reflexivity-and-the-human-uncertainty-principle-2/

 

 

Top-down or bottom-up qualitative coding?

In framework analysis, sometimes described as a top-down or 'a-priori' approach, the researcher decides on the topics of interest they will look for before they start the analysis, usually based on a theory they are looking to test. In inductive coding the researcher takes a more bottom-up approach, starting with the data and a blank-sheet, noting themes as the read through the text.

 

Obviously, many researchers take a pragmatic approach, integrating elements of both. For example it is difficult for a emergent researcher to be completely naïve to the topic before they start, and they will have some idea of what they expect to find. This may create bias in any emergent themes (see previous posts about reflexivity!). Conversely, it is common for researchers to discover additional themes while reading the text, illustrating an unconsidered factor and necessitating the addition of extra topics to an a-proiri framework.

 

I intend to go over these inductive and deductive approaches in more detail in a later post. However, there is also another level in qualitative coding which is top-down or bottom-up: the level of coding. A low 'level' of coding might be to create a set of simple themes, such as happy or sad, or apple, banana and orange. These are sometimes called manifest level codes, and are purely descriptive. A higher level of coding might be something more like 'issues from childhood', fruit, or even 'things that can be juggled'. Here more meaning has been imposed, sometimes referred to as latent level analysis.

 

 

Usually, researchers use an iterative approach, going through the data and themes several times to refine them. But the procedure will be quite different if using a top-down or bottom-up approach to building levels of coding. In one model the researcher starts with broad statements or theories, and breaks them down into more basic observations that support or refute that statement. In the bottom-up approach, the researcher might create dozens of very simple codes, and eventually group them together, find patterns, and infer a higher level of meaning from successive readings.

 

So which approach is best? Obviously, it depends. Not just on how well the topic area is understood, but also the engagement level of the particular researcher. Yet complementary methods can be useful here: the PI of the project, having a solid conceptual understanding of the research issue, can use a top-down approach (in both approaches to the analysis) to test their assumptions. Meanwhile, a researcher who is new to the project or field could be in a good position to start from the bottom-up, and see if they can find answers to the research questions starting from basic observations as they emerge from the text. If the themes and conclusions then independently reach the same starting points, it is a good indication that the inferences are well supported by the text!

 

qualitative data analysis software - Quirkos

 

 

An overview of qualitative methods

There are a lot of different ways to collect qualitative data, and this article just provides a brief summary of some of the main methods used in qualitative research. Each one is an art in its own right, with various different techniques, definitions, approaches and proponents.

More on each one will follow in later articles, and it’s worth remembering that these need to be paired with the right questions, sampling, and analysis to get good results.

Interviews

Possibly the richest, and most powerful tool: talking to someone directly. The classic definition is “conversations with a purpose“, the idea being that there is something you are interested in, you ask questions about it, and someone gives useful responses.

There are many different styles for example how structured your questions are (this paper has a wonderful and succinct overview in the introduction). These can range from a rigid script where you ask the same questions every time, or completely open discussion, where the researcher and respondent have freedom to shape the conversation. A common middle ground are semi-structured interviews, which often have a topic guide, listing particualar issues to discuss, but will allow questions for clarification, or to follow up on an interesting tangent.

Participant Observation

Often the remit of ethnography or sociology, participant observation usually involves watching, living or even participating in the daily life of research subjects. However, it can also involve just watching people in a certain setting, such as a work meeting, or using a supermarket.

This is probably the most time intensive and potentially problematic method, as it can involve weeks or even years of placement for a researcher, often on their own. However, it does produce some of the richest data, as well as a level of depth that can really help explain complex issues. This chapter is a fine starting point.

Focus groups

A common method used in market research, where a researcher leads a group discussion on a particular topic. However, it is also a powerful tool for social researchers, especially when looking at group dynamics, or the reactions of particular groups of people. It’s obviously important to consider who is chosen for the group, and how the interactions of people in the group affect the outcome (although this might be what you are looking for).

It’s usually a quicker and cheaper way of gauging many reactions and opinions, but requires some skill in the facilitator to make sure everyone’s voice is being heard, and that people stay on track. Also a headache for any transcribers who have to identify different voices from muffled audio recordings!

Participant Diaries

Getting people to write a diary for a research project is a very useful tool, and is commonly used in looking at taboo behaviours such as drug use or sexuality, not just because researchers don’t have to ask difficult questions face-to-face, but that data can be collected over a long period of time. If you are trying to find out how often a particular behaviour occurs, a daily or weekly record is likely to be more accurate than asking someone in a single interview (as in the studies above).

There are other benefits to the diary method: not least that the participant is in control. They can share as much or as little as they like, and only on topics they wish to. It can also be theraputic for some people, and is more time flexible. Diaries can be paper based, electronic, or even on a voice recorder if there are literacy concerns. However, researchers will probably need to talk to people at the beginning and end of the process, and give regular reminders.

Surveys

Probably one of the most common qualitative methods are the open ended questions on surveys, usually by post, on-line, or ‘guided’ by someone with a clipboard. Common challenges here are

  • Encouraging people to write more than one word, but less than an essay
  • Setting questions carefully so they are clear, but not leading
  • Getting a good response rate and
  • Knowing who has and hasn’t responded

The final challenge is to make sure the responses are useful, and integrating them with the rest of the project, especially quantitative data.

Field notes

Sometimes the most overlooked, but most vaulable source of information can be the notes and field diaries of researchers themselves. These can include not just where and when people did interviews or observations, but crucial context, like the people who refused to take part, and whether a interviewee was nervous. It need not just be for ethnographers doing long field work, it can be very helpful in organising thoughts and work in smaller projects with multiple researchers.

As part of a reflexive method, it might contain comments and thoughts from the researcher, so there can be a risk of autobiographical overindulgence. It is also not easy to integrate ‘data’ from a research diary with other sources of information when writing up a project for a particular output.

 

This is just a whistle-stop introduction, but more on each of these to follow…