Today we are announcing that a major new version of Quirkos is coming in September! Version 2  will offer big new features that users have requested, including memos, rich text support, new editable reports, an improved interface, and much more.

Memos are a feature that people have been requesting for a while, and we are excited to have this coming in the next version. This allows users to write notes which are attached to specific segments of your text sources. You can write long or short comments, and these can be used in approaches like IPA and in-vivo coding which were difficult to achieve in Quirkos before.

These memos are visible anywhere your text is, so you will see them connected to your text in the quotes overview, in search and query results as well. We’ve made adding and working with memos as simple and intuitive as the rest of Quirkos – jut drag and drop a section of text into the memo column to add a new memo, and type straight away. You can also toggle the memo column open and closed if you want to focus on just your text and coding at any time.

It has been nearly 4 years since we released the very first beta of Quirkos to users, and since then all our updates have been free, and kept backward and forward project compatibility. We don’t charge ongoing fees, or have licences that expire, and these options will continue into the future. For these reasons we think that Quirkos offers the fairest and best value licences for qualitative software.

While there will be a small upgrade fee for users on version 1.x wanting the extra features of 2.x, we will create a final free release for version 1 (1.6) to make sure that it will be backward and forward compatible. We never want to see a situation where people can’t share project files because they are using different versions of Quirkos, and lock their projects into an outdated version. Unlike some other qualitative software packages, we will never do this. So don’t worry: even if you don’t need the new capabilities of 2.0, you won’t be forced to upgrade because of lack of support or backward compatibility. We are qualitative researchers too, and want to minimise these headaches!

We’re also announcing that anyone that buys Quirkos from now until the release date will get a free upgrade to version 2 when it is available, an offer that also applies to anyone that brought Quirkos in July and August.

If you do want to upgrade, there will be a simple process to change your licence code and download the new version, and you will still be able to keep working with your projects without doing anything. We’ll let you know when it’s available and how much the upgrade will be for different users.

We also want to assure people about what isn’t changing in Quirkos 2. We will keep the same interface (with a few tweaks), identical compatibility with Windows, Mac and Linux versions, and all the features that were there before. We also don’t plan to add complexity to Quirkos with the new capabilities, and most people will be able to move to version 2 without getting stuck or needing training. We have thousands of dedicated users in hundreds of universities and other organisations, and keeping them happy is what keeps us going!

But version 2 allows us to work towards a new platform that enables a lot of new and exciting capabilities in the future. There are exciting technical innovations and collaborative capabilities coming in Quirkos 2 in the not too distant future, and we are really looking forward to detailing more information in the coming months. We will have more blog posts soon outlining the new things coming to Quirkos this year, and note that 2.0 is only the tip of the iceberg!

At the National Council for Research Methods ‘Research Methods Festival’ last month, Steve Wright (from the University of Lancaster) mentioned in his talk the frustrations he has with students that do the bog-standard ’12 semi-structured interviews’ methodology for their qualitative research projects. This prompted a lot of discussion and empathy over lunch, with many tutors lamenting how students weren’t choosing some of the more creative methods for qualitative research.

Even a lot of the popular textbooks on qualitative research only mention the basic methods, or some variants on textual data collection (eg Braun and Clarke 2013). Even if it’s not interviews of some kind, transcribed focus groups and other textual methods definitely dominate the literature. Helen Kara has a textbook specifically on Creative Methods, which is well worth a read if you are looking for inspiration. But the value of qualitative research can be magnified by choosing the right imaginative methodology, and thinking outside the box a little to redefine what we can collect and analyse as ‘data’.

This is a huge world, but I wanted to give a taster (with lots of examples) of 10 qualitative methods that can go a lot beyond the default ’12 semi-structured interviews’ and engage with participants in new and exciting ways.

Diaries

OK, we’ve talked about diaries before. But there is much more to diaries than just hand written journals. You can also have audio diaries (Williamson et al 2015) and video diaries (Bates 2013). There are even diary apps for phones (Garcia et al. 2015), which can notify partipants at reguar intervals to find out what they are doing or feeling. Laura Radcliffe and Leighann Spencer gave a great talk on the challenges and advantages of diary apps at RMF 2018. Each have their own benefits and give you a different level of insight into participants lives, but for certain research, especially where you want to minimise recall issues, regular recording in one of these ways can be really useful.

Participant Photography

Although sometimes connected with diaries, getting participants to record their life through Photo Elicitation can get them to reflect on important issues, and provides a good basis for discussion. Usually you give your participants a camera (although with the ubiquity of smartphones this is rarely necessary these days) and ask them to take pictures of things that have meaning to them about your research question. This is the concept of Photo Voice, where you give your paricipants a way to express their lives and experiences pictorially. There’s a nice overview here by Harper (2002).

Art

Many of the ‘creative methods’ focus on different ways to integrate art into research. You can basically use any medium, but the idea is often to get participants to reflect on their life experiences and create something (a drawing, clay sculpture, collage) that expresses something connected to the research. Examples include ‘Target drawings’ Tracy, et al. (2006), clay sculptures, (Or 2015), self-portraits (Esteban-Guitart et al 2016), drama and theatre (Norris 2010) or even quilting (Bacic et al. ND). There are many more listed in this presentation by Mannay (2016). This is a huge field, and always fun to see different ways people have been innovative here. However, a key part of the method is getting participants to either label and explain, or discuss with the researchers and other participants the meaning and different interpretations of their creations.

Walking methods

If your research is connected to a place, or how people experience an area, there are many interesting approaches you can do with participants while walking with them through a place and getting them to explain their world. These have various names and variations such as the ‘walking interview’ Jones et al. (2008), transecting or walking fieldwork (Goschel 2015). You can record these visually, aurally or with notes and pictures, or get participants to reflect on them afterwards.

Mapping / network diagrams

Another good tool for getting people to explore and explain their geographical area with researchers, but mapping tools can also be used to demonstrate other things, such as connections between organisations people use, social networks, or how they see connections between concepts as in mind mapping (Burgess-Allen and Owen-Smith 2010) . There is pictorial narrative mapping Lapum et al. 2015 (which is more like some of the artistic reflection techniques above), body mapping which can be used to show pain (Mukherjee 2002), or getting local people to create and label a map of their area.

Secondary Analysis

To some, this may seem even more boring than just doing qualitative interviews, but secondary analysis of other sources of data can be really interesting and insightful, and avoids a lot of practical and ethical issues. You can do document, media or social media analysis or even re-analyse someone’s existing dataset to see if it can reveal something about a different research question. There’s some more advice on our post here.

Games and activities

When you do focus groups, don’t just facilitate dry discussion: use games and fun activities to get your participants engaged and sharing. You can use sorting and ranking exercises with cards you make with each card representing a part of the research. You can get people to discuss photos, newspaper articles, made up stories about a controversial issues or flip-charts where you get people to come up with ideas or answer difficult questions. Get people to move: show how strongly they agree with a statement by standing at different positions along a line. In each of these situations, the data can be either the outcome (where people stand / what people share) or the discussion that ensures. There’s a whole book of tips and tricks for making focus groups more interesting (and successful): Participatory Workshop (Chambers 2002).

Participatory research

This isn’t always a method in itself, but in some situations it can be really valuable to include participants in the data collection or analysis. In some paradigms they can be seen as the real experts of their own lived experiences, or an ‘insider’ can be a useful co-researcher. Often they are able to make sure that the most relevant questions are being asked, can act as gatekeepers to other participants that might be difficult to reach, or will have their own interpretations of the data that can challenge researchers. It also can shift the power dynamic away from binary researcher and researched. Much more on our blog post on participatory research.

Observation / Ethnography

If you have the time to deeply engage with an organisation or a group of people, researchers can become embedded in their research subject with ethnography or participant observation. Usually a researcher will spend weeks, months or even years watching and learning a research context first hand, and it can give very detailed data and understanding. However, there are shorter variations of observation or ‘rapid ethnographies’ (Vindrola-Padros and Vindrola-Padros 2017) which can be a great complement to other qualitative research methods: verifying and expanding on other sources of data.

Surveys

Now, this again might seem a bit boring, but I think surveys are often overlooked as a qualitative research method. There are a good way to reach out to lots of people, online, in person or by post, and you can be a lot more creative with questions. Get people to explain what they see in a picture. Use one word to express how you feel about something.  Use emoji’s or get people to rate or rank statements. Ask questions about identity in different ways: which Disney princess do you most associate with, and why? Leave space for lots of open ended answers, but choose creative and engaging questions to get people to think and reflect.

Hopefully this post has inspired you to consider or even try out some different qualitatve methods that differ from the normal boring ones. The key with all these is to consider what exactly will constitute the data you collect, and then how you will analyse it. For data that comes back to text or transcripts, Quirkos can be a fun and engaging way to help you analyse differently as well. Give the free trial a go, and see how it makes qualitative analysis a visual method!

We are pleased to announce a bug fix release for Quirkos that takes us to version 1.5.2.

This is a fairly minor update, but includes 4 bug fixes people had reported:

    • Quirks that got ‘stuck’ and couldn’t be dragged
    • An issue with deleting sources that sometimes caused properties to have extra ‘not defined’ responses
    • A bug with some CSV import that led to “ characters being read incorrectly
    • A bug with docx import that would sometimes create extra spaces in the source text

As always, you can download and install the new version over the old one, and the update will not affect your licence or project files. There is no change to the file format, so backward compatibility is also maintained.

For this release, we have also created a new distribution method for Quirkos on Linux – a ‘Snap’ image. This solves several issues that Linux users had seen on some distributions, with unresolvable dependency issues and reports not launching correctly.

If you haven’t used Snap before, it aims to be a cross-distro package installation system, that should take care of most of the dependencies for you. It is included by default in Ubuntu 18.04, but the package manager snap (or snapd) must be installed first.

We’ve tested it on all releases of Ubuntu from 14.04, and Fedora 27/28. Many more distributions should be fine – please let us know if you have any issues, and we will help get around them. We are aware of two dependencies at the moment, on Fedora 28 if you are using the new Wayland driver (to replace the X window system) you will need to install the package qt5-qtwayland. On ubuntu systems using the proprietary Nvidia graphics drivers, you need to manually copy the libs:

sudo cp -r /usr/lib/nvidia-version/* /var/lib/snapd/lib/gl/

This seems to be a known issue with snapd at the moment.

Once you have downloaded it, you can install with a command like:

snap install quirkos_1.5.2_amd64.snap --devmode

and Quirkos can then by started from the command line by typing ‘quirkos’. Note that on some distros you will have to log out and back in again before bash is updated and typing ‘quirkos’ will link to the binary.

Please let us know how you get on, and as always we keep our older binary (32bit) and AppImage available for people that have had better luck with that. We hope to make Quirkos available in the Snap store in the future, which will make getting Quirks even easier for those on Ubuntu. We love Linux and supporting it, so please let us know your feedback, good or bad – there are so many different distributions and configurations of them we can’t test them all!

This now makes the 10th free update since Quirkos was publicly released more than 3 and a half years ago. However, check the blog next week for some very exciting news for later in the year and the future of Quirkos...

Essentially, observation is a type of, or more likely, a part of ethnography. In ethnography, anthropologists (people who study people) turn their observations of people, cultures and organisations into written field notes (a bit like a research diary). While some of this may be reflexive (the participants own thoughts and feelings) most focuses on the activities and interactions of the people being studied.

There are broadly two types of observation. The first is participant observation in which the researcher becomes part of and gets involved in the context, area or group they are studying. The second is direct observation, where the researcher does not take part in the activity or setting, but is more of a fly on the wall – passively watching and recording what is happening.

There are advantages to both approaches: for example it’s easier to observe and take notes with direct observation, while during participant observation you may be actively taking part in the meeting / surfboarding session (Kawulich 2005). However, participant observation can allow for a deeper level of understanding, embedding and acceptance from the study group, allowing for more significant insights. Taking part in the culture/activity can also provide a ‘Walk two moons in their moccasins’ revelation, allowing the researcher to fully understand and empathise with the decisions and actions of participants.

Typically, a participant observer would offer to get involved by volunteering, doing some useful task like taking minutes or driving people around – essentially doing favours that let them help out while being able to see what is going on. It does not need to involve the actual task or skill being researched – for example in an ethnography of two tattoo parlours the author “helped maintain files of tattoo designs, working behind the front desk” although eventually got tattoos herself (Velliquette 1998).

One specific field of observational research is ‘Organisational Ethnography’, where researchers look at organisations, management or work places. (Ybema et al. 2009). Here ethnographers may look at a wide range of organisations from parliament (Crewe 2018) to a steel mill in Sheffield (Ahrens and Mollona 2007).

However there are also methodological limitations to observation. Even with direct observation, there can be an effect from having the researcher in the room – people’s behaviour may not be normal, and maybe modified if participants have a sense of being watched or judged (see the Hawthorne Effect). With time and acceptance of the research, the effect may become less, but it is still difficult to claim pure objectivity in observational research, especially when the researcher is talking part directly in the culture of the researched.

This is why reflexivity is so important in ethnography and participant observation, because the prejudices and interpretations of the researcher need to be untangled (or at least made explicit) from the data.

Observational data

Any method of observation has a myriad of practical and theoretical challenges. The first is to consider what kinds of data will be produced during the observation. Usually these will be field notes, but may also include documents (minutes from meetings, policy), audio, video, music and direct comments from people in the field of study. Many ethnographers use a dictaphone to record either the whole session live, or more likely their own thoughts and reflections afterwards. This creates audio data which probably will need to be at least partially transcribed.

Researchers need to have a loose plan before they start their fieldwork of what kinds of data will be collected and how, so that they can make sure the data can be effectively analysed. However, there will often be unexpected sources and type of data in a long and embedded fieldwork project like this, so prepare for some flexibility. Also, consider the volume of data that participant observation will generate (like most qualitative methods). For one study 40 hours of observation generated 28,000 words when transcribed (Conway 2017).

It’s also worth thinking about triangulation, and paring with other qualitative methods. For example, semi-structured interviews can be a good compliment to observation, as interviews allow you to ask questions one-on-one with people who have been part of the ethnography. These can be used to check assumptions, and ask for questions and clarifications on aspects of culture that are not obvious (e.g. Why do you all wear these hats?). Just remember that these direct questions are generating a different type of data to the observation: the participant during an interview is conscious of being questioned about their culture, and is giving an expressed opinion (These hats are stylish) which may not match the researchers interpretations based solely on observation (People wear hats to emulate the cool kids).

In fact, there is usually a little informal observation going on in most qualitative research projects. It’s hard just to meet a series of people for interviews without watching the culture around them and how they act with others (as Katz (2002) says – social researchers are always in ‘the field’). And often finding the right people to interview (if this is your designated research method) involves some participant observation to identify the most interesting respondents.

Gaining access, consent and trust

Yet for any type of observation, there will be significant issues around access and consent. The first hurdle is to persuade a group of people that it is a good idea to have a nosey researcher hanging out with them for months at a time. It can sometimes be tempting to claim that the research will be useful to them; in getting their situation better understood, or identifying issues and problems in their culture. However, this is a difficult thing to promise. While all good researchers should provide feedback and share findings with participants, the things that an academic researcher is investigating may not match with the immediate problems of participants. Qualitative observation of this type is usually based around fairly speculative exploration, with a sort of grounded theory approach, so there is little guarantee from the start exactly what the area of focus will be.

Usually, gaining access will be done through ‘gatekeepers’ (see more on gatekeepers in this article on recruitment  – https://www.quirkos.com/blog/post/designing-a-qualitative-recruitment-strategy). This may be a senior leader (mayor, tribal leader) in a cultural setting, or manager in an organisation. However, it is worth considering wider issues of consent with the many people a researcher will encounter. Although a senior manager may have given permission for the research, this does not automatically mean that their subordinates also give consent. There may be situations where this is explicit ‘Everyone must take part’ but individuals may not be freely giving consent if they are scared of going against the wishes of their boss.

Getting access for this type of in-depth observation can be a lengthy and fractional process, where researchers are only given access to certain areas at first, and as trust grows they are invited to more closed-off activities (such as weddings or management meetings). Building trust and rapport is an important skill that ethnographers must develop, and to which there are rarely shortcuts – long periods of time are usually required to negotiate access. Indeed, some researchers have come to see the difficulty of negotiating access as an important part of the ethnography itself (Frandsen 2015).

We are going to look more at ethnography in a future blog post, but what ever type of observation you are making, you might consider qualitative analysis software like Quirkos to help analyse and find themes in your qualitative text data. Download a free trial today, and see why people describe Quirkos as ‘intuitive’, ‘colourful’, and even ‘fun’!

This post is based on a talk I gave at ICQI 2018, which itself leads on from a talk from last year on the Entomologies of qualitative coding.

Good qualitative data is rich, and detailed - a fertile medium for understanding and interpreting the world. But the detail of the data comes at a price, usually qualitative data sources are lengthy, and are about a lot of different things. You don't just ask a single question that can be answered with a one word answer, you inquire and explore a range of issues around the topic to draw out detail and explore the 'why' behind the answers.

This means that the analysis of qualitative data starts with reading the data, to get a sense of the landscape of it, but an intermediary stage is coding - and this is the part of qualitative analysis that qualitative software like Quirkos can help with. We create codes which are like themes, and read through the text and put sections of text which are relevant to them into each code. Tagging the data in this way lets us bring quotes together that fit a theme, to eventually support (or disprove) a hypothesis. But what should these codes be? What features do we highlight that help us see the similarities and differences in the data?

Broadly speaking these themes can of two types: very low order, basic descriptive codes, or 'higher level' conceptual codes. It's difficult to describe the process and difference between these types of coding, but you can conceptualise it as moving from the lowest common denominator across the data to the highest (although it's also possible to do it the other way around).

This concept is found often in high-school level math when trying to add fractions. If they have different denominators (the bottom bit which shows how many sections they are) you have to multiply them out to get a common denominator - in other words a number that can be used to divide both fractions. It’s also the same game that entomologists are playing when trying to create a taxonomy of insects or other animals. Think about how you would describe what features are common to the butterflies in the top image? It can be a specific spot of a certain colour (a basic low level feature) or a feature that may look very different, but has a similar purpose - like an antenna (high level).

This is a bit like what qualitative coding often tries to do - find common themes that occur across all the sources. At a very basic level these will probably be very simple. Everyone in our sources is talking about 'Politics' in a general sense, 'The Media' and 'Opinions'. Creating these descriptive codes, and putting text into them is a useful way to start the coding process. It creates a 'map' or list of everything everyone is saying about 'The Media' and we can then read all these quotes together and look for patterns.

But there is a risk of creating codes that are so 'common' and so basic that they are pretty meaningless on their own. The more vague the theme, the more data will fit into it, but the less useful filtering of the data you are getting. Remember, the end goal is to find data that will answer your research question, and it is unlikely that these are as vague as 'What do people say about Politics?'. Usually you are looking for a much more specific insight, such as 'How do libertarian leaning people distance themselves from the policies of the Republican party?'. To do this, and to make a meaningful conclusion, we need to move to something more akin to the highest common denominator. In other words, what are the highest level, most significant and specific insights that are common themes in the data?

These are the 'highest common divisors' - in math the largest number you can give that allows you to divide and compare numbers or fractions. Every number is dividable by itself, and one. In the same way, every thing an individual says is true about themselves, and each word or statement they make is true in itself. However, neither of these is particularly interesting or insightful in itself, without some point of comparison. It's not important that an opinion or experience must be common to everyone to be relevant in qualitative research, in fact that's the strength of this methodology. However, you could argue that dissenting or different views are only interesting in comparison.

However, the highest common denominator codes should be at a high enough conceptual level to cover a variety of opinions, but bring them together under a common theory. It's not just saying people think this, or some people think the other way about their political leanings, but how they create a political identity. High level codes should be a close match to a theoretical interpretation of the world, such as “Gender is a performative act” (Butler 1988). These may be an existing theory, or a new theory you are discovering by applying a grounded theory approach.

But it's usually pretty hard to jump straight to this level of understanding of your data. Maybe you can read though all the sources once and just see a new conceptual understanding of the world emerge. However, this is rare, and you would probably still want to have quotes to illustrate and support your understanding. That's why creating your coding structure of the lowest common denominator first can help you to get to the next levels. And there may be multiple levels of coding, involving moving up, grouping and refining codes to support a deeper hypothesis. It's one reason why qualitative analysis is often described as a cyclical, iterative process.

Quirkos is designed to help you create and manage these different stages of coding with the ‘levels’ feature. This lets you create groups of codes from different coding iterations, and even have some that belong to all or just some of the levels. There’s a lot more information about how they work (and can be used to do other things) in this blog post on levels and groups. However, you can also always download the full version of Quirkos for free and try it for a month. It’s the easiest qualitative analysis software package to learn, as well as being one of the cheapest and most visual.

But remember, that even once you have created and populated a high level coding framework, this is not the same as analysis. You still need to make the leap from coding to qualitative analysis and actually read through the coded data, keep re-conceptualising it, and eventually match it to your research questions so that they can be answered. However, if you can keep coming back to the butterfly categorisation or fraction addition metaphors above, it might help you keep your eye looking out for both the low and high level themes in your research, and developing a rich coding framework that will help your insights and conclusions bubble up from your data.

So this much is obvious: quantitative research uses numbers and statistics to draw conclusions about large populations. You count something that is countable, and process results across the sample.

Qualitative methods are more elusive: however in general they revolve around collecting data from people about an experience. This could be how they used a service, how they felt about something, and could be verbal or written. But it is generally speech or talk, albeit with a variety of levels inferred above and below this (if they are truthful, or if what they say has deeper or hidden meaning). Rather than applying a statistical test to the data, a qualitative researcher must read/listen to the data and make an interpretation of what is being discussed, often hoping to discover patterns or contradictions.

Interpretation is done in both approaches: quantitative results are still examined in context (often compared with other numbers and data), and given a metric of significance such as a p-value or r-squared to assess if the results, or a part of them, are meaningful.

Finally, in general it is seen that a quantitative approach is a positivist paradigm, while qualitative methods fit better with constructionist or pragmatic paradigms (Savin-Baden and Major 2013). However, both are essentially attempting to model and sample something about the world in which we live so that we can simplify and understand it. And it’s not a case of one is better than the other: just like a hammer can’t be used to turn screws, or a screwdriver to hammer in nails, the different methods have different uses. Researchers should always make sure that the question comes first, and that is used to choose the methodology.

But you should also ask, is there a quantitative way to measure what your question is asking? If it’s something as simple as numbers of people, or a quantitative aspect like salary. While there are also quantitative measures of things like anxiety or pain that can be used as a proxy to make inferences across a large population. However, for detailed understanding of these issues and how they affect people, these metrics can be crude and don’t get to the detail of the lived experience.

However, choosing the right approach also depends on the how much is known about the research question and topic area. If you don’t know what the problems are in a field, you don’t know what questions to ask, or how to record the the answers.

I would argue that even in the physical sciences, qualitative research comes first, and sets questions to answer with quantitative methods. Quantitative research projects usually grow from qualitative observations of the physical world, such as 'I can see that ice seems to melt when it gets warm. At what temperature does ice melt?' or qualitative exploration of the existing literature to find things from other research that is surprising or unexplained.

In the classic high-school science experiment above, you would quantitatively measure the melting point of water by taking a sample. You don't try and melt all the ice in the world: you take one piece, and assume that other ice behaves in the same way. In both quantitative and qualitative research, sampling correctly is important. Just as only taking one small piece of impure ice will give you skewed results, so will only sampling one small part of a human population.

In quantitative research, because you are usually only sampling for one question at a time (i.e. temperature) it's best to have a large sample size. Especially when dealing with naturally variable, unrestricted variables (for example like a person's height) the data will tend to form a bell curve with a large majority of the answers in the middle, and a small number of outliers at either end. If we were sampling ice to melt, we might find that most ice melts around the same temperature, but very pure or dirty ice will have a slight difference. We would take the answer to be the statistical average, for the mean by adding up all the results and dividing by the sample size.

You could argue that the same is true for qualitative research. If you are asking people about their favourite ice cream, you'll get a better answer by asking a large number of people, right? Well this might not always be true. Firstly, just as with the ice melting experiment, sampling every piece of ice in the world will not add much more accuracy to your work but will be a lot more work. And with qualitative research, you are generally asking a much more complicated question for each person sampled, so the work increases exponentially as your sample size grows.

As your qualitative data grows, Quirkos can help you manage and make sense of it...

Remember, it's rare that qualitative research aims to give one definitive answer: it's more exploratory, and interested in the outlier cases just as much as the common ones. So in our qualitative research question 'What is your favourite ice cream' people may talk about gelato, sorbet or iced coffee. Are these really ice cream? One could argue that technically they are not, but if people consider them to be ice cream, and we want to know what to sell for desert at our restaurant, this becomes relevant. As a result of qualitative research, we usually learn to ask better questions 'What is your favourite frozen dessert?' might be a better question.

Now our qualitative research has helped us create a good piece of quantitative research. We can do a survey with a large sample size, and ask the question 'What is your favourite frozen dessert?' and give a list of options which are the most common answers from our qualitative research.

However, there can still be flaws with this approach. When answering a survey people don't always say what they mean, and you lose the context of their answers. In surveys there is primacy effect which means that people are lazy, and much more likely to tick the first answer in a list. In this case, the richness of our qualitative answers are lost. We don't know what context people are talking about (while walking along a beach, or in a restaurant or at home?) and we also loose the direct contact with the respondent so we can tell if they are lying or being sarcastic, and we can't ask follow on questions.

That's why qualitative research can still be useful as part of, or following quantitative research, for discovering ‘Why’ – understanding the results in the richness of lived experience. Often research projects will have a qualitative component – taking a subset of the the larger quantitative study and getting an in-depth qualitative insight with them.

There’s no shame in using a mixed methods approach if it is the most appropriate for the subject area. While there is often criticism over studies that ‘tack-on’ a small qualitative component, and don’t properly integrate or triangulate the types of results, this is a implementation rather than paradigm problem. But remember, it’s not a case of one approach vs another, there are no inheriently good or bad approaches. Methods should be appropriate to each task/question and should be servants to the researcher, not ruling them (Silverman 2013).

Quirkos is about as close as a pure qualitative software package as you can find. It's quick to learn, visual and keeps you close to the data. Our focus is on just doing qualitative coding and analysis well, and not to attempt  statistical analysis of qualitative data. We believe that for most qualitative researchers that's the right methodological approach. However, there is capacity for allow some mixed method analysis, so that you can filter results by demographic or other data.

The best way to see if Quirkos works for you is to give it a go! Download our one month free trial of the full version with no restrictions, and see if Quirkos works for your research paradigm.

Last week, a group of software developers from ATLAS.ti, f4analyse, Nvivo (QSR), Transana, QDA Miner (Provalis) and Quirkos were in Montreal for the third international meeting on the creation of a common file format for exchanging qualitative data projects. The initiative is also supported by Dedoose and MAXQDA, which means that all the major qualitative data analysis software (QDAS) providers have agreed to support a standard that will allow researchers to bring data across any existing QDAS platform.

This work has been almost two years in the making already, and so far the first part of the standard was announced last week – a ‘codebook’ exchange file, which lets users share their coding framework, i.e. the list of codes/nodes/themes/Quirks that you use in your project. This is already pretty useful if you have developed a long, or standardised coding framework for analysis, and want to use it in another project in a different qualitative analysis software package.

However, this is really the tip of the iceberg. It is hoped that by early next year, the full standard will be complete and released, allowing for much more complete projects (including text and multimedia sources and coding) to be exchanged between whatever software package you like. Although the official page: qdasoftware.org  (currently redirecting to here http://web.ato.uqam.ca/developpements/formats_echange/QDAS-XML) lists more technical details of the aim and format of the exchange initiative, it’s a necessarily technical. I’d like here to briefly discuss why I think this is the most important piece of news in the last 20 years for qualitative research.

Analysing and coding qualitative data is extremely time consuming, even when using software to help. It can also be mentally and emotionally draining, and the idea of having to redo this work is impossible for most researchers to swallow: it would be like trying to rewrite a novel from scratch – for many large qualitative projects, it is probably a similar amount of work.

And until now, there were very few options to move this project from one piece of software for another. Imagine after writing your novel in Word if you couldn’t share it with the public, or even your editor because they were using a different software package? While some QDAS allow limited import and export of certain features from certain other packages, this can be tortuous or usually impossible. For example, MAXQDA seems to currently be able to import projects from NVivo 8 or 9 (but not the more recent versions 10, 11 or 12) and only by installing MS SQL Server 2008, and only on Windows. You can’t save stuff back again, and every time there is a new version of the software, this procedure has to change again (or like this example, gets stuck in an older version), and your data might get trapped.

If you move to a different university that has a subscription to a different tool from where you worked or studied before you can’t access your data. You can’t work with someone who has a licence for a different qualitative software package, because you probably can’t share your data projects. In the past this has limited cross-institutional research projects I’ve been part of. And if you’ve done most of your work in one package, but want to use one cool feature in another one, you are out of luck.

Qualitative analysis software is expensive, and the university departments which buy them only let you have one at a time. And woe betide you if someone high-up decides they aren’t buying, say, Atlas.ti anymore, you all have to use MAXQDA. All your previous work is probably inaccessible or can only be restored by using painstaking procedures of recollecting and redoing all you had done in your former software.

And even if you finished that previous project, the richness of qualitative data means that there are often many different things that could be read from the same set of sources. For example, a project that interviewed people about job prospects and training might also have interesting data about people’s self-esteem and identity through their career. The current situation where data is trapped in a single, proprietary format really limits potential for revisiting analysis again in the future.

So that’s the internal problem for qualitative researchers. But the impacts to wider society are far greater.

In theory, when writing a research article for publication, the editor or reviewers can ask to examine any of the data for the project, checking for bias or errors in statistical interpretation. But for qualitative research this is made much more difficult due to the large numbers of different formats that data might be in. I feel this is has led to some of the accusations of bias and lack of replicability in qualitative research. It’s really hard to see someone’s analysis process, even if they are reviewing your article for publication – the fundamental basis of trust in science publications.

This links into problems with data archiving. Making an anonymised version of your data publically available is increasingly a requirement with publicly funded research. Some of this is possible since the raw data will likely be transcripts in a common text format. But the working out, the coding and details of your analysis and conclusions may be in for example a .nvp (NVivo project file) or similar. And if you don’t have that exact version of the software or work on a Mac, you can’t open that file. Again, the rapid changing of these file formats does not create much future-proofing – in 10 years from now there may be no software that can open your old project.

This means that data archives of qualitative data are currently of limited use, since they don’t have coded data, or it is shared in a proprietary format that most people can’t open. There is no free ‘reader’ app for most of these proprietary project files.

So why has this happened, and taken so long to fix?

Firstly, there are commercial arguments – it seems to make business sense to lock users to a particular software package, as you make it less likely for them to change to a rival software package. I’m not sure how big a consideration this actually is, but it’s a common practice across many industries. Personally, I am always surprised by the fantastic level of comradery between the ‘rival’ software developers in the meetings about creating the exchange format – we are all here for the users (many are qualitative researchers themselves).

Secondly, it is very hard to develop these open standards, and this was not the first attempt - For example the UK DExT format. There have been several such proposals and specifications previously published, but none of them have attracted support from more than one developer. Getting that cross-developer support is obviously crucial to getting adoption, otherwise you add new complexity and uncertainty to the field:

https://xkcd.com/927/

And this is why I think this QDA-XML exchange format is going to succeed. A great and independent committee, led by Jeanine Evers from KWALON  and Erasmus University Rotterdam have managed to get signed commitments of support from all the major qualitative software developers, and nearly all of them have been working on the standard for the last two years.

There is likely to be good support since decisions made about the format have been negotiated (often at great length) between all the contributing members. Participants in the meetings have a good idea of what their software can and can’t do, and the best way to implement it. It has been an often painful process of compromise for this first version, as many software packages have unique features.

So that is the one caveat – this format will not be 100% comprehensive. A particular pretty output graph you crafted in one software package can’t be shown in another in the same way, as certain ways of working which are unique to a software will be lost in translation.

But, I think that for most users the format will allow them to transfer and preserve 90% of their work, and certainly all the basics; codes and coding, sources and metadata, groupings and categories, notes and memos. These things won’t look exactly the same in all packages (for example Quirkos supports 16 million colours for codes, some don’t support colours at all). However, the important parts of your data and analysis will come through, allowing for greater flexibility and opportunities for sharing, archiving and secondary analysis. To me, this opens the door to a fundamentally better understanding of the world.

An open, liberally licenced (MIT) standard means that anyone can support it, so it is not limited to the current developers, it is very much a future looking initiative. While I suspect it will still be some time in 2019 until this full support appears in releases of your favourite qualitative analysis software (CAQDAS), I think the promise of an open standard is nearer to being delivered than ever before, and that it will fundamentally change for the better the world of qualitative research.

We are happy to announce that the latest version of Quirkos (1.5.1) is now available for everyone to download for Windows, Mac and Linux! As ever, it's a free update that won't effect your licence or projects. Just install over your old version and get going straight away. Projects aren't changed at all, so you can keep working with people using old versions, Quikors has no backward or forward compatibility issues with our new releases. While most of the updates are technical and bug fixes, we have one exciting new feature to talk about:

Codebook interchange (QDA-XML support)

We are excited to include support in Quirkos for the new QDA-XML codebook standard – something you probably haven't heard of, but is part of a really exciting initiative.

For the last few years, a team of qualitative researchers led by KWALON and developers at Atlas.TI, Dedoose, f4analyse, Nvivo (QSR), MAXQDA, Transana, QDA Miner (Provalis) and Quirkos have been working together to support a standard format for sharing qualitative data. The aim is to allow users to move their projects from any software package to any other, something that has not been possible until now. While some software has allowed importing of data from some other software, this has been piecemeal.

It has been a lot of work to get to this stage, and there is a lot more to do - see my next blog post! However, we are announcing today the availability in Quirkos of the first bit of the standard, which will allow you to move your coding framework, or codebook. Effectively, you can now export your list of codes/themes/Quirks from Quirkos to any other package that supports the standard, and also import into Quirkos a codebook created elsewhere.

This is a basic first step, and at the moment only Quirkos, QDA Miner and f4analyse have releases available that support the standard, but it is expected that updates from other vendors in the coming months will improve this situation. We are also putting the finishing touches on a standard that will allow you to move your complete project, including the sources, coding/highlights, notes/memos and sets/cases/properties. It is hoped that software supporting the complete standard will be available in Spring of 2019.

Other improvements in this release include:

Updated licence manager

Some users have seen a 'Trial Expired' message following upgrading their operating system or installing system updates. We've improved the way we handle licences on your computer so fewer people should be affected by these issues in the future. Linux users should also see improvements to trials and licences after this update.

This does mean that going back to older versions of Quirkos will now take you back to the trial stage until you re-enter your code, but we don't expect users should have any reason to do this, and just let us know if you have any issues.

Improved support for older graphics drivers in Windows

Most of the crashes we've had reported are caused by outdated graphics drivers in Windows on computers and laptops with Intel integrated graphics. Unfortunately, many computer manufacturers (such as Lenovo, Dell and HP) block the installation of newer drivers that would fix the problem. I know this has been very frustrating for the few people it affects, however, we have released a solution in this version.

If you are getting frequent crashes, please use your file browser to go to the directory you installed Quirkos. By default this will be C:\Program Files (x86)\Quirkos and then go into the 'bin' folder. In there you will see many files, but three with the Quirkos logo – these are the main 'exe' files that launch Quirkos. Try running Quirkos_a.exe or if you still get crashes or graphical glitches, the Quirkos_s.exe file. These two alternate versions bypass the graphics system in your computer, each resulting in slower but more stable performance in these cases. If one of these is working for you, just right click and choose 'Create Shortcut' and drag the Shortcut file onto the Desktop. This will allow you to easily launch the Quirkos in this new 'safe mode'.

We think this is only affecting a small number of people, but I know was very frustrating since we could not find a way to update the graphics and fix your system. I hope this will now provide a much better experience, but PLEASE let us know if you are having these problems (or any others!), and we can help. We always try to take a 'no-one left behind' approach, and will do everything we can to get Quirkos working on your computer.

General improvements

The new release has a bunch of improvements behind the scenes that should make Quirkos quicker. The download and install size has been reduced, leaving more room on your computer and a faster download. It should also start quicker in the future. We've also fixed a few minor bugs people reported, with a better password dialogue and fixes to line breaks on some systems. In Windows 10, we've fixed an issue where resizing a window would sometimes make things appear off the screen.

Unfortunately, we can no longer support Mac OS 10.9 - this version is now over 5 years old and you should do a free update to a more recent version to stay secure. However, older versions of Quirkos will continue to work on 10.9.

If you ever have any problems with Quirkos, or suggestions for new features or how we could make it work better for you, please get in touch. We are proud of how quickly we can respond, and improve Quirkos for you and the community.

So get the new release today, or if you haven't tried it before, download our free trial of the complete version you can evaluate for a full month!

Developing systems for the monitoring and evaluation of services, interventions and programmes (or programs to use the American English spelling) is a particular skill that requires great flexibility. As each intervention to be investigated is different, and the aims of the project and funders and service users vary, evaluations have to draw on a diverse toolkit of methods.

Qualitative methods are often an important part of this approach. While many evaluations (and service delivery partners) would prefer to demonstrate a quantitative impact such as cost-benefit, things like user satisfaction, behaviour change and expected long-term impacts can be difficult or costly to put a quantitative figure to. Investigating smaller demographic subsets can also be challenging, especially when key groups are represented in too small a number to realistically sample to statistical significance. There are also situations where qualitative methods can give a depth of detail that is invaluable, especially when considering detailed suggestions on improvements.

Too often monitoring and evaluation is an overlooked part of service delivery, tacked on at the end with little budget or time to deliver in. But a short qualitative evaluation can often provide some useful insight without the resources for a detailed assessment with a full quantitative sample and modelling.

But managing qualitative data comes with it's own set of challenges. Often monitoring will require looking at data over a long time frame, and the end consumers of evaluations can be sceptical of the validity of qualitative data, and need to be shown how it fits their deliverable criteria. But qualitative analysis software can help on both these points (and more). It can help 'show your working out' and demonstrate how certain statements in the qualitative data support conclusions, manage large amounts of longitudinal data of different types and basically ensure that the evaluation can focus on what they do best – choosing the right approach to collecting monitoring data, and interpreting it for the end users.

Let's look at the first aspect here – managing and collating qualitative data. Regardless of the methodology chosen, such as focus groups, interviews and open-ended surveys, qualitative software can be used to keep all the different data together in one place, allowing for cross-referencing across sources, as well as looking at results from one method. But it also makes it much easier to draw in other 'qualitative' documents to provide context, such as project specifications or policy documents. It also can help collate informal sources of data, such as comments and feedback from service users that were collected outside the formal discovery process.

But my favourite aspect that qualitative software helps facilitate is the development and re-application of assessment criteria. Usually there will be fairly standard aspects for evaluation, such as impact, uptake, cost-effectiveness, etc. But funders and commissioners of M&E may have their own special interests (such as engagement with hard-to-reach populations) which need to be demonstrated.

In qualitative software these become the framework for coding the qualitative data: assigning supportive statements or evidence to each aspect. In our qualitative analysis software, Quirkos, this are represented as bubbles you add data to, sometimes called nodes or themes in other software. However, once you have developed and refined a framework that matches set evaluation criteria, you can reuse this in other projects – tweaking slightly to match the specifications of each project.

That way, it is easy to show comments from users, service delivery staff, or other documentation that supports each area. This not only helps the researcher in their work, but also in communicating the results to end users. You can show all (or some) of the data supporting conclusions in each area, as well as contrasts and differences between subsets. In Quirkos, you would use the Query view and the side-by-side comparison view to show how impact was different between groups such as gender or age. The visual overviews that software likes Quirkos creates can help make funders and budget holders get a quick insight into qualitative data, that usually is time consuming to digest in full (it also makes for great visual presentation slides or figures in reports).

Of course, all this can be done with more 'traditional' qualitative analysis approaches, such as paper and highlighters, or a huge Excel spreadsheet. But dedicated qualitative software makes sorting and storing data easier, and can save time in the long run by creating reports that help communicate your findings.

I know a lot of evaluators, especially for small projects, feel that learning qualitative analysis tools or setting up a project in qualitative software is not worth the time investment. But if it has been a while since you have tried qualitative software, especially 'new-kids-on-the-block' like Quirkos, it might be worth looking again. Since Quirkos was initially designed for participatory analysis, it can be learnt very quickly, with a visual interface that keeps you close to the data you are investigating.

It's also worth noting its limitations: Quirkos is still text only, so exploring multimedia data is not possible, and it takes a very pure-qual philosophy, so there are few built-in tools for quantitative analysis, although it does support exploration of mixed method and discrete data. If you need these extra features you should look at some of the more traditional packages such as Nvivo and Atlas.ti, bearing in mind the extra learning requirement that comes along with more powerful tools.

We think that for most qualitative evaluations Quirkos will have more than enough functionality, with a good trade-off between power and ease of use. There's a free trial you can download, and our licences are some of the cheapest (and most flexible) around. And if you had any specific questions about using Quirkos for monitoring and evaluation, we'd love to hear from you (support@quirkos.com)and are always happy to help you out learning and using the software. For more on using Quirkos in this field, check out our M&E feature overview.

References and Resources:

The AEA (American Evaluation Association) has a rather outdated list of qualitative software packages:
http://www.eval.org/p/cm/ld/fid=81

The British CAQDAS Network has independent reviews of qualitative software and training courses:
https://www.surrey.ac.uk/computer-assisted-qualitative-data-analysis/support/choosing

Better Evaluation has just one link to a chapter giving very general overview of choosing qualitative software:
http://www.betterevaluation.org/en/resources/choosing_qual_software

You’ve probably heard of or even used a reference management software like EndNote, Mendeley or the free and open-source Zotero. However, while these tools are great for doing your references at the end of a project and integrating with Word or LibreOffice, there are still major advantages to using qualitative analysis software like Quirkos.

This video will give you an overview of how to structure either a systematic or literature review, or even just the literature for a project.

To summarise, while most reference management software focuses on the bibliographical data of the source, CAQDAS/QDAS tools focus on the content of the article itself. While they can still store and export information like publication year, author and titles, they allow you to dive into the text of the article itself, and start to cross-reference particular themes and topics within the literature.

And this is where a literature review gets really interesting. Create a coding framework for key questions in your research, and code specific sections of articles or books that cover that topic. Once you have done this across different articles, you will have a quick, easy and referenced way to write the literature review section of a thesis. When you are talking about, for example, different interpretations of the concept of stigma in the literature, you can show quotes from different authors that agree or disagree, and use this to structure the question.

In Quirkos, if you give sources names like (Goffman 1963), copying and pasting quotes from your project file into Word will automatically give you the quote and reference/source name, formatted in just the right way. For more tips and tricks, we've covered systematic reviews on this blog before.

If you'd like to see how Quirkos can take some of the pain out of reference management and literature reviews, you can try the full version for a month with no restrictions. Download the trial today and see for yourself!