Quantitative vs. qualitative research

quantitative vs qualitative research


So this much is obvious: quantitative research uses numbers and statistics to draw conclusions about large populations. You count something that is countable, and process results across the sample.

 

Qualitative methods are more elusive: however in general they revolve around collecting data from people about an experience. This could be how they used a service, how they felt about something, and could be verbal or written. But it is generally speech or talk, albeit with a variety of levels inferred above and below this (if they are truthful, or if what they say has deeper or hidden meaning). Rather than applying a statistical test to the data, a qualitative researcher must read/listen to the data and make an interpretation of what is being discussed, often hoping to discover patterns or contradictions.

 

Interpretation is done in both approaches: quantitative results are still examined in context (often compared with other numbers and data), and given a metric of significance such as a p-value or r-squared to assess if the results, or a part of them, are meaningful.

 

Finally, in general it is seen that a quantitative approach is a positivist paradigm, while qualitative methods fit better with constructionist or pragmatic paradigms (Savin-Baden and Major 2013). However, both are essentially attempting to model and sample something about the world in which we live so that we can simplify and understand it. And it’s not a case of one is better than the other: just like a hammer can’t be used to turn screws, or a screwdriver to hammer in nails, the different methods have different uses. Researchers should always make sure that the question comes first, and that is used to choose the methodology.

 

But you should also ask, is there a quantitative way to measure what your question is asking? If it’s something as simple as numbers of people, or a quantitative aspect like salary. While there are also quantitative measures of things like anxiety or pain that can be used as a proxy to make inferences across a large population. However, for detailed understanding of these issues and how they affect people, these metrics can be crude and don’t get to the detail of the lived experience.

 

However, choosing the right approach also depends on the how much is known about the research question and topic area. If you don’t know what the problems are in a field, you don’t know what questions to ask, or how to record the the answers.

 

I would argue that even in the physical sciences, qualitative research comes first, and sets questions to answer with quantitative methods. Quantitative research projects usually grow from qualitative observations of the physical world, such as 'I can see that ice seems to melt when it gets warm. At what temperature does ice melt?' or qualitative exploration of the existing literature to find things from other research that is surprising or unexplained.

 

In the classic high-school science experiment above, you would quantitatively measure the melting point of water by taking a sample. You don't try and melt all the ice in the world: you take one piece, and assume that other ice behaves in the same way. In both quantitative and qualitative research, sampling correctly is important. Just as only taking one small piece of impure ice will give you skewed results, so will only sampling one small part of a human population.

 

In quantitative research, because you are usually only sampling for one question at a time (i.e. temperature) it's best to have a large sample size. Especially when dealing with naturally variable, unrestricted variables (for example like a person's height) the data will tend to form a bell curve with a large majority of the answers in the middle, and a small number of outliers at either end. If we were sampling ice to melt, we might find that most ice melts around the same temperature, but very pure or dirty ice will have a slight difference. We would take the answer to be the statistical average, for the mean by adding up all the results and dividing by the sample size.

 

You could argue that the same is true for qualitative research. If you are asking people about their favourite ice cream, you'll get a better answer by asking a large number of people, right? Well this might not always be true. Firstly, just as with the ice melting experiment, sampling every piece of ice in the world will not add much more accuracy to your work but will be a lot more work. And with qualitative research, you are generally asking a much more complicated question for each person sampled, so the work increases exponentially as your sample size grows.

 


As your qualitative data grows, Quirkos can help you manage and make sense of it...

 

Remember, it's rare that qualitative research aims to give one definitive answer: it's more exploratory, and interested in the outlier cases just as much as the common ones. So in our qualitative research question 'What is your favourite ice cream' people may talk about gelato, sorbet or iced coffee. Are these really ice cream? One could argue that technically they are not, but if people consider them to be ice cream, and we want to know what to sell for desert at our restaurant, this becomes relevant. As a result of qualitative research, we usually learn to ask better questions 'What is your favourite frozen dessert?' might be a better question.

 

Now our qualitative research has helped us create a good piece of quantitative research. We can do a survey with a large sample size, and ask the question 'What is your favourite frozen dessert?' and give a list of options which are the most common answers from our qualitative research.

 

However, there can still be flaws with this approach. When answering a survey people don't always say what they mean, and you lose the context of their answers. In surveys there is primacy effect which means that people are lazy, and much more likely to tick the first answer in a list. In this case, the richness of our qualitative answers are lost. We don't know what context people are talking about (while walking along a beach, or in a restaurant or at home?) and we also loose the direct contact with the respondent so we can tell if they are lying or being sarcastic, and we can't ask follow on questions.

 

That's why qualitative research can still be useful as part of, or following quantitative research, for discovering ‘Why’ – understanding the results in the richness of lived experience. Often research projects will have a qualitative component – taking a subset of the the larger quantitative study and getting an in-depth qualitative insight with them.

 

There’s no shame in using a mixed methods approach if it is the most appropriate for the subject area. While there is often criticism over studies that ‘tack-on’ a small qualitative component, and don’t properly integrate or triangulate the types of results, this is a implementation rather than paradigm problem. But remember, it’s not a case of one approach vs another, there are no inheriently good or bad approaches. Methods should be appropriate to each task/question and should be servants to the researcher, not ruling them (Silverman 2013).

 

Quirkos is about as close as a pure qualitative software package as you can find. It's quick to learn, visual and keeps you close to the data. Our focus is on just doing qualitative coding and analysis well, and not to attempt  statistical analysis of qualitative data. We believe that for most qualitative researchers that's the right methodological approach. However, there is capacity for allow some mixed method analysis, so that you can filter results by demographic or other data.

 

The best way to see if Quirkos works for you is to give it a go! Download our one month free trial of the full version with no restrictions, and see if Quirkos works for your research paradigm.

 

 

The importance of the new qualitative data exchange standard

qda xml qualitative exchange

 

Last week, a group of software developers from ATLAS.ti, f4analyse, Nvivo (QSR), Transana, QDA Miner (Provalis) and Quirkos were in Montreal for the third international meeting on the creation of a common file format for exchanging qualitative data projects. The initiative is also supported by Dedoose and MAXQDA, which means that all the major qualitative data analysis software (QDAS) providers have agreed to support a standard that will allow researchers to bring data across any existing QDAS platform.

 

This work has been almost two years in the making already, and so far the first part of the standard was announced last week – a ‘codebook’ exchange file, which lets users share their coding framework, i.e. the list of codes/nodes/themes/Quirks that you use in your project. This is already pretty useful if you have developed a long, or standardised coding framework for analysis, and want to use it in another project in a different qualitative analysis software package.

 

However, this is really the tip of the iceberg. It is hoped that by early next year, the full standard will be complete and released, allowing for much more complete projects (including text and multimedia sources and coding) to be exchanged between whatever software package you like. Although the official page: qdasoftware.org  (currently redirecting to here http://web.ato.uqam.ca/developpements/formats_echange/QDAS-XML) lists more technical details of the aim and format of the exchange initiative, it’s a necessarily technical. I’d like here to briefly discuss why I think this is the most important piece of news in the last 20 years for qualitative research.

 

Analysing and coding qualitative data is extremely time consuming, even when using software to help. It can also be mentally and emotionally draining, and the idea of having to redo this work is impossible for most researchers to swallow: it would be like trying to rewrite a novel from scratch – for many large qualitative projects, it is probably a similar amount of work.

 

And until now, there were very few options to move this project from one piece of software for another. Imagine after writing your novel in Word if you couldn’t share it with the public, or even your editor because they were using a different software package? While some QDAS allow limited import and export of certain features from certain other packages, this can be tortuous or usually impossible. For example, MAXQDA seems to currently be able to import projects from NVivo 8 or 9 (but not the more recent versions 10, 11 or 12) and only by installing MS SQL Server 2008, and only on Windows. You can’t save stuff back again, and every time there is a new version of the software, this procedure has to change again (or like this example, gets stuck in an older version), and your data might get trapped.

 

If you move to a different university that has a subscription to a different tool from where you worked or studied before you can’t access your data. You can’t work with someone who has a licence for a different qualitative software package, because you probably can’t share your data projects. In the past this has limited cross-institutional research projects I’ve been part of. And if you’ve done most of your work in one package, but want to use one cool feature in another one, you are out of luck.

 

Qualitative analysis software is expensive, and the university departments which buy them only let you have one at a time. And woe betide you if someone high-up decides they aren’t buying, say, Atlas.ti anymore, you all have to use MAXQDA. All your previous work is probably inaccessible or can only be restored by using painstaking procedures of recollecting and redoing all you had done in your former software.

 

And even if you finished that previous project, the richness of qualitative data means that there are often many different things that could be read from the same set of sources. For example, a project that interviewed people about job prospects and training might also have interesting data about people’s self-esteem and identity through their career. The current situation where data is trapped in a single, proprietary format really limits potential for revisiting analysis again in the future.

 

So that’s the internal problem for qualitative researchers. But the impacts to wider society are far greater.

 

In theory, when writing a research article for publication, the editor or reviewers can ask to examine any of the data for the project, checking for bias or errors in statistical interpretation. But for qualitative research this is made much more difficult due to the large numbers of different formats that data might be in. I feel this is has led to some of the accusations of bias and lack of replicability in qualitative research. It’s really hard to see someone’s analysis process, even if they are reviewing your article for publication – the fundamental basis of trust in science publications.

 

This links into problems with data archiving. Making an anonymised version of your data publically available is increasingly a requirement with publicly funded research. Some of this is possible since the raw data will likely be transcripts in a common text format. But the working out, the coding and details of your analysis and conclusions may be in for example a .nvp (NVivo project file) or similar. And if you don’t have that exact version of the software or work on a Mac, you can’t open that file. Again, the rapid changing of these file formats does not create much future-proofing – in 10 years from now there may be no software that can open your old project.

 

This means that data archives of qualitative data are currently of limited use, since they don’t have coded data, or it is shared in a proprietary format that most people can’t open. There is no free ‘reader’ app for most of these proprietary project files.

 


So why has this happened, and taken so long to fix?

 

Firstly, there are commercial arguments – it seems to make business sense to lock users to a particular software package, as you make it less likely for them to change to a rival software package. I’m not sure how big a consideration this actually is, but it’s a common practice across many industries. Personally, I am always surprised by the fantastic level of comradery between the ‘rival’ software developers in the meetings about creating the exchange format – we are all here for the users (many are qualitative researchers themselves).

 

Secondly, it is very hard to develop these open standards, and this was not the first attempt - For example the UK DExT format. There have been several such proposals and specifications previously published, but none of them have attracted support from more than one developer. Getting that cross-developer support is obviously crucial to getting adoption, otherwise you add new complexity and uncertainty to the field:


xkcd standards

https://xkcd.com/927/

 

And this is why I think this QDA-XML exchange format is going to succeed. A great and independent committee, led by Jeanine Evers from KWALON  and Erasmus University Rotterdam have managed to get signed commitments of support from all the major qualitative software developers, and nearly all of them have been working on the standard for the last two years.

 

There is likely to be good support since decisions made about the format have been negotiated (often at great length) between all the contributing members. Participants in the meetings have a good idea of what their software can and can’t do, and the best way to implement it. It has been an often painful process of compromise for this first version, as many software packages have unique features.

 

So that is the one caveat – this format will not be 100% comprehensive. A particular pretty output graph you crafted in one software package can’t be shown in another in the same way, as certain ways of working which are unique to a software will be lost in translation.

 

But, I think that for most users the format will allow them to transfer and preserve 90% of their work, and certainly all the basics; codes and coding, sources and metadata, groupings and categories, notes and memos. These things won’t look exactly the same in all packages (for example Quirkos supports 16 million colours for codes, some don’t support colours at all). However, the important parts of your data and analysis will come through, allowing for greater flexibility and opportunities for sharing, archiving and secondary analysis. To me, this opens the door to a fundamentally better understanding of the world.

 

An open, liberally licenced (MIT) standard means that anyone can support it, so it is not limited to the current developers, it is very much a future looking initiative. While I suspect it will still be some time in 2019 until this full support appears in releases of your favourite qualitative analysis software (CAQDAS), I think the promise of an open standard is nearer to being delivered than ever before, and that it will fundamentally change for the better the world of qualitative research.

 

Quirkos 1.5.1 is released!

quirkos 1.5.1


We are happy to announce that the latest version of Quirkos (1.5.1) is now available for everyone to download for Windows, Mac and Linux! As ever, it's a free update that won't effect your licence or projects. Just install over your old version and get going straight away. Projects aren't changed at all, so you can keep working with people using old versions, Quikors has no backward or forward compatibility issues with our new releases. While most of the updates are technical and bug fixes, we have one exciting new feature to talk about:

 

 

Codebook interchange (QDA-XML support)

We are excited to include support in Quirkos for the new QDA-XML codebook standard – something you probably haven't heard of, but is part of a really exciting initiative.


For the last few years, a team of qualitative researchers led by KWALON and developers at Atlas.TI, Dedoose, f4analyse, Nvivo (QSR), MAXQDA, Transana, QDA Miner (Provalis) and Quirkos have been working together to support a standard format for sharing qualitative data. The aim is to allow users to move their projects from any software package to any other, something that has not been possible until now. While some software has allowed importing of data from some other software, this has been piecemeal.


It has been a lot of work to get to this stage, and there is a lot more to do - see my next blog post! However, we are announcing today the availability in Quirkos of the first bit of the standard, which will allow you to move your coding framework, or codebook. Effectively, you can now export your list of codes/themes/Quirks from Quirkos to any other package that supports the standard, and also import into Quirkos a codebook created elsewhere.


This is a basic first step, and at the moment only Quirkos, QDA Miner and f4analyse have releases available that support the standard, but it is expected that updates from other vendors in the coming months will improve this situation. We are also putting the finishing touches on a standard that will allow you to move your complete project, including the sources, coding/highlights, notes/memos and sets/cases/properties. It is hoped that software supporting the complete standard will be available in Spring of 2019.


Other improvements in this release include:

 

 

Updated licence manager


Some users have seen a 'Trial Expired' message following upgrading their operating system or installing system updates. We've improved the way we handle licences on your computer so fewer people should be affected by these issues in the future. Linux users should also see improvements to trials and licences after this update.

This does mean that going back to older versions of Quirkos will now take you back to the trial stage until you re-enter your code, but we don't expect users should have any reason to do this, and just let us know if you have any issues.

 

 

Improved support for older graphics drivers in Windows


Most of the crashes we've had reported are caused by outdated graphics drivers in Windows on computers and laptops with Intel integrated graphics. Unfortunately, many computer manufacturers (such as Lenovo, Dell and HP) block the installation of newer drivers that would fix the problem. I know this has been very frustrating for the few people it affects, however, we have released a solution in this version.


If you are getting frequent crashes, please use your file browser to go to the directory you installed Quirkos. By default this will be C:\Program Files (x86)\Quirkos and then go into the 'bin' folder. In there you will see many files, but three with the Quirkos logo – these are the main 'exe' files that launch Quirkos. Try running Quirkos_a.exe or if you still get crashes or graphical glitches, the Quirkos_s.exe file. These two alternate versions bypass the graphics system in your computer, each resulting in slower but more stable performance in these cases. If one of these is working for you, just right click and choose 'Create Shortcut' and drag the Shortcut file onto the Desktop. This will allow you to easily launch the Quirkos in this new 'safe mode'.


We think this is only affecting a small number of people, but I know was very frustrating since we could not find a way to update the graphics and fix your system. I hope this will now provide a much better experience, but PLEASE let us know if you are having these problems (or any others!), and we can help. We always try to take a 'no-one left behind' approach, and will do everything we can to get Quirkos working on your computer.

 

 

General improvements

The new release has a bunch of improvements behind the scenes that should make Quirkos quicker. The download and install size has been reduced, leaving more room on your computer and a faster download. It should also start quicker in the future. We've also fixed a few minor bugs people reported, with a better password dialogue and fixes to line breaks on some systems. In Windows 10, we've fixed an issue where resizing a window would sometimes make things appear off the screen.

 

Unfortunately, we can no longer support Mac OS 10.9 - this version is now over 5 years old and you should do a free update to a more recent version to stay secure. However, older versions of Quirkos will continue to work on 10.9.


If you ever have any problems with Quirkos, or suggestions for new features or how we could make it work better for you, please get in touch. We are proud of how quickly we can respond, and improve Quirkos for you and the community.


So get the new release today, or if you haven't tried it before, download our free trial of the complete version you can evaluate for a full month!

 

 

Qualitative analysis software for monitoring and evaluation

monitoring and evaluation with qualitative software

 

Developing systems for the monitoring and evaluation of services, interventions and programmes (or programs to use the American English spelling) is a particular skill that requires great flexibility. As each intervention to be investigated is different, and the aims of the project and funders and service users vary, evaluations have to draw on a diverse toolkit of methods.


Qualitative methods are often an important part of this approach. While many evaluations (and service delivery partners) would prefer to demonstrate a quantitative impact such as cost-benefit, things like user satisfaction, behaviour change and expected long-term impacts can be difficult or costly to put a quantitative figure to. Investigating smaller demographic subsets can also be challenging, especially when key groups are represented in too small a number to realistically sample to statistical significance. There are also situations where qualitative methods can give a depth of detail that is invaluable, especially when considering detailed suggestions on improvements.


Too often monitoring and evaluation is an overlooked part of service delivery, tacked on at the end with little budget or time to deliver in. But a short qualitative evaluation can often provide some useful insight without the resources for a detailed assessment with a full quantitative sample and modelling.

 

But managing qualitative data comes with it's own set of challenges. Often monitoring will require looking at data over a long time frame, and the end consumers of evaluations can be sceptical of the validity of qualitative data, and need to be shown how it fits their deliverable criteria. But qualitative analysis software can help on both these points (and more). It can help 'show your working out' and demonstrate how certain statements in the qualitative data support conclusions, manage large amounts of longitudinal data of different types and basically ensure that the evaluation can focus on what they do best – choosing the right approach to collecting monitoring data, and interpreting it for the end users.

 


Let's look at the first aspect here – managing and collating qualitative data. Regardless of the methodology chosen, such as focus groups, interviews and open-ended surveys, qualitative software can be used to keep all the different data together in one place, allowing for cross-referencing across sources, as well as looking at results from one method. But it also makes it much easier to draw in other 'qualitative' documents to provide context, such as project specifications or policy documents. It also can help collate informal sources of data, such as comments and feedback from service users that were collected outside the formal discovery process.


But my favourite aspect that qualitative software helps facilitate is the development and re-application of assessment criteria. Usually there will be fairly standard aspects for evaluation, such as impact, uptake, cost-effectiveness, etc. But funders and commissioners of M&E may have their own special interests (such as engagement with hard-to-reach populations) which need to be demonstrated.


In qualitative software these become the framework for coding the qualitative data: assigning supportive statements or evidence to each aspect. In our qualitative analysis software, Quirkos, this are represented as bubbles you add data to, sometimes called nodes or themes in other software. However, once you have developed and refined a framework that matches set evaluation criteria, you can reuse this in other projects – tweaking slightly to match the specifications of each project.


That way, it is easy to show comments from users, service delivery staff, or other documentation that supports each area. This not only helps the researcher in their work, but also in communicating the results to end users. You can show all (or some) of the data supporting conclusions in each area, as well as contrasts and differences between subsets. In Quirkos, you would use the Query view and the side-by-side comparison view to show how impact was different between groups such as gender or age. The visual overviews that software likes Quirkos creates can help make funders and budget holders get a quick insight into qualitative data, that usually is time consuming to digest in full (it also makes for great visual presentation slides or figures in reports).

 


Of course, all this can be done with more 'traditional' qualitative analysis approaches, such as paper and highlighters, or a huge Excel spreadsheet. But dedicated qualitative software makes sorting and storing data easier, and can save time in the long run by creating reports that help communicate your findings.


I know a lot of evaluators, especially for small projects, feel that learning qualitative analysis tools or setting up a project in qualitative software is not worth the time investment. But if it has been a while since you have tried qualitative software, especially 'new-kids-on-the-block' like Quirkos, it might be worth looking again. Since Quirkos was initially designed for participatory analysis, it can be learnt very quickly, with a visual interface that keeps you close to the data you are investigating.

 

It's also worth noting its limitations: Quirkos is still text only, so exploring multimedia data is not possible, and it takes a very pure-qual philosophy, so there are few built-in tools for quantitative analysis, although it does support exploration of mixed method and discrete data. If you need these extra features you should look at some of the more traditional packages such as Nvivo and Atlas.ti, bearing in mind the extra learning requirement that comes along with more powerful tools.


We think that for most qualitative evaluations Quirkos will have more than enough functionality, with a good trade-off between power and ease of use. There's a free trial you can download, and our licences are some of the cheapest (and most flexible) around. And if you had any specific questions about using Quirkos for monitoring and evaluation, we'd love to hear from you (support@quirkos.com)and are always happy to help you out learning and using the software. For more on using Quirkos in this field, check out our M&E feature overview.

 

 

References and Resources:

 

The AEA (American Evaluation Association) has a rather outdated list of qualitative software packages:
http://www.eval.org/p/cm/ld/fid=81

 

The British CAQDAS Network has independent reviews of qualitative software and training courses:
https://www.surrey.ac.uk/computer-assisted-qualitative-data-analysis/support/choosing

 

Better Evaluation has just one link to a chapter giving very general overview of choosing qualitative software:
http://www.betterevaluation.org/en/resources/choosing_qual_software

 

 

Using qualitative analysis software for literature reviews

qualitative software for literature reviews

 

You’ve probably heard of or even used a reference management software like EndNote, Mendeley or the free and open-source Zotero. However, while these tools are great for doing your references at the end of a project and integrating with Word or LibreOffice, there are still major advantages to using qualitative analysis software like Quirkos.

This video will give you an overview of how to structure either a systematic or literature review, or even just the literature for a project.

 

 

To summarise, while most reference management software focuses on the bibliographical data of the source, CAQDAS/QDAS tools focus on the content of the article itself. While they can still store and export information like publication year, author and titles, they allow you to dive into the text of the article itself, and start to cross-reference particular themes and topics within the literature.


And this is where a literature review gets really interesting. Create a coding framework for key questions in your research, and code specific sections of articles or books that cover that topic. Once you have done this across different articles, you will have a quick, easy and referenced way to write the literature review section of a thesis. When you are talking about, for example, different interpretations of the concept of stigma in the literature, you can show quotes from different authors that agree or disagree, and use this to structure the question.


In Quirkos, if you give sources names like (Goffman 1963), copying and pasting quotes from your project file into Word will automatically give you the quote and reference/source name, formatted in just the right way. For more tips and tricks, we've covered systematic reviews on this blog before.

 

If you'd like to see how Quirkos can take some of the pain out of reference management and literature reviews, you can try the full version for a month with no restrictions. Download the trial today and see for yourself!

 

 

Qualitative analysis software

learn qualitative software

Qualitative Analysis Software

Articles on using and learning Qualitative Analysis Software in general, and Quirkos in particular. Also known as CAQDAS software or QDA software tools.

 

 

General qualitative software articles

Starting a qualitative research thesis, and choosing a CAQDAS package

For those about to embark on a qualitative Masters or PhD thesis, we salute you! More and more post-graduate students are using qualitative methods in their research projects, or...

Comparing qualitative software with spreadsheet and word processor software
An article was recently posted on the excellent Digital Tools for Qualitative Research blog on how you can use standard spreadsheet software like Excel to do qualitative analysis. There are many other articles describing this kind of approach, for example Susan Eliot or Meyer and Avery (2008). However, it’s also possible to use word processing software...

Does software lead to the homogenisation of qualitative research?
In the last couple of weeks there has been a really interesting discussion on the Qualrs-L UGA e-mail discussion group about the use of software in qualitative analysis. Part of this was the question of whether qualitative software leads to the ‘homoginisation’ of qualitative research and analysis...

Practice Projects and learning Qualitative Data Analysis Software
Coding and analysing qualitative data is not only a time consuming, it’s a difficult interpretive exercise which, like learning a musical instrument, gets much better with practice. However, lots of students starting their first major qualitative or mixed method research project will benefit from completing a smaller project first...

Stepping back from qualitative software and reading coded qualitative data
There is a lot of concern that qualitative analysis software distances people from their data. Some say that it encourages reductive behaviour, prevents deep reading of the data, and leads to a very quantified type of qualitative analysis...

Reflections on qualitative software from KWALON 2016
Last week saw a wonderful conference held by the the Dutch network for qualitative research KWALON, based at the Erasmus University, Rotterdam. The theme was no less than the future of Qualitative...

Include qualitative analysis software in your qualitative courses this year
A new term is just beginning, so many lecturers, professors and TAs are looking at their teaching schedule for the next year. Some will be creating new courses, or revising existing modules,...

Using qualitative analysis software to teach critical thought
It's a key part of the curriculum for British secondary school and American high school education to teach critical thought and analysis. It's a vital life skill: the ability to...

Fracturing and choice in qualitative analysis software
Fundamental to the belief behind starting Quirkos was a feeling that qualitative research has great value to society, but should be made accessible to more people. One of the problems that we...

What can CAQDAS do for you: The Five-Level QDA
I briefly mentioned in my last blog post an interesting new article by Silver and Woolf (2015) on teaching QDA (Qualitative Data Analysis) and CAQDAS (Computer Assisted Qualitative Data...

The CAQDAS jigsaw: integrating with workflows
I'm increasingly seeing qualitative research software as being the middle piece of a jigsaw puzzle that has three stages: collection, coding/exploring, and communication. These steps...

Is qualitative data analysis fracturing?
Having been to several international conferences on qualitative research recently, there has been a lot of discussion about the future of qualitative research, and the changes happening in the...

Paper vs. computer assisted qualitative analysis
I recently read a great paper by Rettie et al. (2008) which, although based on a small sample size, found that only 9% of UK market research organisations doing qualitative research were using...

Analysing text using qualitative software
I'm really happy to see that the talks from the University of Surrey CAQDAS 2014 are now up online (that's 'Computer Assisted Qualitative Data Analysis Software' to you and me). It...



 

Quirkos tutorials, guides and comparisons



7 unique things that make Quirkos awesome
Quirkos is now 3 years old! To celebrate, we’re taking a break from our regular programming of qualitative method posts to remind everyone why Quirkos is the best qualitative analysis software around...

Quirkos vs Nvivo: Differences and Similarities
I’m often asked ‘How does Quirkos compare to Nvivo?’. Nvivo is by far the largest player in the qualitative software field, and is the product most researchers are familiar with. So when looking at the alternatives like Quirkos...

How Quirkos can change the way you look at your qualitative data
We always get a lot of inquiries in December from departments and projects who are thinking of spending some left-over money at the end of the financial year on a few Quirkos licences...

Making qualitative analysis software accessible
Studies and surveys seem to show that the amount of qualitative research is growing, and that more and more people are using software to help with their qualitative analysis (Woods et al. 2015)...

Tips for managing mixed method and participant data in Quirkos and CAQDAS software
Even if you are working with pure qualitative data, like interview transcripts, focus groups, diaries, research diaries or ethnography, you will probably also have some categorical data about...

Sharing qualitative research data from Quirkos
Once you've coded, explored and analysed your qualitative data, it's time to share it with the world. For students, the first step will be supervisors, for researchers it might be peers...

Teaching qualitative analysis software with Quirkos
When people first see Quirkos, we often hear them say My students would love this! The easy learning curve, the visual feedback and the ability to work on Windows or Mac appeal...

Tips and advice from one year of Quirkos
This week marks the one-year anniversary of Quirkos being released to the market! On 6th October 2014, a group of qualitative researchers, academics and business mentors met in a bar in...

Freeing qualitative analysis from spreadsheet interfaces
The old mantra is that a picture tells a thousand words. You've probably seen Hans Rosling's talks on visualising quantitative data, or maybe even read some of Edward Tufte's books...

10 reasons to try qualitative analysis with Quirkos
Quirkos is the newest qualitative research software product on the market, but what makes it different, and worth giving the one-month free trial a go Here's a guide to the top 10 benefits to...

Levels: 3-dimensional node and topic grouping in Quirkos
One of the biggest features enabled in the latest release of Quirkos are 'levels', a new way to group and sort your Quirks thematically. While this was always an option in previous...

How to organise notes and memos in Quirkos
Many people have asked how they can integrate notes or memos into their project in Quirkos. At the moment, there isn't a dedicated memo feature in the current version of Quirkos (v1.0),...

Upgrade from paper with Quirkos
Having been round many market research firms in the last few months, the most striking things is the piles of paper, or at least in the neater offices - shelves of paper! When we talk to small...

True cross-platform support
Another key aim for Quirkos was to have proper multi-platform support. By that, I mean that it doesn't matter if you are using a desktop or laptop running Windows, a Mac, Linux, or a tablet,...

 

Quirkos release information


Announcing Quirkos v1.5
We are happy to announce the immediate availability of Quirkos version 1.5! As always, this update is a free upgrade for everyone who has ever brought a licence of Quirkos, so download now and enjoy the new features and improvements...

Quirkos 1.4.1 is now available for Linux
A little later than our Windows and Mac version, we are happy to announce that we have just released Quirkos 1.4.1 for Linux. There are some major changes to the way we release and package our Linux version, so we want to provide some technical details of these, and installation instructions...
Quirkos 1.4.1 is now available for Linux

Quirkos version 1.4.1 is here
Since Quirkos version 1.4 came out last year, we have been gathering feedback from dozens of users who have given us suggestions, or reported problems and bugs. This month we are releasing a small update...

Quirkos version 1.4 is here!
It's been a long time coming, but the latest version of Quirkos is now available, and as always it's a free update for everyone, released simultaneously on Mac, Windows and Linux with...

An early spring update on Quirkos for 2016
About this time last year, I posted an update on Quirkos development for the next year. Even though February continues to be cold and largely snow-drop free in Scotland, why not make it a...

What's in your ideal qualitative analysis software
We will soon start work on the next update for Quirkos. We have a number of features people have already requested which we plan to add to the next version, including file merge, memos, and a...

Quirkos for Linux!
We are excited to announce official Quirkos support for Linux! This is something we have been working on for some time, and have been really encouraged by user demand to support this Free and...

Quirkos 1.3 is released!
We are proud to announce a significant update for Quirkos, that adds significant new features, improves performance, and provides a fresh new look. Major changes include: PDF import Greater...

Quirkos v1.1 is here!
We are excited to announce that the first update for Quirkos can now be downloaded from here! Version 1.1 adds two main new features: batch import, and mutli-language reports. If you...

Spring software update for Quirkos
Even in Edinburgh it's finally beginning to get warmer, and we are planning the first update for Quirkos. This will be a minor release, but will add several features that users have been...

Quirkos is launched!
It's finally here! From today, anyone can download the full 1.0 release version of Quirkos for Windows or Mac OS X! Versions for Linux and Android will be appearing later in the month, but since...

Announcing Pricing for Quirkos
At the moment, (touch wood!) everything is in place for a launch next week, which is a really exciting place to be after many years of effort. From that day, anyone can download Quirkos, try it free...

Quirkos is just weeks away!
It's been a long time since I've had time to write a blog article, as there are so many things to put in place before Quirkos launches in the next few weeks. But one-by-one everything is...

Touching Text
Presenting Quirkos at the CAQDAS 2014 conference this month was the first major public demonstration of Quirkos, and what we are trying to do. It's fair to say it made quite a splash! But...

 


 

Qualitative coding and analysis

analysis and coding of qualitative data

Articles on the analysis and coding of qualitative data

 

Qualitative analysis

What is qualitative analysis?

How do you actually analyse qualitative data? How do you turn the results from your research into findings that can answer your research questions? Analysing qualitative data requires drawing meaning from it...

Making the leap from qualitative coding to analysis
So you spend weeks or months coding all your qualitative data. Maybe you even did it multiple times, using different frameworks and research paradigms. You've followed our introduction guides and everything is neatly...

Circles and feedback loops in qualitative research
The best qualitative research forms an iterative loop, examining, and then re-examining. There are multiple reads of data, multiple layers of coding, and hopefully, constantly improving theory and insight into the underlying lived... world.

Stepping back from qualitative software and reading coded qualitative data
There is a lot of concern that qualitative analysis software distances people from their data. Some say that it encourages reductive behaviour, prevents deep reading of the data, and leads to a very quantified type of qualitative analysis...

What actually is Grounded Theory? A brief introduction
“It's where you make up as you go along!” For a lot of students, Grounded Theory is used to describe a qualitative analytical method, where you create a coding...

An introduction to Interpretative Phenomenological Analysis
Interpretative Phenomenological Analysis (IPA) is an increasingly popular approach to qualitative inquiry and essentially an attempt to understand how participants experience and make meaning of their world...

Triangulation in qualitative research
Most qualitative research will be designed to integrate insights from a variety of data sources, methods and interpretations to build a deep picture. Triangulation is the term used to describe this comparison and meshing of different data...

Analytical memos and notes in qualitative data analysis and coding
There is a lot more to qualitative coding than just deciding which sections of text belong in which theme. It is a continuing, iterative and often subjective process, which can take weeks or even months. During this time...



Coding Qualitative Data

Developing and populating a qualitative coding framework in Quirkos
In previous blog articles I've looked at some of the methodological considerations in developing a coding framework. This article looks at top-down or bottom-up approaches, whether you...

Qualitative coding with the head and the heart
In the analysis of qualitative data, it can be easy to fall in the habit of creating either very descriptive, or very general theoretical codes. It's often a good idea to take a step...

Play and Experimentation in Qualitative Analysis
In the last blog post article, I talked about the benefits of visualising qualitative data, not just in the communication and dissemination stage, but also during data analysis. For newcomers to the...

Top-down or bottom-up qualitative coding
In framework analysis, sometimes described as a top-down or 'a-priori' approach, the researcher decides on the topics of interest they will look for before they start the analysis, usually...

Turning qualitative coding on its head
For the first time in ages I attended a workshop on qualitative methods, run by the wonderful Johnny Saldana. Developing software has become a full time (and then some) occupation for me...

In vivo coding and revealing life from the text
Following on from the last blog post on creating weird and wonderful categories to code your qualitative data, I want to talk about an often overlooked way of creating coding topics using...

Against entomologies of coding
I was recently privileged to chair a session at ICQI 2017 entitled “The Archaeology of Coding”. It had a fantastic panel of speakers, including...

Integrating policy analysis into your qualitative research
It’s easy to get seduced by the excitement of primary data collection, and plan your qualitative research around methods that give you rich data from face-to-face contact with participants. But some research questions may be better illustrated or even mostly answered by analysis of existing documents...

Merging and splitting themes in qualitative analysis
To merge or to split qualitative codes, that is the question... One of the most asked questions when designing a qualitative coding structure is “How many codes should I...

Balance and rigour in qualitative coding frameworks
Training researchers to use qualitative software and helping people who get stuck with Quirkos, I get to see a lot of people’s coding frameworks. Most of the time they are great, often they are fine but have too many codes, but sometimes they just seem to lack a little balance...

Word clouds and word frequency analysis in qualitative data
What’s this blog post about? Well, it’s visualised in the graphic above! In the latest update for Quirkos, we have added a new and much requested feature, word clouds! I'm sure you've used these pretty tools before, they show a random display of all the words in a source of text...

Building queries to explore qualitative data
So, you've spent days, weeks, or even months coding your qualitative data. Now what Hopefully, just the process of being forced to read through the data, and thinking about the...

6 meta-categories for qualitative coding and analysis
When doing analysis and coding in a qualitative research project, it is easy to become completely focused on the thematic framework, and deciding what a section of text is about. However...

 


 

Qualitative methods blog posts

qualitative methods

Articles on qualitative methods

 

 

This series aims to introduce qualitative methods and some of the main approaches in collecting qualitative data.

 

 

Why qualitative research?
There are lies, damn lies, and statistics It's easy to knock statistics for being misleading, or even misused to support spurious findings. In fact, there seems to be a growing backlash at the...

What is a Qualitative approach
The benefit of having tastier satsumas is difficult to quantify: to turn into a numerical, comparable value. This is essentially what qualitative work does: measure the unquantifiable quality of...

An overview of qualitative methods
There are a lot of different ways to collect qualitative data, and this article just provides a brief summary of some of the main methods used in qualitative research. Each one is an art in its own...

Thinking About Me: Reflexivity in science and qualitative research
Reflexivity is a process (and it should be a continuing process) of reflecting on how the researcher could be influencing a research project. In a traditional positivist research paradigm...

 

 

Qualitative Interviews

 

10 tips for semi-structured qualitative interviewing
Many qualitative researchers spend a lot of time interviewing participants, so here are some quick tips to make interviews go as smooth as possible: before, during and after! 1. Let your...

Designing a semi-structured interview guide for qualitative interviews
Interviews are a frequently used research method in qualitative studies. You will see dozens of papers that state something like We conducted n in-depth semi-structured interviews with...

 

 

Focus Groups

Considering and planning for qualitative focus groups
This is the first in a two-part series on focus groups. This week, we are looking at some of the why you might consider using them in a research project...

Tips for running effective focus groups
In the last blog article I looked at some of the justifications for choosing focus groups as a method in qualitative research. This week, we will focus on some practical tips to make sure that focus groups run smoothly...

 

 

Participatory Methods

Participatory Qualitative Analysis
Engaging participants in the research process can be a valuable and insightful endeavour, leading to researchers addressing the right issues, and asking the right questions. Many funding...

Participant diaries for qualitative research
I've written a little about this before, but I really love participant diaries! In qualitative research, you are often trying to understand the lives, experiences and motivations of...

 

 

Qualitative and mixed method surveys

Bringing survey data and mixed-method research into Quirkos
Later today we are releasing a small update for Quirkos, which adds an important feature users have been requesting: the ability to quickly bring in quantitative and qualitative data from any...

The importance of keeping open-ended qualitative responses in surveys
I once had a very interesting conversation at a MRS event with a market researcher from a major media company. He told me that they were increasingly ‘costing-out’ the qualitative open-ended questions from customer surveys...
 

How to set up a free online mixed methods survey
It's quick and easy to set up an on-line survey to collect feedback or research data in a digital format that means you can quickly get straight to analysing the data. Unfortunately, most...


 

Qualitative evaluations

Qualitative evaluations: methods, data and analysis
Evaluating programmes and projects are an essential part of the feedback loop that should lead to better services. In fact, programmes should be designed with evaluations in mind, to make sure that...

Using Quirkos for Systematic Reviews and Evidence Synthesis
Most of the examples the blog has covered so far have been about using Quirkos for research, especially with interview and participant text sources. However, Quirkos can take any text source you can...
 

Qualitative evidence for evaluations and impact assessments
For the last few months we have been working with SANDS Lothians, a local charity offering help and support for families who have lost a baby in miscarriage, stillbirth or soon after birth. They...


 

Sampling and sample sizes

Sampling considerations in qualitative research
Two weeks ago I talked about the importance of developing a recruitment strategy when designing a research project. This week we will do a brief overview of sampling for qualitative research...
 

Reaching saturation point in qualitative research
A common question from newcomers to qualitative research is, what's the right sample size How many people do I need to have in my project to get a good answer for my research...
 

Triangulation in qualitative research
Most qualitative research will be designed to integrate insights from a variety of data sources, methods and interpretations to build a deep picture. Triangulation is the term used to describe this comparison and meshing of different data...

 

 

Recording and Transcribing

Recording good audio for qualitative interviews and focus groups
Last week's blog post looked at the transcription process, and what's involved in getting qualitative interview or focus-group data transcribed. This week, we are going to step...
 

Transcribing your own qualitative data
In a previous blog article I talked about some of the practicalities and costs involved in using a professional transcribing service to turn your beautifully recorded qualitative interviews and...

Transcription for qualitative interviews and focus-groups
Audio and video give you a level of depth into your data that can't be conveyed by words alone, letting you hear hesitations, sarcasm, and nuances in delivery that can change your...


 

 

What is qualitative analysis?

what is qualitative analysis

 

How do you actually analyse qualitative data? How do you turn the results from your research into findings that can answer your research questions?


Analysing qualitative data requires drawing meaning from it, and getting to some higher level of interpretation than reading the data at face value. This is the process that can seem difficult for newcomers to qualitative techniques, or those used to quantitative methods where the interpretation seems more obvious. However, in statistical analysis of data, the result is often a single figure which still needs to be interpreted in context, by looking at things like the sample size, population and normal distribution.


Qualitative analysis also requires this final stage of understanding the results in context, but before then it is often necessary to digest the large amounts of data in some way. While words and meanings can’t be averaged in the same way as numerical values can across different participants or cases, multiple readings of qualitative data may give you a personal perception of the important themes in the data.


This is probably the most important part of your analysis, reading and re-reading your data to become familiar with it and understand what is interesting and surprising. However, you will need to present this insight to others in some digestible way. That’s why I generally describe qualitative analysis as being a cycle of three stages:

 

Interrogate, summarise, connect

 

First, ask questions of the data to interrogate it. What do people say about this? What words to people use to describe that? When you are reading through the transcripts of your data create a series of specific lines of inquiry that map onto your research questions. Through this process you can start creating a mental or thematic summary of the data that answers your research question for each of the sources or each of the important topics in your data.


The next step is to draw connections from your summaries, so that you can find common themes across the different sources of your data. So perhaps a particular opinion is shared by lots of respondents, or only some of them. You will also want to look for connections between your themes, to see which parts of your research questions are interrelated, such as if people are often talking negatively about a particular service. One way to do this is to break down data to smaller themes that are common across all sources, then building up to a deeper level of analysis and understanding by looking at these common themes in the wider context of the data. Codes and coding can help you through this, we’ll look at this later.

 

Although the above gives a general approach to qualitative analysis, but there are a large number of different approaches to how you read and interpret your data. Some of the most common are below, but each of these is really deserving of their own blog post, or chapter in a textbook, so make sure you read about the different options before you settle on one.

 

Which ever approach you have to reading and interpreting the data, there are two general approaches to dealing with the themes from the data. The first is generally called ‘grounded theory’, emergent coding or inductive (data driven) analysis. Here, the researcher lets the data suggest the themes and codes that will be used to summarise and explore the data. It is assumed that there is no pre-conception of what will be interesting in the data, and the coding framework will grow organically with multiple readings.


The alternative is framework or structured analysis where the coding and analysis are driven by pre-existing theory. In this approach the topics used to explore the data are defined before reading the data, based on the research questions and existing literature or research. Usually the next stages will involve coding and categorising the sources to make sense of the volume and depth in the data in these stages:

 

  Develop a ‘framework’ – a list of topics or nodes
 
(except in grounded theory where this is inductive)
  ‘Code’ sections of data to one or more topics
  ‘Retrieve’ data at topics
  Explore connections between the data

 

If you were to illustrate these stages with examples, it might look like this:

 

Research Question:
  “What is seen as a healthy breakfast?”

Break down into themes:
  Healthy, sugar, portions, marketing

Find quotes that fit these themes:
  “Muesli is very sweet, but feels healthy”

Draw conclusions across the data:
  Most people thought Muesli was healthy

 


While CAQDAS or QDA software like Quirkos can help you with each of these steps of the coding process, this is only the middle stage of the process after your reading. You still need to examine the results of the coding and make the jump to proper analysis and interpretation of the data – and that’s the next post to read!

 

Once you are ready to experiment with coding and qualitative analysis, give Quirkos a try with the one month free trial. It’s the most straightforward and intuitive qualitative analysis software around, with licences that don’t expire, free updates and true Windows, Mac and Linux compatibility. We do qualitative software in all the right ways, so you can focus on your data, not battling software.

 

Managing coding frameworks in Quirkos

managing qualitative coding frameworks

If you are doing inductive coding or grounded theory, your coding framework can get complex, quickly. If you have hundreds of codes, they can become difficult to mangage which can slow down your coding - the part of your analysis you want as efficient and effective as possible so you can focus on identifying bigger trends.

 

Fortunately, this is what qualitative analysis software is best at - and whether you are using Nvivo, Atlas.ti or Quirkos there are ways to organise and sort your coding themes. In Quirkos the whole interface is designed to give you great flexibility to group and work with your codes, and this week we are going to look at some of the different ways to do this.

 

Occasionally I will see coding frameworks that have not been sorted at all, and they can look something like this:

 

messy coding

 

If you make no attempt to sort and group your codes, they can become very difficult to work with. The example above has around 100 codes/quirks, but finding the right one is hard, and there is no structure to help you find them thematically. If you are working with other people, it is also nearly impossible to understand what is being investigated - and the same is true for long lists of codes. However, in Quirkos you can move the bubbles around just by dragging them, so you can quickly create rudimentary clusters:

 

better mess

 

Already this is looking a lot better, and you can probably now see what the research topic is about... I've just created simple clusters by theme, with composers in the bottom left, playwrights to the right, and genres in the middle. This is quite usable now, and is well worth the 5 minute investment this took to sort. However, we can make even better. Quirkos supports millions of different colours, so you can also colour-code your quirks by theme:

 

coloured codes

 

Much prettier, but importantly it's also quicker to find Quirks. If I know that all my composers are shades of red, my eye is drawn to the bubbles immediately, and the colour of the highlight stripes in the coded text immediately tells me what has been coded. This is preserved in the exports as well, so reviewing and sharing data with others gets an extra dimension of organisation and information.

 

There are still no groups in the above example, and Quirkos makes these very easy to create - just drag bubbles onto each other to create hierarchies:

 

 

Now it's getting beautiful! And so much quicker to find the right codes. The only disadvantage to this view is that the sub-categories aren't directly visible. This isn't usually a problem because just hovering the mouse over the parent will allow the subcategories to pop out. But if you are doing a lot of direct coding to the sub and sub-sub categories, it can help to have them always expanded. You can do this with the 'Quirks as Tree view':

 

 

This is most similar to the list views in Nvivo and other qualitative analysis software, so can be a good way to get used with Quirkos, or use if you don't like the bubble paradigm. There are also options in the View button to arrange the bubbles by size, or alphabetically. Levels are another way to group themes in Quirkos, which can create non-hierarchical categories.

 

Finally, don't forget to merge! If you have too many codes, some of them are probably superfluous, and can either be merged into other more general topics or deleted all together. Quirkos gives you a lot of flexibility to work with your codes, and manage them so that you can keep focusing on your data. Download a free trial to see for yourself, and get in touch with us if you have any questions.