Starting out in Qualitative Analysis

Qualitative analysis 101

 

When people are doing their first qualitative analysis project using software, it’s difficult to know where to begin. I get a lot of e-mails from people who want some advice in planning out what they will actually DO in the software, and how that will help them. I am happy to help out individually, because everyone’s project is different. However, here are a few pointers which cover the basics and can help demystify the process. These should actually apply to any software, not just Quirkos!

 

First off: what are you going to be able to do? In a nutshell, you will read through the sources, and for each section that is interesting to you and about a certain topic, you will ‘code’ or ‘tag’ that section of text to that topic. By doing this, the software lets you quickly see all the sections of text, the ‘quotes’ about that topic, across all of your sources. So you can see everything everyone said about ‘Politics’ or ‘Negative’ – or both.

 

You can then look for trends or outliers in the project, by looking at just responses with a particular characteristic like gender. You’ll also be able to search for a keyword, and generate a report with all your coded sections brought together. When you come to write up your qualitative project, the software can help you find quotes on a particular topic, visualise the data or show sub-section analysis.  

 

So here are the basic steps:

 

1.       Bring in your sources.
I’m assuming at this stage that you have the qualitative data you want to work with already. This could be any source of text on your computer. If you can copy and paste it, you can bring it into Quirkos. For this example let’s assume that you have transcripts from interviews: this means that you have already done a series of interviews, transcribed them, and have them in a file (say a Word document or raw text file). I’d suggest that before you bring them in, just have a quick look through and correct them in a Word Processor for typos and misheard words. While you can edit the text in Quirkos later, while using a Word or equivalent you have the advantage of spell checkers and grammar checkers.

 

Now, create a new, unstructured project in Quirkos, and save it somewhere locally on your computer. We don’t recommend you save directly to a network location, or USB stick, as if either of these go down, you will have a problem! Next, bring in the sources using the (+) Add Source button on the bottom right. You can bring in each file one at a time, or a whole folder of files in one go, in which case the file name will become the default source name. Don’t forget, you can always add more sources later, there is no need to bring in everything before you start coding. Now your project file (a little .qrk file you named) will contain all the text sources in one place. With Quirkos files, just backing up and copying this file saves the whole project.

 


2.       Describe your sources
It’s usually a good idea to describe some characteristics of your qualitative sources that you might use later to look for differences or similarities in the data. Often these are basic demographic characteristics like age or gender, but can also be things about the interview, such as the location, or your own notes.

 

To do this in Quirkos, click on the little grid button on the top right of the screen, and use the source properties. The first thing you can do here is change the name of the sources from the default (either a sequential number like ‘Source 7’ or the file name. You can create a property with the square [+] ‘Quickly add a new property’ button. The property (eg Gender) and a single value (eg Male) can be added here. The drop down arrow next to that property can be used later to add extra values.

 

The reason for doing this is that you can later run ‘queries’ which show results from just certain sources that have properties you defined. So you can do a side-by-side comparison of coded responses from men next to women. Don’t forget, you can add properties at any time, so you can even create a variable for ‘these people don’t fit the theory’ after you’ve coded, and try and see what they are saying that makes them different.

 

 

3.       Create your themes
Whatever you call them: themes, nodes, bubbles, topics or Quirks, these are the categories of interest you want to collect quotes about from the text. There are two approaches here, you can try and create all the categories you will use before you start reading and coding the text (this is sometimes called a framework approach), or you can add themes as you go (grounded theory). (For much much more on these approaches, look here and here.)

 

In Quirkos, you create themes as coloured bubbles, which grow in size the more text is added. Just click on the grey (+) button on the top right of the canvas view to add a new theme. You can also change the name, colour, level in this dialogue, or right click on the bubble and select ‘Quirk Properties’ at any time. To group, just drag and drop bubbles on top of each other.

 

 

4.       Do your coding
Essentially, the coding process involves finding every time someone said something about ‘Dieting’ and adding that sentence or paragraph to the ‘Dieting’ bubble or node. This is what is going to take the most time in your analysis (days or weeks) and is still a manual process. It’s best to read through each source in turn, and code it as you go.

 

However, you can also use the keyword search to look for words like ‘Diet’ or ‘eating’ and code from the results. This makes it quicker, but there is the risk of missing segments that use a keyword you didn’t think to search for like ‘cut-down’. The keywords search can help when you (inevitably) decide to add a new topic halfway through, and the first few interviews haven’t been coded for the new themes. You can use the search to look for related terms and find those new segments without having to go over the whole text again.

 

 

5.       Be iterative
Even if you are not using a grounded theory approach, going back over the data a second time, and rethinking codes and how you have categorised things can be really useful. Trust me: even if you know the data pretty well, after reading it all again, you will see some topics in a slightly different light, or will find interesting things you never thought would be there.

 

You may also want to rearrange your codes, especially if you have grouped them. Maybe the name you gave a theme isn’t quite right now: it’s grown or got more specific. Some vague codes like ‘Angry’ might need to be split out into ‘Irate’ and ‘Annoyed’. Depending on your approach, you  will probably constantly tweak and adjust the themes and coding so they best represent the intersection of your research questions and data.

 

 

6.       Explore the data.
Once your qualitative data is all coded, the big advantages of using CAQDAS software come into play. Using the database of your tagged text, you can choose to look at it in anyway: using any of the source properties, who did the coding or when, or whether a result comes from any particular group of codes. This is done using the 'Query' views in Quirkos.

 

In Quirkos there are also a lot of visualisation options that can show you the overall shape and structure of the project, the amount of coding, and connections that are emerging between the sources. You can then use these to help write your outputs, be they journal articles, evaluations or a thesis. Software will generate reports that let you share summaries of the coded data, and include key statistics and overviews of the project.


While it does seem like a lot of work to get to this stage, it can save so much time at the final stages of writing up your project, when you can call up a useful quote quickly. It also can help in the future to have this structured repository of qualitative data, so that secondary analysis or adding to the dataset does not involve re-inventing the wheel!

 

Finally, there is no one-size-fits-all approach, and it's important to find a strategy that fits with your way of working. Before you set out, talk to peers and supervisors, read guides and textbooks, and even go on training courses. While the software can help, it's not a replacement for considered thinking, and you should always have a good idea about what you want to do with the data in the end.

 

 

Qualitative evidence for evaluations and impact assessments

qualitative evidence for charities

For the last few months we have been working with SANDS Lothians, a local charity offering help and support for families who have lost a baby in miscarriage, stillbirth or soon after birth. They offer amazing services, including counselling, peer discussion groups and advice to health professionals, which can help ease the pain and isolation of a difficult journey.

 

We helped them put together a compilation of qualitative evidence in Quirkos. This has come from a many sources they already have, but putting it together and pulling out some of the key themes means they have a qualitative database they can use for quickly putting together evaluations, reports and impact assessments. Many organisations will have a lot of qualitative data already, and this can easily become really valuable evidence.

 

First, try doing an ‘audit’ for qualitative data you already have. Look though the potential sources listed below (and any other sources you can think of), and find historical evidence you can bring in. Secondly, keep these sources in mind in day-to-day work, and remember to flag them when you see them. If you get a nice e-mail from someone that they liked an event you ran, or a service they use, save it! It’s all evidence, and can help make a convincing case for funders and other supporters in the future.

 

Here are a few potential sources of qualitative feedback (and even quantitative data) you can bring together as evidence for evaluations and future work:

 

 

1.  Feedback from service users:

Feedback from e-mails is probably the easiest to pull together, as it is already typed up. Whenever someone complements your services, thank them and store the comments as feedback for another day. It is easy to build up a virtual ‘guest-book’ in this way, and soon you will have dozens of supportive comments that you can use to show the difference your organisation makes. Even when you get phone calls, try and make notes of important things that people say. It’s not just positive comments too, note suggestions and if people say there is something missing  – this can be evidence to funders that you need extra resources.

You can also specifically ask for stories from users you know well, these can form case studies to base a report around. If you have a specific project in mind, you can do a quick survey. Ask former users to share their experience on an issue, either by contacting people directly, or asking for comments through social media. By collating these responses, you can get quick support for the direction of a project or new service.

 


2. Social media

Comments and messages of support from your Facebook friends, Twitter followers, and pictures of people running marathons for you on Instagram are all evidence of support for the work you do. Pull out the nice messages, and don’t forget, the number of followers and likes you have are evidence of your impact and reach.

 


3. Local (and international) news

A lot of charities are good at running activities that end up in the local news, so keep clippings as evidence of the impact of your events, and the exposure you get. Funders like to work with organisations that are visible, so collect and collate these. There may also be news stories talking about problems in the community that are related to issues you work on, these can show the importance of the work you do.

 


4. Reports from local authority and national organisations

Keep an eye out for reports from local council meetings and public sector organisations that might be relevant to your charity. If there are discussions on an area you work on, it is another source of evidence about the need for your interventions.


There may also be national organisations or local partners that work in similar areas – again they are likely to write reports highlighting the significance of your area, often with great statistics and links to other evidence. Share and collaborate evidence, and together the impact will be stronger!

 

5. Academic evidence

One of the most powerful ways you can add legitimacy to your impact assessment or funding applications is by linking to research on the importance of the problems you are tackling, or the potential benefits of your style of intervention. A quick search in Google Scholar (scholar.google.com) for keywords like ‘obesity’ ‘intervention’ can find dozens of articles that might be relevant. The journal articles themselves will often be behind ‘paywalls’ that mean you can’t read or download the whole paper. However, the summary is free to read, and probably gives you enough information to support your argument one way or another. Just link to the paper, and refer to it as (‘Author’s surname’, ‘Year of Publication’) for example (Turner 2013).

 

It might also be worth seeking out a relationship with a friendly academic at a local university. Look through Google (or ask through your networks) for someone that works in your area, and contact them to ask for help. Researchers have their own impact obligations, so are sometimes interested in partnering with local charities to ensure their research is used more widely. It can be a mutually beneficial relationship…

 

 

 

Hopefully these examples will help you think through all the different things you already have around you that can be turned into qualitative evidence, and some things you can seek out. We will have more blog posts on our work with local charities soon, and how you can use Quirkos to collate and analyse this qualitative evidence.

 

 

What's in your ideal qualitative analysis software?

Qualitative feature request

 

We will soon start work on the next update for Quirkos. We have a number of features people have already requested which we plan to add to the next version, including file merge, memos, and a lot of small tweaks and changes to the interface to show more data and make some operations easier.


However, there is still time to let us know what you would like to see in future versions of Quirkos. How about Word import, where highlights can be turned into already coded data? Do you want to see wordclouds and keyword counting?


As for the memos: how would you like these to look? Do you just need memos for parts of text, or for each time you code something as well? How should these be displayed when working, and integrated with the reports? Are these best as a separate section, or integrated as side notes on the rest of the data? Should it be possible to code memos? What about the terminology we use - is it confusing? 


As we grow, it's a challenge to think of all the different ways people want to use Quirkos, including people working on very long qualitative text sources, as well as small snippets from open-ended questions in surveys. We would love to hear your feedback, either by dropping us an email (info@quirkos.com) or by completing this short survey with some questions on what is and isn’t working well for you in Quirkos, and also what features are most important for you in the future.


We are also starting to assemble a team of intrepid beta testers, who have volunteered to try out early releases of Quirkos and test how they work. Since we support so many different platforms (and soon Android as well) it becomes very difficult for us to make sure Quirkos behaves properly on so many different operating systems and computers. So if you were interested in getting involved, again drop us an e-mail, and you’ll get a great chance to shape Quirkos and contribute to making it work just the way you want!


Finally, it’s worth reiterating that these comments really do make a direct difference on what we choose to do. We are a small company, with a smallish number of users at the moment so we can be very responsive. And most of the additions from previous updates were things requested by users. So come and join us, and let’s try and make Quirkos the dream qualitative software for everyone!