The validity chain: rigour in interview
data
This is a resource file which supports the regular public program "areol" (action research and evaluation on line) offered twice a year beginning in mid-February and mid-July. For details email Bob Dick bdick@scu.edu.au or bd@uq.net.au
... in which I identify some of the ways in which the validity of interview data can be increased
In many forms of qualitative research, there is an easy and reasonably safe defences against accusations of poor rigour. You can adopt a pure constructivist position, support it by reference to the literature, and choose your examiners carefully. (I wouldn't personally choose this approach; but I believe it is defensible with some examiners.)
If you are doing action research, this defence may be more difficult. Action research is often intended to lead to change as well as to understanding. To my mind, that presumes that there is some sort of reality to be changed. Further, attempts to change it presumably require that your perceptions of that reality are, in some sense, accurate.
In other than an extreme constructivist position, I think that there are potential threats to the validity of your study. Part of your task in your thesis is to be able to argue that you have done enough to overcome or control these threats.
To take just one example...
Consider what can occur in interviews and conversations with informants. Threats to the validity of your claims exist at each of the gaps between your informants' experience and your eventual conclusions. Consider the following sample of gaps and threats and strategies:
Your informants may misperceive or selectively perceive or misunderstand their situation;
It helps if you too were present in the situation they are reporting. If so, you can increase their perception by skilful briefing, by having observers, by debriefing in such a way that perceptions are pooled, by drawing their attention to certain happenings, and so on.To improve your interpretations of the situation they experienced you may be able to use triangulation, of many varieties. Sometimes you can use processes which allow your informants to recognise and deal with the inconsistencies inherent in their account. But by and large, if they didn't perceive it, or didn't perceive it accurately when it happened, there may not be much you can do.
They may not have stored in memory all of what they initially perceived.
Here, your strategies are much as for the first gap, above.
They may no longer be conscious of what they originally remembered.
There are a number of strategies you can use to help them recall those aspects of the situation which have faded from conscious memory. Most depend on the quality of the rapport you have established with them.Assuming you've established good rapport, you can improve their recall by such strategies as ...
- allowing enough time for their recall to build;
- by using multiple questions to approach an issue a little at a time;
- by first asking about those aspects of the situation which are most easily recalled (this will then aid recall of other aspects);
- by using other multiple-step elicitation procedures, for example first by asking them to recollect the situation in detail and then by asking the more difficult questions about what it meant.
For whatever reason, they may be reluctant to report to you all that they recall.
Here again, quality rapport will help. Their trust in you is also important. You may be able to enhance it by being clear about:
- who you are;
- what your task is;
- why you are doing it;
- what arrangements you have reached with the person who hired you;
- who gets the data, and in what form;
- as far as you can, what you understand to be the motives of those who hired you;
- any guarantees about anonymity and/or confidentiality;
- what access they have to any data that is reported to someone else.
They may not report all that they are able and willing to.
This can partly be overcome by good questioning techniques. For instance you can ...
- use open-ended or closed-ended questions appropriately;
- plan your questions so that they cover fully the aspects of the situation that interest you;
- use multiple questions, differently phrased, to query aspects which you expect to be important;
- use follow-up interviews to clarify ambiguities and the like;
- derive some or all of your questions from multiple theories, using each theory to check that you have sufficient coverage of the aspects of interest.
However, even if you do all of this, you may sometimes have no way of knowing what questions to ask. To overcome this you can begin each interview in as open-ended a way as possible, and keep the informant talking. You can save the planned questions for later in the interview.
You may not hear or understand it well.
You can avoid this to some extent by listening well, for instance. You can rephrase what you think they said in your own words, in such a way that it is easy for them to correct it if you have it wrong. You can record the interview and ask a colleague to interpret it independently.
You may not record it accurately.
Some care in documenting your study can help. Having high-fidelity recording equipment and testing it beforehand is useful (though this has other costs; personally I'd rather conduct two interviews than record and play back one interview).
You may misinterpret it.
For instance, perhaps you and they use language differently. Perhaps you do not understand their customs or culture. Perhaps you allow your own interpretation to be determined by the theories you use, or by prior information you've received, or by your assumptions about or your perceptions of your informants.You can reduce these biases by ...
- becoming more aware of your biases through regular critical reflection, seeking feedback from colleagues, using multiple theories, and so on;
- taking time to learn the local customs, language and usage;
- by involving different interviews in the interview, perhaps including people drawn from the community or organisation in which you are doing your research;
- most importantly, looking out for and actively seeking information which disconfirms you own assumptions and interpretations.
You can also do much to overcome your own biases of interpretation by asking your informants to help you with interpretation.
You may be mistaken about the implications and conclusions.
Checking your conclusions against related literature may help. Within an action research approach, you can use a short cycle so that implications are tested in action soon after they are formed.
And so on. These are all just examples &emdash; I could have identified some further potential gaps, or (for each of the gaps above) suggested other ways of making your data collection more robust.
Other forms of information collection have their own threats to validity and ways of managing them.
_____
Copyright (c) Bob Dick 1999. This document may be copied if it is not included in documents sold at a profit, and this and the following notice are included.
This document can be cited as follows:
Dick, B. (1999)
The validity chain [On line]. Available
at
http://www.uq.net.au/action_research/arp/validchain.html
