navbar 4Resource papers in action research
 

Sources of rigour in action research:
addressing the issues of trustworthiness
and credibility 1 

 

This is a resource file which supports the regular public program "areol" (action research and evaluation on line) offered twice a year beginning in mid-February and mid-July.  For details email Bob Dick  bdick@scu.edu.au  or  bd@uq.net.au  

... in which the issues of rigour in qualitative research generally are addressed by examining some of the ways in which it is achieved in action research

 

Contents

Abstract

Action research may be described briefly as a family of research methodologies with dual aims of action (that is, change) and research (that is, understanding).  These aims can be said to justify a pragmatic definition of rigour: an outcome of those research methods which yield useful models.  From this perspective, such models are those which practitioners can use to predict the effect of their actions.

Action research tends to be participative.  Especially in its more participative forms it is likely to be regarded as credible and trustworthy by its participants.  However, rigour can still be an issue with a wider audience.  There, action research draws upon many sources of rigour which are also to be found in other qualitative approaches.  These include: the use of multiple methodologies, multiple sources of information; multiple processes for data collection and analysis; comparing data and interpretations to those from other sources including the literature; and so on.

Other sources of rigour capitalise on the specific features of action research, particularly its cyclic and action-oriented nature.  Data collection and interpretation tend to co-occur; later cycles can therefore test both data and interpretations from earlier cycles.  Most cycles contain action, each action a test of the assumptions which underlie it.  Attention to these tactics can strengthen the rigour of the research.  Some of the principles which these tactics illustrate may also be of relevance to other varieties of qualitative research.

 

As an adult I was educated first as engineer, then as experimental psychologist.  These enhanced my natural scepticism.  You can probably imagine the implicit agenda of much of this education: that subjectivity and imprecision were dangerous, to be guarded against.  Experimental research had evolved to protect the researcher against these threats.  Objectivity and quantification were to be valued.

Not only that.  I was taught to value pure research over applied, laboratory research over research in the field, academia over the world of practice.  I was led to believe that the conceptual challenges were in the laboratory and the academy.  According to this prevailing view field research was weak and undemanding.

By accident I then became an applied psychologist.  I found that in some field settings experimental research was often difficult or impossible, and sometimes unethical.  As well, the understanding I drew on in my work was multi-dimensional and complex.  Not infrequently it would have taken me a lifetime to research the assumptions I had to make before I could act.

Quasi-experimentation sometimes offered a way out, but often did not.  Good quasi-experimentation is complex, as an examination of Campbell and Stanley (1966) demonstrates.  Poor quasi-experimentation (and there is much of it, I think) is sometimes nearly useless.

Eventually I was drawn towards an examination of action research.  Some of its practices severely challenged the assumptions my earlier education had given me.  It was usually participative.  It was seldom quantitative.  The requirements for action often required an efficiency and economy that seemed to require that I cut methodological corners.

On the other hand it did offer the possibility of combining research with action.  This appealed.  It has continued to appeal.

In practical situations nowadays my first step is often to define the "givens".  I then examine them with a view to finding a way to convert them from liabilities to assets.  Extending this view to action research led me eventually to a change of opinion about the qualities of action research.  I decided that in fact it did have some features which allowed it to pursue high levels of rigour &endash; higher, sometimes, than quasi-experimental methods can achieve in the same setting.

In the rest of this paper I explore ways of making assets of some of its features.  In particular I address its participative nature, its thrust towards action, its use of qualitative data, and above all the need for it to be responsive to the situation and the people who are members of the client group.

Before doing so, however, I should perhaps offer you a brief account of action research as a family of research methodologies.

 

What is action research?

There are several varieties of action research.  Several other methodologies, some derived from it, also share many of its features.  For the most part, they have in common that they are oriented towards achieving two sets of outcomes at the same time: action and research.  You could say that action research is true to label in that it is action and research.

This is to define it by its purpose: to research action, and to action (that is, act on) research.  It might also be defined in terms of its characteristics.  It is above all responsive and flexible.  Indeed, to be used integrally with situations of change management, this is a necessity.  It achieves this responsiveness and flexibility through being cyclic.  In content and in methodology its later cycles are informed by its earlier cycles.

In a word, it is an emergent methodology.  Method and data and interpretation and action develop simultaneously, and from cycle to cycle.

Further, there are cycles within cycles within cycles (within cycles ...).  Larger cycles span whole phases of a research program.  At the other end of the continuum, researchers may be fine tuning their actions moment by moment as their understanding of a situation grows.

In most of its forms, action research is also participative.  In the interests of building commitment to action, the actors are involved at least as informants.  More often than not they also share other roles with the researcher.

Finally, though this is by no means a necessary characteristic, it tends to be qualitative.  Language provides the vehicle by which the participants in an action research program communicate.  That communication often provides a large part of the data which are analysed in coming to an understanding of the situation.

I might sum up the characteristics under four headings.  Action research tends to be participative, qualitative, action-oriented, and emergent.  From some theoretical perspectives each of these might be regarded as a potential threat to rigour.  In what follows, I demonstrate that on the contrary each of them can be drawn upon as a source of rigour.

 

Participative

In most qualitative research people take part at least as informants.  Even with this most elementary form of participation it is possible to use multiple sources of information to increase understanding.

In the spirit of triangulation (Jick, 1979) &endash; the use of multiple data sources &endash; it can be further enhanced by increasing the variety of informants.  For instance, a "maximum diversity" sample of people will often provide a wider sample of data than will a random sample.  Strauss and Corbin (1990) discuss related issues under the label of theoretical sampling in their form of grounded theory.  So does Glaser (e.g.  1992) in his quite different approach.  (This is not the only similarity between action research and grounded theory.)

This approach increases the diversity of information by increasing the diversity of informants.  Along with other varieties of triangulation it is also available in action research.

Most forms of action research seek to involve higher levels of participation than those which involve participants only as informants.  In fact, for many action researchers this is an ideological commitment.  My own position, though based on a strong favouring of high involvement, is that it is limiting to regard participation as obligatory.  I think it offers more options to treat it as a design choice.  For a recent argument along similar lines, see the paper by my colleague Pam Swepson (1998).

One purpose is perhaps more specific to action research: participation increases commitment.  The greater the involvement of participants, the greater their likely commitment to whatever action is planned.  Beyond this, there are advantages to be tapped both in the range of participants, and the depth of involvement.

To take the first of these, I find the concept of stakeholders useful.  Who are the people who have any stake in the situation being researched?  In corporate settings they would include suppliers and customers and perhaps competitors as well as employees.  In community organisations they might include a variety of people who refer people to a program, for instance, or who have other involvement with the program clients.

In any event, deeper involvement of participants can further increase the diversity of data, and eventually therefore of understanding.  There is a continuum of participation ranging from minimal to substantial.  (I have discussed this elsewhere, for example in sessions 4 and 5 of my on-line action research course, 1999.) As participation increases so do the opportunities to use participants as a source of challenge to each other and to the researcher.

It is when participants are involved as co-researchers that the advantages can become greatest.  Participants are then still informants, but can also become interpreters and research designers.  The discussion between different participants, and between participants and researchers, can offer a dialectic.  This can challenge weak or inconsistent data or interpretations.

Such dialectic is the purpose of the participant groups in action research’s close cousin, action learning &endash; see some of the chapters in the recent book edited by Alan Mumford (1997), especially those in Part III.  Argyris (1993; Argyris and Schön, 1996) in his "action science" also seeks to create a climate in which participants can challenge each other and the researcher.  In action learning "sets" (as they are called) or action science groups it is possible to develop a climate of supportive critique through which only quality interpretations are allowed to survive.

 

Qualitative

Ultimately, many of the advantages of participation depend upon the use of conversational language as the data language and the language of interpretation.  Conversational language allows participants access to the data, involvement in its interpretation and application, and the confidence to challenge the interpretations of the researcher.

At the heart of the system I use there lies a simple but useful notion.  I can capture its essence in three principles:

  • focus on the themes which emerge from more than one participant;
     
  • if there is agreement about the nature or the interpretation of a theme, in later data collection test and challenge that agreement by looking for exceptions;
     
  • if there is disagreement about the nature or interpretation of a theme, in later data collection look for explanations for the disagreement.

Let me take each of these in turn and explore their implications in more detail.

Focus.  As distinct from much qualitative research, action research wishes for enough understanding to serve the purpose, whatever that purpose might be.  I’ve read somewhere a partly tongue-in-cheek definition of an engineer as "someone who knows enough to get the job done".  (Perhaps it was Bill Powers who said it.) Especially in those varieties of action research which emphasise action, this is usually the orientation.  The focus on overlap allows selectivity of the data.

(The cyclic nature of action research (see later) also aids economy of data collection.  Data are interpreted as they are gathered.  Carrying forward the interpretations reduces the need to accumulate great amounts of data.)

Testing agreements.  When apparent agreement is found, every attempt is then made to challenge and test that agreement by searching for exceptions.  The effect is to define the boundary conditions of the agreement: its extent.  When exceptions are found, these then provide the disagreement which leads us to ...

Explaining disagreements.  It is here that the quality of dialogue (as described by Bohm, 1996) is to be found.  In conversation which is both confronting and supportive the participants challenge each others’ perceptions and assumptions, or those of the researcher, until an explanation is found which satisfies all of them.

Here again you might note the similarities to grounded theory, especially its data-driven and emergent nature.

The purpose of generating agreement and understanding is to inform action, which follows.

 

Oriented to action

I said earlier that action research was action and research.  It seeks to achieve these two aims continuously.  To do so, it alternates action with critical reflection.  Informed by understanding, the action provides the change.  Out of the attempt to produce change, a greater understanding emerges.

The critical reflection usefully has at least two components.  Reflection after the event draws lessons from the action and its consequences.  What worked?  What didn’t work?  In the light of what worked and what didn’t, which of our assumptions about action were correct?  What do we now understand?  How will we now act?

I have found that, for many people, planning before the event can improve the quality of reflection during and after the event.  I used to encourage people to ask three questions before action.  These were questions about the situation, the outcomes, and therefore the desired actions.  I encouraged people to express their action plans in the form of theoretical assumptions in terms of situation, actions and outcomes.  However, the quality of their reflection seemed to improve further when I added another three questions, to give six in all:

1a  What in my view are the salient features of the situation?

1b  Why do I think those are the salient features?

2a  Given that situation, what do I think are desirable outcomes?

2b  Why do I think those outcomes are desirable in that situation?

3a  What actions do I think will yield those outcomes in that situation?

3b  Why do I think those actions will yield those outcomes in that situation?

The "what?" questions produced the action plan.  The "why?" questions surfaced the assumptions on which the action plan was based.  More aware of their assumptions, people were then better able to pay attention during action to what they were doing and with what consequences.  After the event their critical reflection was often richer, and their planning for the next step was more assured.

In short, it is possible to set up an action research process in such a way that both planned actions and assumptions are tested, immediately, in action.

 

Responsive

It will now be apparent what style of action research I use.  It is driven by the data, by the situation, by the people.  I am often expected to design an intervention or an action research study in detail at the outset.  But at that point I don’t really know enough to do so with any confidence.  Wherever I start may be the wrong place.

By being responsive to the situation, I can refine my interpretations and actions as they proceed.  As importantly, I can also continue to improve my methodology informed by my growing understanding.  So can those I am working with.

Behind this feature lies the imperative of engaging in change.  It might therefore be thought that action research is a very different form of qualitative research.  Different enough, perhaps, that one would take on its practices only with caution.  I think this is an unduly pessimistic view.  It seems to me that grounded theory demonstrates that a similar logic can be extended to some other forms of qualitative research.

 

Rigorous

I have tried to identify here the sources of rigour in action research.  From the perspective of some paradigms, some of its distinguishing features might be regarded as liabilities.  I believe I have shown here that it is also possible to treat them as assets.

  • Participation can mean more informants and therefore richer data.  Involving participants as interpreters and co-researchers allows the assumptions of the researcher to be challenged.
     
  • Qualitative data is to be found in conversation, in dialogue.  If the appropriate climate can be developed, in the dialectic of conversation deeper understanding can emerge.
     
  • Because action research is an action-oriented approach, plans are tested immediately in action.  But so can assumptions be tested.  It is in this sense that it has been said that, if you want to understand a system, try to change it.  Action and research can inform each other.
     
  • Above all, action research is emergent.  As understanding grows, so action becomes better informed, and so does the methodology which is being used.

It is this emergence, this responsiveness to data and situation and people, which suits action research to change programs.  Even without change, however, emergence is a quality which can be built into many research programs.  I hope that I have offered you here some thoughts which you can use to inform your own research practice.

 

Notes

  1.  A paper presented at the Association for Qualitative Research Conference "Issues of rigour in qualitative research" at the Duxton Hotel, Melbourne, Victoria, 6-10 July 1999. [ back ]

 

References

Argyris, C.  (1993) Knowledge for action: A guide to overcoming barriers to organisational change.  San Francisco: Jossey-Bass.

Argyris, C., and Schön, D.A.  (1996) Organisational learning II: theory, method and practice.  Reading: Addison-Wesley.

Bohm, D.  (1996) On dialogue.  London: Routledge.

Campbell, D.T., and Stanley, J.C.  (1966) Experimental and quasi-experimental designs for research.  Chicago: Rand McNally.

Dick, B.  (1999) Stakeholders and participation.  Session 4 of Areol - action research and evaluation on line.  URL http://www.scu.edu.au/schools/sawd/areol/areol-session04.html

Dick, B.  (1999) Achieving participation.  Session 5 of Areol - action research and evaluation on line.  URL http://www.scu.edu.au/schools/sawd/areol/areol-session05.html

Glaser, B.  (1992) Basics of grounded theory analysis: emergence vs forcing.  Mill Valley, Ca.: Sociology Press.

Jick, T.D.  (1979) Mixing qualitative and quantitative methods: triangulation in action.  Administrative Science Quarterly, 24, 602-611.

Mumford, A., ed.  (1997) Action learning at work.  Aldershot, UK: Gower.

Schön, D.A.  (1983) The reflective practitioner: how professionals think in action.  New York: Basic Books.

Schön, D.A.  (1987) Educating the reflective practitioner: towards a new design for teaching and learning in the professions.  San Francisco: Jossey-Bass.

Strauss, A., and Corbin, J.  (1990) Basics of qualitative research: grounded theory procedures and techniques.  Newbury Park: Sage.

Swepson, P.  (1998) Separating the ideals of research from the methodology of research, either action research or science, can lead to better research.  Action Research International, Paper 1.  Available on-line: http://www.scu.edu.au/schools/sawd/ari/ari-swepson.html

 _____