31 Jan The integration of means and ends in evaluation research

This article is a follow-up to the one appearing in the last newsletter entitled ‘The separation of means and ends in development: a short case study’. In that article I wrote about the experiences of a member of a learning group, to which I am the adviser, in the context of the SML principle that, in designing and implementing learning and development processes, the processes should match the desired outcomes.

The learning group member is a senior manager in a new health service organisation where there had been an apparent disconnection between the desire in the organisation to develop the culture towards greater consultation, collaboration, cross disciplinary working and interdependency and the method chosen by those at the top of the organisation to encourage this; namely a series of didactic 3 hour presentation sessions delivered to all staff by members of the senior management group. I described how the learning group member, in making her contributions to the delivery of the presentation sessions, had been able to depart from the prescribed ‘script’ by promoting active discussion amongst her audience in response to issues raised by them. The feedback she had received at the time indicated that those attending her presentations had valued them highly, particularly the opportunities to interact with colleagues from other parts of the organisation. She felt that this was evidence that supported her decision to try, if only on a very small scale, to establish a better match between developmental means and desired ends in the way she had done.

In the time that has passed since the first of these two articles the organisation has established a culture change steering group to formally evaluate and take forward the intervention described in the first article. In the leaning group meetings that have taken place during this time the manager has reviewed her experiences as a contributor to the steering group. These have been a source of significant personal learning to her, related to her own personal learning and development goals. However, the part of the continuing story that I want to tell here is about her involvement in the formal evaluation of the intervention. She is carrying this out with a senior, like minded, colleague on behalf of the steering group. Her participation in learning group meetings has included her deliberations on the pros and cons of different approaches to this task and the research methodologies involved. What is particularly interesting is that, along with her colleague, she sees the relevance of the same SML principle to the evaluation. There is exactly the same need to avoid potential disconnections between means and ends when evaluating learning and development processes as there is when designing them in the first place. To this end they have chosen to adopt an action research approach to the evaluation; one which involves ‘collaborative enquiry’.

Evaluation projects of various kinds are common in most organisations, from the evaluation of the introduction of a piece of new technology or a departmental reorganisation to the very familiar evaluation of training courses. However, even if the purpose of such investigations is to improve the way things are done, rather than to prove or disprove their effectiveness, they tend to follow change as an add on, rather than to precede it or to be a wholly integral part of it. Action research, on the other hand, starts with the idea that research should lead to change and that change should be incorporated into the research process itself. It proposes that if we want to understand anything better we should start by attempting to change it. Further, action research has often been linked with action learning where individuals can be helped to use action research methods to pursue their learning. That others can be significant in research and learning is also a feature of some action research in another way. This is the idea that there should be active collaboration between the action researcher and their research subjects. The aim is to increase shared understanding to the extent that researchers and researched can become ‘co-researchers’ in ‘collaborative inquiry’, i.e. the research is done with and for people rather than on them.

These are some of the reasons why action research appealed to our two senior managers as a means to evaluate and monitor the organisation’s efforts to change the culture. However, it appealed to them for another reason too. This reason takes us back pretty much to the beginning of the story. In exploring the potential of action research in this context it became clear to the two managers that elements of the organisation’s vision, such as the importance of being reflective or of being critical of the way things are done in the organisation, including one’s own professional practice, or of sharing understanding and knowledge, could be mirrored in the evaluation process itself. Rather than the desired culture being no more than rather high sounding statements that staff were told about in a presentation, and actually had some difficulty remembering after the event, it had the potential to begin to become real to those participating in the evaluation research. In this way, their evaluation seeks to contribute actively to the cultural change rather than merely comment on how well it has been done so far.

Ben Bennett