The first step in planning an evaluation is to consider why an evaluation might be needed and clarify the program that is to be evaluated (for example through a program logic model). Once the program or project is understood more thoroughly, this can then direct the evaluation.

Who will conduct the evaluation?

When choosing the evaluator, there is a choice between an internal and external evaluator. There are various advantages and disadvantages to choosing either option. It depends on the context, organization, program/project and skills needed. However, external evaluators may be able to provide a new perspective, criticism and perceived objectivity. An internal evaluator will have the advantage of familiarity and be able to support the use of findings and continuation of evaluation in the organisation.

The main determining factor in the choice of an external or internal evaluator is often the perceived objectivity (where correct or not) by others and the accountability of government funds [1]. However the general skills that are required to conduct the evaluation also need to be considered.

Considering ethical and cultural issues

Ethics in program evaluation refers to the potential risk of harm to people participating in the evaluation, whether as informants or as evaluators. The types of harm can range from loss of privacy, damage to vulnerable groups, or physical or mental harm to informants or researchers. The Australasian Evaluation Society has produced Guidelines for the Ethical Conduct of Evaluation [2].

The potential risk of harm varies with different evaluation designs, and is an important consideration for the quality and ethics of the evaluation project. An application for an external ethics review can involve substantial work and time. Research involving animal and human participants requires approval from a recognised ethics committee. Other data collection may be considered ‘continuous improvement’ rather than research and not require external ethics approval. In each case it is important to consider the potential benefits as well as how an ethics approval process will impact on the cost and timeframe.

If you have further questions or concerns regarding ethics involved in conducting research or evaluation, please contact us.

References

1] Conley-Tyler, M. (2005). A fundamental choice: internal or external choice.
2] http://www.aes.asn.au/