Framing the evaluation involves determining its scope and writing the key evaluation questions. In an objectives-based evaluation, the focus will generally be on whether the program has met the expected outcomes at a minimum.

Key Evaluation Questions

Key Evaluation Questions (KEQ) are the overall theme or topic area that the evaluation will focus on, such as effectiveness or efficiency etc. Usually there are about 5 to 7 Key Evaluation Questions. Most issues generally come under 5 to 7 themes, if there are any more than this the evaluation can be too large in scope to be practical.

The evaluation questions are broken into the key evaluation questions and then sub questions to define that question further. Sub questions specify particular areas of interest and provide guidance as to what questions should be considered to answer the Key Evaluation Question. When you formulate a question you can ask, “will this question answer another broader question? Or do the other questions contribute to answering it?”

It’s important to remember that these are not the questions you would ask someone in an interview. The interview questions are targeted to gather information that will be aggregated and analysed to answer these overarching questions.
The process of creating KEQs involves looking at:
»» what you absolutely need to know for stakeholders or reporting requirements
»» what you want to know
»» and then finally what you’d like to know if there was the time and budget that would allow it.

Conducting a stakeholder analysis and involving them in the design of the evaluation can ensure the evaluation has a useful process and product. A stakeholder analysis can look at who needs to be engaged or informed throughout the evaluation, what expectations will they have and how will they benefit.

For refining the Key Evaluation Questions it is helpful to use the program logic or key theme areas.

Program Logics and Key Evaluation Questions

A logic model identifies what the project is trying to achieve, and at what level these achievements or outcomes might be expected. After you have developed your logic model, you can use this to set the key evaluation questions and indicators of their achievement (if used). This may also involve identifying the assumptions behind the logic that should be tested or identifying what components of the program need to be evaluated. Evaluation questions can be targeted at any level of the program logic. A number of examples are provided in the table below.

General theme areas of Key Evaluation Questions

The following five themes encompass most evaluation questions.

Next steps…

Determine what information you need.

CFRE works directly with agencies and project teams to help them achieve their goals and priorities. From senior leadership groups to small program teams, our consultants walk alongside you to guide the design, development and improvement of your programs and services.

For more information, contact the team at cfre@ds.org.au