The Centre for Family Research and Evaluation (CFRE) was contracted in 2018 for a data project managed by the Outcome Practice and Evidence Network (OPEN).
Based at the Centre for Excellence in Child and Family Welfare, OPEN connects practice experience with research and evaluation expertise including facilitating small projects to assist agencies to improve their outcome measurement and use of evidence. In this instance, a Non-Government Organisation based in Victoria requested support in reviewing the outcomes measurements currently being used to evaluate their Strengthening Family Connections (SFC) Program.
The objective of this project was to revise the current evaluation strategy for the SFC program and make recommendations to reduce the volume of questions and ensure measurement tools remained fit for purpose. To ensure that tools are measuring the stated program objectives, CFRE consulted with program staff to gain insight into the program aims and objectives, as well as the agency’s current experience of administering the questionnaires. A thorough review of program documents was also carried out.
This review considered various issues of data collection including accuracy, reliability, validity, age-appropriateness, and the burden on staff and participants. If not targeted well, data collection can pose a real barrier for program delivery and can disrupt the rapport that facilitators work hard to build with participants. The proposed changes to the adult and child questionnaires put the organisation in a better position to improve the overall administration of the surveys and ensure that measurement tools are targeted more specifically toward the intended program outcomes. A number of further practical recommendations were also made.