Who evaluates

On this page

Evaluation is everyone’s responsibility

Everyone working in, or for, a Commonwealth entity has a responsibility to consider:

What are the objectives of the program or activity I am working towards?
Are there ways to design better solutions, deliver better services and provide stronger policy advice?
How is progress being monitored and success being measured?
Is my work making a difference - what outcomes for Australians are being achieved?

Factors to consider

The purpose, nature and design of an evaluation activity will influence who needs to be involved. Factors that need to be considered include:

  • the type of evaluation required
  • the skills, resources and expertise necessary to conduct the evaluation
  • the intended use of evaluation findings (who are the stakeholders and how will the results be used?)
  • the level of evaluation capability in an entity or company, or available across the Commonwealth more broadly.

If you are a Commonwealth official with a role or an interest in evaluation, you can join the Commonwealth Evaluation Community of Practice and other relevant professional networks to help find the skills and expertise you need to carry out your evaluation.

Different roles and responsibilities

The breadth, scale and diversity of activities managed across the Commonwealth is significant, which means lots of people have very different roles and responsibilities.

To embed a culture of evaluation, everyone working in, or for, a Commonwealth entity or company has to be involved. Individuals should be continually questioning how well government activities and programs are meeting their objectives, and how they could be improved. This is a critical part of good performance management and accountability.

Some of the different roles associated with evaluation are outlined below, including potential areas of interest.

It should be noted that these are simply examples and may differ between entities.

Role Potential areas of interest
Project manager I have to commission and manage an evaluation and I want to know how I can talk to evaluators.
Policy officer I want to know if my policy is achieving the outcomes I thought it would and what lessons will help me to develop new policies.
Grant administrator I want to know if my grant program is on track, reaching the people it should and having the outcome I want.
Budget officer I want to know what baseline data will be used to assess the impacts of a new program or activity, and what evidence will be available to demonstrate that the anticipated benefits have been realised.
Regulator I want to know there is evidence to demonstrate my regulatory activity is achieving its objectives, and that I am collecting and using data to provide performance insights and drive continuous improvement. 
Internal evaluator I want to support my entity to develop and deliver great policy and programs through the use of high-quality evaluation and building internal evaluation capability.
Program manager I want to know if my program is on track and reaching the intended targets and how its delivery might be improved.
Senior manager I want to know what is working and why, and is there credible evidence to back this up?
Accountable Authority (Head of entity) I want to know that what I tell my minister, the Parliament and the Australian people is supported by robust evidence and ethical insights.

Managing an evaluation

Evaluations can be carried out by the program manager, an internal evaluation unit (if available), an external provider, or a mixture of all of these.

Multi-disciplinary teams

Ideally, individuals who are (or will be) recruited to carry out monitoring and evaluation tasks will have the time, resources and skills to do so. This will typically involve a multidisciplinary team.

The team must also be aware of any specific ethical and cultural considerations relevant to the evaluation, and ensure that there is the right mix of skills, experience and cultural knowledge to contribute to a credible and useful evaluation.

Different roles

There are different roles involved with managing an evaluation, including:

  • commissioning the evaluation (for example, receiving required authority for the evaluation to be undertaken, identifying and obtaining financial resourcing, scoping the requirements)
  • project/contract managing the evaluation (that is, managing and engaging with who is undertaking the evaluation)
  • undertaking the evaluation – designing and executing the evaluation framework/design, including obtaining ethics approval if required, undertaking data collection and analysis and delivering reports.

It is worth noting that evaluations can come with risks. For example, not knowing enough about evaluation, not giving enough consideration to ethical requirements and not having a good understanding of cultural context. Many of these things can be solved using the right skills, good communication and working together.

With evaluation, the processes you follow are as important as the content that is produced.

Decision tree: Who should conduct the evaluation

A graphic which shows two graphics at the top, labelled, “who should conduct the evaluation?” and beneath it, “do we have the expertise to find out if the program or activity works?”, Three sections with boxes, pros and cons, flow beneath this and read “Yes, enough”- “Evaluation done entirely in-house”-“Pros- existing, deeper knowledge of entity, program and systems, - may -reduce costs, - makes use of existing practice wisdom (particularly if the entity has an internal evaluation unit), -are already part of the entity structure and culture, - can draw on existing relationships with clients and stakeholders, -opportunity to build skills and capability” – “Cons- may not have sufficient skills to do the evaluation well, - may need to redistribute internal staff and resources, possible impact on service or program delivery, -adds to staff workload, -can be difficult to be objective”. “Yes, partially”- “Partly in-house, partly in collaboration with an external evaluator or other relevant specialists” – “Pros-introduces an outside perspective, adds a degree of objectivity, increases perception of independence of the evaluation, makes use of inhouse knowledge and systems, -draws on consultant expertise and experience of other evaluations, -allows relevant technical skills to be sources externally (eg data specialists, survey specialists), -reduces need to redistribute internal resources, -opportunity for staff to acquire new skills and capability”, “Cons- can be costly, - lack of knowledge and understanding of the program target group and time required to get up to speed, - evaluator seen as an outsider”. No- evaluation done solely by an external evaluation professional” “Pros- increases objectivity and offers an external perspective, -reduce workload less direct involvement of staff, - make use of greater knowledge and expertise in technical aspects of evaluation, - can be more efficient, -may be less expensive than doing evaluation badly”. Cons-“ can be expensive, -lack of knowledge and understanding of service type, entity culture etc, - lack of knowledge and understanding of the program/target group, -time required to get up to speed”.

Different management tasks

Different management tasks arise depending on who is involved in certain evaluative activities. For example, when using an external evaluator you will need to ensure:

  • an appropriate process is used to select and manage them according to the Commonwealth Procurement Rules
  • they have the right clearances and systems in place to conduct the evaluation appropriately
  • they consult with internal experts to access the right knowledge about the particular government program or activity
  • the approach and outputs of the external evaluators are sound and deliver value-for-money.

If internal staff and/or intended beneficiaries are involved, you should ensure processes are well documented, in line with your entity’s internal processes and procedures. It is also better practice to conduct relevant training in specific evaluation options to ensure that quality and ethical standards are maintained.[1]

Options on who to involve

Some options for who to involve in an evaluation 

  • Community: conducting an evaluation by using the broader community or groups of intended beneficiaries
  • Expert review: conducting an evaluation by using someone with specific content knowledge or expert judgement and professional expertise
  • External consultant: contracting an external consultant to conduct the evaluation
  • Hybrid – internal and external evaluation: a combination of internal staff and an external (usually expert) opinion to jointly conduct an evaluation
  • Internal staff: conducting an evaluation using staff from the implementing agency
  • Learning alliances: bringing together different groups to conduct the evaluation
  • Peer review: conducting an evaluation using Individuals/organisations who are working on similar projects.

There are pros and cons of using different options for who will do an evaluation, so you will need to weigh up each option to decide what is best for your program and/or activity.  If you have an in-house evaluation unit in your entity, they will be able to assist with your choice.

Support

[1] Decide who will conduct the evaluation (Better Evaluation)

Evaluation Handbook December 2017 (New Zealand Department of the Prime Minister and Cabinet)

Back to top