Evaluation
Back to Home

Evaluation

Evaluation

Evaluation is a process that makes evidence based judgements about the actual or likely effects of public interventions
Performance audit Review
Concept
Ref: 36.850

Principles

The main purposes of evaluations

Evaluations contribute to the design of interventions, including providing input for setting political priorities, they assist in an efficient allocation of resources, they improve the quality of the intervention, and they report on the achievements of the intervention (i.e. accountability). Evaluation and audit are not the same. They have similarities and differences.

The evaluator

The evaluation could be carried out by specialists from within the organisation, mainly by external professionals, or a mixture of the two. Unlike performance audits, where objectivity is ensured by the organisational independence of auditors, objectivity and lack of bias tends to be assured through methods and stakeholder participation rather than the organisational independence of the evaluator.

Stakeholder involvement

Reaching judgements about intervention in terms of the criteria above involves adopting the standpoint of one or more of the stakeholders in the intervention; For example:‘Economy’ of an intervention may be framed in terms of its budgetary cost (i.e. adopting the standpoint of the
budgetary authority
), or in terms of the costs to businesses and citizens (i.e. adopting the standpoint of those upon whom the impact of the intervention falls). For this reason, ‘evaluation committees’ or ‘steering groups’ are usually included, made up of stakeholders or their representatives who meet regularly with the evaluator during the evaluation process to help facilitate the work of the evaluator. The information produced during an evaluation is discussed amongst the participants in the evaluation process and is often captured in a series of reports culminating in a final report containing the findings, results, conclusions and recommendations. The report, or its content in a suitably amended format, is then communicated or made available to intended or potential users. Evaluation is thus a key source of information for those within an organisation responsible for managing the implementation of interventions or the developing new policies. These stakeholders are frequently involved in the evaluation process itself and are able to feedback some evaluation results directly without waiting for the finalisation of the report. Equally evaluations can provide information to those responsible for setting political priorities. However, it is always stressed in the evaluation community that evaluation is no substitute for political judgement; it is one source of information available to political decision makers that has to be balanced against others. In addition, public sector organisations also aim to use evaluations cumulatively to improve decision making and acquire knowledge and organisational learning (e.g. by creating a stock of evaluation results, setting up internal networks of users and/or practitioners and performing secondary analysis or meta-evaluation on evaluations carried out).

Types of evaluation criteria

  • Relevance of interventions to the needs they aim to satisfy e.g. Did the intervention address the priorities of the target group?;
  • Economy, efficiency, effectiveness with which
    results
    and
    impacts
    are produced e.g. What were the costs-benefits to budgets and/or stakeholders?, What measures could be taken to improve delivery? Did the intervention achieve its objectives?; and
  • Sustainability of the effects on interventions e.g. What results will persist after the intervention has ceased?
Evaluations usually address questions based on one or more of these types of criteria. In doing this, evaluations frequently examine the cause and effect, the assumptions underlying interventions and the political objectives.

Elements of the context

  • what the subject of the evaluation is - it could be a project, a programme, a regulatory measure, a financial instrument, a policy, a theme or even an organisation.
  • what the relationship between the evaluation and the management system is - in some management systems regular evaluation is an extension of performance monitoring and reporting arrangements in other cases it is used mostly on an ad hoc basis to address specific questions or issues that have arisen;
  • who is sponsoring or carrying out the evaluation - the organisation sponsoring or carrying out the evaluation could either be evaluating its own activities and achievements or those of other organisations.
  • how it is funded - e.g. funding could come from within a programme being evaluated and thus be controlled by programme managers or from funds set aside for evaluation with commissioning and control of funds independent of the management of the programme.
  • what the purpose is - a number of purposes are possible and purposes could be combined within a single evaluation. The most commonly cited purposes are to aid decision making about interventions, the setting of political priorities, and resource allocation and to contribute to accountability and organisational learning.
  • who the intended users are - evaluations can address issues or questions of relevance to one or more stakeholders of the intervention (e.g. managers, political authorities, the addressees of the intervention or the wider public).
  • whether it is prospective or retrospective - evaluation is mostly carried out retrospectively (i.e. after or during the implementation of an intervention) to assess its outcomes, but is increasingly being carried out prospectively to provide assessments of the likely outcomes of proposed interventions (e.g. regulatory measures or expenditure programmes).
These elements of the context of the evaluation are usually outside the control of the evaluation process itself. Nevertheless, evaluation project managers should consider the context when making decisions about the elements of the evaluation process that largely determine its quality i.e.:
  • the management arrangements;
  • the identity of the evaluator;
  • the participants in the process;
  • the issues addressed i.e. the questions asked;
  • the methods employed for collecting and analysing data, and
  • the timetable and arrangements for reporting the results.

Definition

Evaluation is an important element of the internal control system. Evaluation at the Commission is the "judgement of interventions according to their results, impacts and the needs they aim to satisfy". There is no single agreed definition of evaluation. However, there is a general consensus that evaluation is a process that makes evidence based judgements about the actual or likely effects of public interventions such as expenditure programmes, financial instruments and regulatory measures. These judgements are formed on the basis of a disciplined inquiry involving procedures which draw heavily on techniques from the fields of social science and public policy for collecting and analysing quantitative and qualitative information.

Resources

Evaluation societies

International organisations

  • OECD Development Assistance Committee - including development cooperation reviews (including with the European Commission), information on high level performance indicators, details of publications and more links
  • UN Development programme is a portal to the Evaluation Office function within the UNDP and includes evaluation reports, publications, documentation about the evaluation function in UNDP, its evaluation agenda, a search database and links to other evaluation websites
  • UNICEF gives an introduction to evaluation, lists the evaluation policy in the UNICEF, has an evaluation database, identifies good practises as a part of the evaluation process and provides links to other evaluation websites
  • World Bank Independent Evaluation Group Information about the World Bank's evaluation systems, and copies of evaluation reports

Supreme audit institutions

Related documents

Standards GUID 9020
Last Modified: 21/07/2020 15:21   Tags: