[section separator="true"]
[section-item 9]
[row]
[column 12]
[toc-this]
Definitions
The principle of effectiveness concerns the extent to which the objectives pursued are achieved through the activities undertaken.
Risks and issues related to achieving the stipulated aims or objectives
Issues of effectiveness arise when an entity or intervention does not produce the expected outputs, results or impacts. General risks in this area can include:
- faulty policy design, e.g. inadequate assessment of needs, unclear or incoherent objectives, inadequate means of intervention or impracticability of implementation; and
- management failures, e.g. objectives not being met, management not prioritising the achievement of objectives.
An audit of effectiveness is therefore concerned with measuring the extent to which the different types of objectives have been achieved:
- operational objectives (outputs): the audit assesses the extent to which the intended outputs have been produced and normally involves the examination of the operations internal to the organisations which are responsible for the implementation of the intervention;
- immediate objectives (results): the audit assesses whether the intervention had clear and positive results for direct addressees at the end of their participation and normally involves examining monitoring information produced by the implementing organisations as well as obtaining information from direct addressees;
- intermediate and global objectives (impacts): the examination extends beyond the boundaries of the audited entity and seeks to measure the impacts of the public intervention. This requires the audit to take account of exogenous factors and to produce evidence that the impacts observed are actually produced by the public intervention concerned and are not the consequences of such factors.
The audit of effectiveness will therefore concentrate on outputs, results or impacts:
Assessing impact is difficult
There can be considerable difficulty involved in assessing the impact of an intervention, i.e. the extent to which the global and even intermediate objectives of this intervention have been achieved. The difficulty arises because the objectives are usually expressed in such broad terms that they cannot be associated with measurable indicators and the extent of their achievement is therefore difficult to verify. Likewise, when the objectives are more clearly identified, the collection and analysis of the required audit evidence would involve disproportionate audit resources if this information is not readily available within the audited entity. Furthermore, it is difficult to assess whether the impacts observed are really the effects of the intervention rather than exogenous factors. In such instances, the audit approach should therefore first consider whether relevant and reliable evaluation information is available and can be used as audit evidence.
Assessing outputs or results is often more practical
A more feasible audit objective will often be to assess the outputs or results of an intervention, i.e. the extent to which operational or immediate objectives have been achieved. Provided that the objectives are "SMART" - specific, measurable, achievable, relevant and timely - in accordance with the Commission's performance and risk management approach, and that their achievement is monitored by performance indicators, this is likely to provide a clear and suitable reference basis for assessing effectiveness.
Resources
Example of faulty policy design: “Centrally managed EU interventions for venture capital: in need of more direction
[link title="(SR%2017%2F19)" link="https%3A%2F%2Fwww.eca.europa.eu%2FLists%2FECADocuments%2FSR19_17%2FSR_venture_capital_EN.pdf" icon="external-link" /]
.”
Audit question: Had the design of the interventions been underpinned by impact assessments and evaluations and had they been properly implemented?
Conclusion: Interventions were not based on a proper assessment of market needs. In our view, increasing budgetary resources for venture capital funds without properly quantifying the funding gap may lead to the risk that such funds cannot be absorbed.
Example of management failure: “EU support to Morocco - Limited results so far
[link title="(SR%2009%2F19)" link="https%3A%2F%2Fwww.eca.europa.eu%2FLists%2FECADocuments%2FSR19_09%2FSR_Morocco_EN.pdf" icon="external-link" /]
.”
Audit sub-question: Were the objectives of EU budget support (BS) to Morocco achieved?
Sub-conclusion: Less than half of the BS targets had been achieved. Although progress had been made towards the objectives of the BS programmes in the sectors audited, it was limited. The implementation of ministries’ budgets had improved, but significant amounts remained to be spent. This puts the added value of BS into question.
Audit sub-question: Was the objective of visibility of EU support achieved?
Sub-conclusion: Visibility of the EU’s financial cooperation, in particular as regards the results achieved, was insufficient. For example, no joint press releases were issued and there were no signs acknowledging EU support. While the political dialogue was suspended, the Ministry of Foreign Affairs had asked the other ministries not to communicate openly about any EU support received.
[/toc-this]
[/column]
[/row]
[/section-item]
[section-item 3]
[row]
[column 12]
[panel panel-style="boxed" title="Related%20documents" icon="book" class="ref-panel"]
[standards]
[link new-window title="ISSAI%20300%2F11" link="https%3a%2f%2fwww.issai.org%2fwp-content%2fuploads%2f2019%2f08%2fISSAI-300.pdf%23page%3D8" /]
[link new-window title="ISSAI%203000%2F17-20" link="https%3a%2f%2fwww.issai.org%2fwp-content%2fuploads%2f2019%2f08%2fISSAI-3000-Performance-Audit-Standard.pdf%23page%3D8" /]
[link new-window title="GUID%203910%2F7-9" link="https%3a%2f%2fwww.issai.org%2fwp-content%2fuploads%2f2019%2f08%2fGUID-3910-Central-Concepts-for-Performance-Auditing.pdf%23page%3D6" /]
[link new-window title="GUID%203910%2F42-45" link="https%3a%2f%2fwww.issai.org%2fwp-content%2fuploads%2f2019%2f08%2fGUID-3910-Central-Concepts-for-Performance-Auditing.pdf%23page%3D19" /]
[/standards]
[rules]
[link new-window title="Art%2033%20FR" link="https%3A%2F%2Feur-lex.europa.eu%2Flegal-content%2FEN%2FTXT%2F%3Furi%3DCELEX%3A32018R1046%23033" /]
[/rules]
[/panel]
[/column]
[/row]
[row]
[column 12][/column]
[/row]
[row]
[column 12]
[toc fixed="true" selectors="h2%2Ch3" class="basic-toc" /]
[/column]
[/row]
[/section-item]
[/section]
