[section separator="true"]
[section-item 9]
[row]
[column 12]
[toc-this]
Principles
The approach to gaining this knowledge will vary, depending on the nature of the subject and the amount of knowledge that the team possesses at the outset, including information available in the permanent files and that already obtained by the ECA's financial auditors. It might involve simple documentary review and a single brainstorming session when a great deal is already known or a much larger data-gathering exercise when the topic is new or complex. When acquiring the knowledge, auditors should pay attention to conditions that may be indicative of potential problems. More in-depth knowledge will help the auditor in the risk assessment.
Instructions
Objectives and indicators
An understanding of the [a-glossary term="objective"]objectives[/a-glossary]
and logic of the intervention is the starting point in planning a performance audit. When identifying objectives and indicators, the auditor should consider the following sources:
- The legislation setting up the programme or policy: Objectives, indicators and targets can be set out to a varying degree of detail in the legislation or mentioned in the non-binding recitals.
[link new-window title="Strategic%20and%20management%20plans" link="https%3A%2F%2Fec.europa.eu%2Finfo%2Fstrategy%2Fstrategy-documents%2Fstrategic-plans-management-plans_en" icon="external-link" /]
and annual activity reports of the Directorate-General responsible for the programme or policy: These documents should detail objectives, indicators and targets (in case of plans), achievements (in case of activity reports) and planned/used resources for each “Activity” defined in the [link new-window title="budget%20of%20the%20Commission" link="https%3A%2F%2Feur-lex.europa.eu%2Fbudget%2Fwww%2Findex-en.htm" icon="external-link" /]
.- The explanatory memorandum and impact assessment accompanying the legislation proposal may also provide indications.
[toggles]
[toggle title="Pay%20attention%20to"]
- legal basis that is uncertain, complicated, or subject to significant change
- gaps in the needs assessment
- lack of, unclear objectives regarding economy, efficiency and effectiveness
- objectives without indicators
- indicators do not measure achievement of economy, efficiency, and effectiveness
- indicators encouraging the wrong behaviour (e.g. pursuit of short-term goals to the exclusion of long-term goals)
- objectives are not clearly prioritised,
- objectives contradictory or incompatible, either in the EU policy or programme, or between EU and national priorities
- no clear link between objectives and activities
- complex activities or programme delivery method
- Identifying final beneficiary is difficult
[/toggle]
[/toggles]
Materiality
The human, administrative and financial resources allocated to the audit area should be determined to confirm its materiality:
- For audit topics using grants, this involves analysing budgetary appropriations allocated, and amounts committed and paid.
- For audit topics using financial instruments, this involves analysing not only budgetary appropriations and amounts paid from the EU budget, but also amounts made available to final recipients;
- For audits of topics with insignificant EU budget, confirming the materiality by nature and context of the topic requires collecting information about the regulatory positive or negative impact on economic actors or member states. Sources of information may be Commission’s own estimates in the impact assessment, scientific studies, official statistics, or viewpoints from experts in the field.
[toggles]
[toggle title="Pay%20attention%20to"]
- activities funded by other sources (other EU instruments or co-financing)
- large number of transactions
- high proportion of commitments or payments made late in the financial year
- activities involving large amounts of cash or high-value goods
- insufficient, under-qualified, inexperienced or poorly motivated staff and/or inadequate recruiting procedures
- difficult to determine the cost of inputs
- expenditure increasing beyond expectations
- budget targets are consistently missed to a significant degree
- significant cost and time overruns on projects
[/toggle]
[/toggles]
Responsibilities of key actors
A cornerstone of performance audit is holding auditee management accountable only for that over which they have control; it is therefore fundamental to define the auditee. The audit team should identify management responsibility for the intervention on the part of the Commission, other EU institutions, bodies and agencies, member states, beneficiary states and international organisations:
- In instances of shared or indirect management between the Commission on the one hand and member states, beneficiary states or international organisations on the other, the Commission may have little real direct involvement in the on-going management and implementation of co-financed interventions, although it always has overall
[link new-window title="responsibility" link="https%3A%2F%2Feur-lex.europa.eu%2Flegal-content%2FEN%2FTXT%2FPDF%2F%3Furi%3DCELEX%3A12012E%2FTXT%26from%3DEN%23page%3D140" icon="external-link" /]
for managing the EU budget. In this context, the ECA requires its performance audits to take as their starting point the Commission's management, including the methods implemented, to ensure that management by member states is appropriate.
- In instances where the EU budget is insignificant, the legislation defines the Commission’s supervisory role and implementing powers, which may be subject to an
[link new-window title="opinion%20of%20a%20committee" link="https%3A%2F%2Fec.europa.eu%2Finfo%2Fimplementing-and-delegated-acts%2Fcomitology_en" icon="external-link" /]
of representatives of EU member states and which can be more or less binding on the Commission.
[toggles]
[toggle title="Pay%20attention%20to"]
- new initiatives set up in haste
- rapid implementation of the programme after the decision on the legal base or, at the other extreme, slow implementation
- activities with which the audited entity has no or limited experience
- imposition of unwanted responsibilities upon organisations, administrations or beneficiaries
- unclear division of responsibilities between the organisations
- activities of different organisations are poorly coordinated
- large number of sub-contractors
[/toggle]
[/toggles]
Key management and control processes
The audit team:
- Determines the key management and control processes for the activity/activities which concern the potential audit topic. This can be done by reviewing regulations and internal procedure manuals and by way of interview.
- Identifies the types of information and reports currently used by the auditee for the management and control of its activities.
- Identifies the IT systems producing management information and how the management ensures the reliability of these systems.
- Inquires about the auditee’s
[tooltip title="a%20system%20that%20contains%20a%20list%20of%20files%20and%20records%20maintained%20by%20the%20organisation"]formal registry system[/tooltip]
to mitigate the risk that some important documents are overlooked.
- In addition, the audit team needs to become familiar with the risk management of the auditee. For instance each Directorate-General in the Commission is required to systematically analyse risks in relation to its main activities at least once a year, develop appropriate action plans to address them, and assign staff responsible for implementing those plans. These are internal Commission documents, but risks are reported annually in the
[link new-window title="management%20plans" link="https%3A%2F%2Fec.europa.eu%2Finfo%2Fstrategy%2Fstrategy-documents%2Fstrategic-plans-management-plans_en" icon="external-link" /]
.
[toggles]
[toggle title="Pay%20attention%20to"]
Organisation and management
- organisation often subject to structural changes
- decentralised management & complex organisation
- absence of common supporting administrative structure
- geographically dispersed organisation
- objectives not communicated to all levels of management
- ethical values/integrity poorly established (tolerance towards irregularities, no code of conduct)
- strong pressure on management to achieve unrealistic objectives or meet unrealistic deadlines (e.g. high rate of commitment of budget appropriations)
- lack of job descriptions
- no policy for staff rotation
- lack of supervision and monitoring
- supervision and control functions non-existent or unsuitable
Internal control
- eligibility/selection criteria unclear or inconsistent with objectives (too wide, too restrictive or not relevant)
- complex contractual or tendering rules
- complex or lack of control systems to monitor economy, efficiency and effectiveness
- management and financial information poor, not used or used inappropriately
- operations not fully subject to the usual controls
- limited checks in relation to activities traditionally considered to be particularly prone to irregularities (e.g. urgent operations)
- on-the-spot inspection or monitoring rights not taken up or infrequently used
- contracts frequently awarded without competition
- critical factors that could endanger achievement of objectives not regularly assessed
- lack of a cost accounting system
- inadequate audit system (coverage, quality, reporting, follow-up)
Information and IT systems
- information technology that is obsolete, highly complex, or includes many different and/or incompatible computer systems
- Incompatible IT systems of the entities involved in managing the programme
- lack of or complaints about management information system or a system to monitor actual performance against plan
[/toggle]
[/toggles]
Previous audits and evaluations
The audit team should consider previous audits and evaluations undertaken in the subject area, both in order to avoid duplication of work and to follow up on significant findings and recommendations that relate to the potential audit question. Such audits and evaluations may include those performed by the ECA, Commission's Internal Audit Service or [link new-window title="Commission%E2%80%99s%20interim%20or%20final%20evaluations" link="https%3A%2F%2Fec.europa.eu%2Finfo%2Flaw%2Flaw-making-process%2Fevaluating-and-improving-existing-laws%2Fevaluating-laws_en" icon="external-link" /]
. Auditors can use evaluations and [multi-link title="reports%20of%20other%20SAIs" title_1="EUROSAI%20database" link_1="https%3A%2F%2Fwww.eurosai.org%2Fen%2Fdatabases%2Faudits%2F" title_2="Environmental%20audits" link_2="https%3A%2F%2Fwgea.org%2Faudit-database%2Fintosai-wgea-audit-database%2F" title_3="IT%20audits" link_3="https%3A%2F%2Fwww.intosaicommunity.net%2Fwgita%2Faudit-collection%2F" /]
[icon color="%2328A86C" name="external-link" /]
to inspire themselves about possible audit methodology.
[toggles]
[toggle title="Pay%20attention%20to"]
- evidence of poor performance (e.g. 3Es) or management (e.g. legality and regularity, fraud) in the past
- Issues with European added value or with EU funds replacing national government expenditure
- poor sustainability (no ownership; projects set up without proper dialogue with beneficiaries; beneficiaries highly dependent on EU)
- evidence of weaknesses in the design or performance of control systems
- employment of resources that are unnecessary, of too high quality, or that could have been obtained at lower cost
- past audit findings not addressed
- lack of, or poor, evaluation, and/or or no follow-up of evaluation results.
[/toggle]
[/toggles]
Resources
[icons-list icon-size="2" separator="line" icon-vertical-alignment="middle" vertical-alignment="middle"]
[icon-list-item title="ECA%20special%20reports%20in%20BibliotECA" description="collection%20of%20ECA%20special%20reports%20with%20a%20search%20function." link="https%3A%2F%2Feca-europa.primo.exlibrisgroup.com%2Fdiscovery%2FcollectionDiscovery%3Fvid%3D352ECA_INST%3AVU1%26collectionId%3D81212743590006191" icon="external-link" linking="new-window" /]
[icon-list-item title="Database%20of%20other%20SAIs'%20audit%20reports" description="contains%20basic%20information%20and%20links%20concerning%20audit%20reports%20issued%20by%20EUROSAI%20members." link="https%3A%2F%2Fwww.eurosai.org%2Fen%2Fdatabases%2Faudits%2F" icon="external-link" linking="new-window" /]
[icon-list-item title="Database%20of%20environmental%20audits" description="maintained%20by%20the%20INTOSAI%20Working%20group%20on%20environmental%20auditing.%20Any%20INTOSAI%20member%20can%20contribute%20to%20the%20database." link="https%3A%2F%2Fwgea.org%2Faudit-database%2Fintosai-wgea-audit-database%2F" icon="external-link" linking="new-window" /]
[icon-list-item title="Database%20of%20IT%20audits" description="maintained%20by%20the%20INTOSAI%20Working%20group%20on%20IT%20audit.%20Any%20INTOSAI%20member%20can%20contribute%20to%20the%20database." link="https%3A%2F%2Fwww.intosaicommunity.net%2Fwgita%2Faudit-collection%2F" icon="external-link" linking="new-window" /]
[/icons-list]
[/toc-this]
[/column]
[/row]
[/section-item]
[section-item 3]
[row]
[column 12]
[panel panel-style="boxed" title="Related%20documents" icon="book" class="ref-panel"]
[standards]
[link new-window title="ISSAI%20300%2F37" link="https%3a%2f%2fwww.issai.org%2fwp-content%2fuploads%2f2019%2f08%2fISSAI-300.pdf%23page%3D26" /]
[link new-window title="ISSAI%203000%2F100" link="https%3a%2f%2fwww.issai.org%2fwp-content%2fuploads%2f2019%2f08%2fISSAI-3000-Performance-Audit-Standard.pdf%23page%3D25" /]
[/standards]
[/panel]
[/column]
[/row]
[row]
[column 12][/column]
[/row]
[row]
[column 12]
[toc fixed="true" selectors="h2%2Ch3" class="basic-toc" /]
[/column]
[/row]
[/section-item]
[/section]
