[section separator="true"]
[section-item 9]
[row]
[column 12]
[toc-this]
Principles
Benchmarking in an audit should focus only on those aspects that are within the control of the leadership of the organisations audited.
Definition
Benchmarking is a process for comparing an organisation’s (programme’s) methods, processes, procedures, products and services against those of organisations (programmes) that consistently distinguish themselves in the same categories.
It helps to identify opportunities to improve the [link title="economy" link="%2Faware%2FPA%2FPages%2FConcepts%2FEconomy.aspx" /]
, [link title="efficiency" link="%2Faware%2FPA%2FPages%2FConcepts%2FEfficiency.aspx" /]
, [link title="effectiveness" link="%2Faware%2FPA%2FPages%2FConcepts%2FEffectiveness.aspx" /]
and performance of an organisation. It is undertaken for two reasons: either to learn and improve or to get and keep a competitive advantage.
Benchmarking may be used to:
- stimulate an objective review of processes, practices and systems;
- develop criteria and identify potentially better ways of operating;
- lend more credibility to audit recommendations.
Benchmarking involves making an objective and reasoned judgement based on comparisons, usually resulting in a ranking of performance.
However, a simple comparison of spending in member states in a given policy area, without any analysis, is not benchmarking, as readers must form their own judgement on what the comparative data means.
Benchmarking in private and public sectors
Benchmarking in the private sector usually concentrates on organisational performance and takes the form of standards, results and/or process benchmarking. Such benchmarking exercises have [link title="best%20practice" link="%2Faware%2FPA%2FPages%2FConcepts%2FBest-practice.aspx" /]
as their primary focus, with the aim of increasing an organisation’s profit or ensuring business continuity. It is used as an internal, voluntary management tool whose results are kept confidential.
Benchmarking in the public sector unlike in the private sector, is often imposed on an organisation, either as part of the budgeting and or/audit process, or it is politically driven. A disadvantage of compulsory benchmarking is that performance measures drive behaviour (what gets measured gets done), so that managers often concentrate on measurable results and indicators rather than actual results. (For example, ensuring that invoices are paid on time, regardless of whether the amount is due and correct.) Unlike in the private sector, because of accountability issues, benchmarking in public-sector organisations should be seen to be open and transparent.
Benchmarking by comparator
Internal benchmarking involves the comparison of units or sub-units of the same entity. It either aims to improve the organisation (by learning from others), or it can be disciplinary, as a way of imposing hierarchical control by central headquarters.
External benchmarking compares organisations with the same or similar characteristics. It can be competitive, with the aim of ‘getting the edge’ over rivals, or it can be non-competitive, when it is essentially about ‘learning from others’. It can take the form of [link title="high-level%20comparisons" link="%2Faware%2FPA%2FPages%2FExamination%2FHigh-level-comparisons.aspx" /]
on a number of aspects such as organisational policy, organisational structures, and how each organisation tackles a specific problem.
Benchmarking by approach
Performance benchmarking
Process benchmarking
High level comparisons
Public policy benchmarking and policy system benchmarking (e.g. benchmarking of welfare-state regimes or school or university education systems) are usually carried out by evaluators and researchers. Such benchmarking works only if comparators have similar regimes. They are most commonly used for making international comparisons (for example, ranking of education policies and systems by the OECD/PISA Programme for International Student Assessment, or the Human Development Index by the UNDP). Such international benchmarking raises a number of issues related to cooperation when benchmarking at the macro level, such as the choice of criteria and indicators, and the fact that best practice is not transferable due to different national structures, institutional environments and cultural preferences.
Instructions
Benchmarking approaches in audit
The main benchmarking approaches, which can be used in an audit, are [link title="Performance%20benchmarking" link="%2Faware%2FPA%2FPages%2FExamination%2FPerformance-benchmarking.aspx" /]
, [link title="Process%20benchmarking" link="%2Faware%2FPA%2FPages%2FExamination%2FProcess-benchmarking.aspx" /]
and [link title="High-level%20comparisons" link="%2Faware%2FPA%2FPages%2FExamination%2FHigh-level-comparisons.aspx" /]
. Many of the ECA’s audits include elements of both process and performance benchmarking. In practice, it may be quite difficult to rank performance, especially when comparing entities in different geographical locations, with the result that audit reports focus on best practice that may be generalised. Process benchmarking can also be useful for identifying good practices.
Public policy benchmarking and policy system benchmarking fall outside the remit of the ECA’s audit work, because the policy outcomes (like economic growth, unemployment, etc.) are the result of numerous activities by various actors and thus are not within the control of the organisations audited.
Before deciding whether to use benchmarking in an audit, audit teams should reflect on its potential added value, existence of equivalent comparator, data availability, costs and auditee’s buy-in.
Added value
How can benchmarking add value to the audit? Is it to identify good practice or to rank performance?
Consider how the benchmarking results will be presented in the report: in the main report or in an appendix? Will the comparison show ranking, or good practice? How will comparisons be put in context? How can best practice be illustrated effectively?
Equivalent comparator
Do relevant and reliable comparators exist? How realistic are the comparisons? Will the auditee, and especially external comparators, be content for any internal information to be published?
In the ECA audit context, most comparisons are likely to be within different parts of the same organisation or within the EU Institutions or agencies, or between member state bodies.
Data availability
Availability. Does the auditee have its own performance measurement? If so, how can the results be used for the audit? Is data and information available to allow benchmarking? If not, is it possible to obtain relevant, reliable and timely data in a systematic way?
Reliability and relevance. Are performance measurement indicators accurate and reliable? Are the indicators appropriate to measure the performance of the service or activity? Is like being compared with like? For performance measures, ensure that source data will be acceptable to the audited entity, and the reliability or relevance of the information collected will not be challenged.
Cost considerations
Costs. What are the costs of benchmarking? Costs will vary depending on the complexity of the comparison, the availability, volume and reliability of data, the sensitivity of the audited body and the use of external experts. Is the added value of the benchmarking justified by the cost of carrying out the exercise?
Experts and other in-house support. Is there a need for [link title="external%20specialist%20advice" link="%2Faware%2FGAP%2FPages%2FSpecific%2FExternal-experts.aspx" /]
or is an in-house expert or experienced auditor available to advise?
Communication with auditee
Consult and keep the auditee informed about the proposal to carry out the benchmarking, the justification for it, the potential added value and how the results will be used in the report. Their buy-in is important to minimise the risk of non-acceptance of findings, and for the commitment necessary to address audit recommendations once the audit is complete. Will the auditee accept the results of the benchmarking exercise?
Resources
[icons-list icon-size="2" separator="line" icon-vertical-alignment="middle" vertical-alignment="middle"]
[icon-list-item title="Checklist%20for%20main%20steps%20in%20benchmarking" description="to%20be%20followed%20by%20the%20audit%20team%20from%20an%20early%20stage%20in%20the%20planning%2C%20through%20the%20audit%20until%20the%20reporting%20phase." link="%2Faware%2FDocuments%2FChecklist-benchmarking.docx" icon="file-word-o" /]
[icon-list-item title="Benchmarking%20case%20studies%20from%20the%20ECA's%20reports" description="The%20case%20studies%20are%20used%20for%20illustrative%20purposes%2C%20to%20give%20more%20information%20on%20how%20benchmarking%20can%20be%20used%20in%20practice%2C%20to%20highlight%20good%20practices%2C%20elements%20that%20went%20well%20and%20not%20so%20well%2C%20and%20how%20the%20results%20of%20the%20exercise%20were%20reported." link="%2Faware%2FDocuments%2FBenchmarking-Case-studies.docx" icon="file-word-o" /]
[/icons-list]
[/toc-this]
[/column]
[/row]
[/section-item]
[section-item 3]
[row]
[column 12]
[toc fixed="true" selectors="h2%2Ch3" class="basic-toc" /]
[/column]
[/row]
[/section-item]
[/section]
