Reasoning and structure of special reports

We determine the structure of a special report according to the most effective, compelling way of conveying the messages to the reader. The report should be structured around the audit questions actually answered by the audit.

Performance audit

Reporting

Ref: 38.100

[section separator="true"]

[section-item 9]

[row]

[column 12]

[toc-this]

Principles

Special reports should follow a standard layout, in five main sections (Executive summary, Introduction, Audit scope and approach, Observations, and Conclusions and

[link title="recommendations" link="%2Faware%2FGAP%2FPages%2FRecommendations.aspx" /]

), but within this be structured to help the reader follow and understand the arguments and findings being presented and provide a logical thread between the audit purpose, observations and conclusions. The logical progression of argument should be clearly signposted by means of the appropriate use of

[link title="headings%20and%20sub-headings" link="%2Faware%2FGAP%2FPages%2FHeadings.aspx%23Heading-types" /]

. The audit results should conclude against the stated audit questions; the observations should be presented persuasively and supported by sufficient information and explanations for the reader to understand the extent and significance of these observations; and the conclusions and recommendations should follow logically from the facts and arguments presented. The information given should convince the reader of the validity of the findings, the reasonableness of the conclusions, and the benefit of implementing the recommendations.

Instructions

Conclusions argumentation

The overall conclusion of the audit should be supported by a pyramid of conclusions, based on a logical progression of the argument. A sound structure makes the arguments more powerful, and helps the reader understand them and be convinced of the content. Readers should never find themselves questioning why they are reading a particular sentence, paragraph or section, or how it fits in with the overall argument. The raw material for the pyramid of conclusions is evaluative statements, derived from the audit findings. Evaluative statements are complete sentences which give considered opinion about something. (The types of statements mirror the

[link title="types%20of%20headings" link="%2Faware%2FGAP%2FPages%2FHeadings.aspx%23Heading-types" /]

[toggles]

[toggle title="Types%20of%20statements"]

“The Commission’s structure" is a label; it tells you what is inside (like the label on a pot of jam), but no more.
“The Commission’s structure has 82 hierarchical levels" is a statement, but is not evaluative; it does not say whether having 82 levels is a good thing or a bad thing. (We might suspect, but the assertion does not spell it out for the reader.)
“The Commission’s structure has too many levels in the hierarchy" is an evaluative statement; it includes a judgement about the situation (“too many"), but, perhaps, the statement in isolation is not that helpful to the reader.
“The Commission’s structure is unnecessarily complex and hinders effective decision-making" provides further information still, and explains what the consequence is of the flaws in Commission structure. It is the most helpful to the reader and is therefore perhaps the best. It is also, however, the longest; there is frequently a tension between clarity and length.

[/toggle]

[/toggles]

The evaluative statements should be organised in a logically convincing structure. Traditional reporting techniques often use a “funnel principle", starting with a wide range of points and narrowing down to a key message. The structure will make the argumentation more powerful by turning this on its head, starting with the overall message and opening out to develop the supporting messages. It is frequently the case that not all findings are needed for the conclusions and not all areas covered by the audit are material enough to be reported. Starting with the overall message will help to be ruthless in selecting issues for inclusion, and avoid the natural temptation to include as much of the findings as possible. Including findings that sit outside the main arguments – and therefore structure – of the report will dilute its impact. This is also how journalists – who are in the business of maximising impact – operate; the overall heading gives the key message of an article, expanded on in the first paragraph, and gradually including more detail as the article goes on. We are not journalists, of course, but we can learn from how journalists present their material. The advantage of a sound structure is that the high level statements can subsequently be used as

[link title="headings" link="%2Faware%2FGAP%2FPages%2FHeadings.aspx" /]

within the report. They also provide a very convenient starting point for drafting the executive summary or press release later on.

Qualities of good pyramid of conclusions

Good pyramid of conclusions requires that

The overall conclusion is clearly stated and evaluative. All sub-conclusions are evaluative. The purpose is to convert factual evidence into judgements - typically about how well the Commission or other auditee did something.
Any higher level conclusion inevitably follows from the sub-conclusions. In other words each group of sub-conclusions obeys the
[multi-link title="mutually%20exclusive%20and%20collectively%20exhaustive" title_1="AWARE" link_1="%2Faware%2FPA%2FPages%2FPlanning%2FBreaking-down-questions.aspx%23Mutually-exclusive-and-collectively-exhaustive" title_2="Wikipedia" link_2="https%3A%2F%2Fen.wikipedia.org%2Fwiki%2FMECE_principle" /]
principle.
The lowest level of the structure is firmly rooted in audit findings. There are no strict rules how deep the structure should go. As soon as you get down to a level where the statements are relatively uncontroversial statements of fact that do not need any further supporting evidence you can stop. Three levels represent a minimum; in practice it is generally helpful to go down at least one more level to provide evidence. It is perfectly acceptable to develop more levels in the hierarchy in some places than others, depending on the nature of the subject matter.
It answers an audit question, be it the original one stated in the task plan or a modified one.

The nature of our audits means that the subject matter is often messy and rarely fits naturally into the neat, precise pyramids, obeying all the rules that we are looking for. For example, we may find that there is an unexpected overlap between a conclusion down one “arm" of the pyramid at level 3, and a separate arm at level 2. Strictly speaking, this should not be allowable, but can be sometimes acceptable. As auditors, we aim to make our audit work and conclusions as objective as possible. When developing a sound structure, there is need for subjective judgement. There is no right or wrong. Several structures might be possible and comparably sound. The pyramid of conclusions can evolve as the report is drafted. The structure chosen should be changed if necessary, rather than broken. Including findings that sit outside the structure will dilute its impact. The corollary is that if a particular point, sitting outside the structure, is so significant that it must go in the report, then the structure and/or the key messages should be changed.

Tools to develop a sound structure

The audit team should initiate the drawing conclusions process thereby allowing those not directly involved in the audit to review the audit findings with the audit team and draw the appropriate conclusions. The structure can make the argumentation more understandable, if it replicates the thinking that takes place in a natural conversation. This is the idea behind the diner party concept.

[toggles]

[toggle title="The%20dinner%20party%20concept"] The idea is that you are talking – at a dinner party, or it could be in the local pub – to someone you do not know very well. They ask about your work, and you tell them you have just finished an audit of XYZ. “How interesting", they say, “what did you find?" At this point, if you are to hold their attention, you do not say: “well, the relevant EU regulation was ABC, and the methodology we adopted was to carry out six on-the-spot audit visits to member states, supplemented by a questionnaire to project promoters" and so on. You do not have time to say that much, so you must say the most important thing about the audit: the overall conclusion – the level 1 conclusion (“The ECA is a great place to work"). If, then, your companion still appears to be interested, he/she may ask, “why did you conclude that?" In response, you can then give your supporting, level 2 conclusions (this was because of the reward package, colleagues, climate etc.) Again, in the (slightly unlikely) event that you are questioned about one of these level 2 assertions, you will have the supporting justification to hand. If the logic underpinning your pyramid is sound, your new friend will have no difficulty in agreeing with your argument. [/toggle]

[/toggles]

The structure can be made more natural also by using the

[link title="Situation-Complication-Question" link="%2Faware%2FDocuments%2FSituation-complication-question-example.docx" icon="file-word-o" /]

-Answer approach, where the answer is provided by the pyramid of conclusions. Looking for patterns – and so potential gaps – in the evaluative statements can help with developing a sound structure of conclusions.

Paragraph structure

We support each conclusion (typically in the form of an evaluative heading in the observation section) with a set of paragraphs, which are sequentially numbered. A good rule is: only one issue in each paragraph. In general, the first sentence or two in the paragraph should set the scene for the rest of it, by explaining what the paragraph is about and summarising the main point made. The remainder of the paragraph might then expand on this initial point – for example, by justifying it, or adding further details.

Integration of audit questions

The report should be structured around the audit questions (or sub-questions, if one main audit question was identified).The questions set out the purpose of the audit, and provide the focus for both the structure of the observations section and the questions to be answered by the conclusions. The report should set out the questions actually answered, i.e. those on which conclusions were reached, rather than those originally approved in the task plan but that were not answered. While the audit questions set out in planning might provide a good structure for the final report, they need not do so. Audit questions are formulated at the start of the audit on necessarily limited information. When the audit has been carried out, audit teams have much fuller knowledge of the subject matter. What seemed like an important question at the outset, once investigated, may not result in compelling findings. Conversely, when reviewing the evidence collected, auditors may take the view that a particular issue warrants a higher profile in the final report. And while audit questions will be very familiar to the audit team and even to the Commission, they are unknown to the wider readership of our reports. If the report adopts a different structure from the one the auditee is familiar with, in the interests of

[link title="no%20surprises%20approach" link="%2Faware%2FGAP%2FPages%2FCommunication-in-audit.aspx" /]

, the auditee should be informed in advance. One option – which has been used successfully by some audit teams – is to discuss the outline report with the auditee; any feedback received can then be taken into account, as appropriate, when drafting the full report. Where questions need to be set out in the text and where the answer should be given depend to some extent on the nature of the material being reported, and there is therefore scope for discretion. One possible way of handling this issue is as follows:

put the relevant audit question in each section of the observations, and make sure that you also provide an answer to that question in that section (rather than not answering it at all, or not answering it until the conclusions and recommendations section); and
answer the overall audit question in the Conclusions and
[link title="recommendations" link="%2Faware%2FGAP%2FPages%2FRecommendations.aspx" /]
section (as well as the Executive Summary). The Conclusions and recommendations section also provides an opportunity to repeat, as necessary, the answers to the level 2 questions.

Resources

[icons-list icon-size="2" separator="line"]

[icon-list-item title="Example%20of%20developed%20pyramid%20of%20conclusions%20and%20outline%20report" icon="file-word-o" link="%2Faware%2FDocuments%2FConclusion-hierarchy-example.docx" /]

[/icons-list]

[/toc-this]

[/column]

[/row]

[/section-item]

[section-item 3]

[row]

[column 12]

[panel panel-style="boxed" title="Related%20documents" icon="book" class="ref-panel"]

[standards]

[link new-window title="ISSAI%20300%2F39" link="https%3a%2f%2fwww.issai.org%2fwp-content%2fuploads%2f2019%2f08%2fISSAI-300.pdf%23page%3D29" /]

[link new-window title="ISSAI%203000%2F118" link="https%3a%2f%2fwww.issai.org%2fwp-content%2fuploads%2f2019%2f08%2fISSAI-3000-Performance-Audit-Standard.pdf%23page%3D28" /]

[link new-window title="GUID%203920%2F115%2C121" link="https%3a%2f%2fwww.issai.org%2fwp-content%2fuploads%2f2019%2f08%2fGUID-3920-The-Performance-Auditing-Process.pdf%23page%3D49" /]

[/standards]

[/panel]

[/column]

[/row]

[row]

[column 12][/column]

[/row]

[row]

[column 12]

[toc fixed="true" selectors="h2%2Ch3" class="basic-toc" /]

[/column]

[/row]

[/section-item]

[/section]

Last Modified: 27/04/2023 15:56 Tags:

Reasoning and structure of special reports // <![CDATA[ _spBodyOnLoadFunctionNames.push("setupPageDescriptionCallout"); // ]]>

Reasoning and structure of special reports

Principles

Instructions

Conclusions argumentation

Qualities of good pyramid of conclusions

Tools to develop a sound structure

Paragraph structure

Integration of audit questions

Resources

Reasoning and structure of special reports