Reasoning and structure of special reports

Reasoning and structure of special reports

Reasoning and structure of special reports

We determine the structure of a special report according to the most effective, compelling way of conveying the messages to the reader. The report should be structured around the audit questions actually answered by the audit.
Performance audit
Reporting
Ref: 38.100

Principles

Special reports should follow a standard layout, in five main sections (Executive summary, Introduction, Audit scope and approach, Observations, and Conclusions and recommendations), but within this be structured to help the reader follow and understand the arguments and findings being presented and provide a logical thread between the audit purpose, observations and conclusions. The logical progression of argument should be clearly signposted by means of the appropriate use of headings and sub-headings. The audit results should conclude against the stated audit questions; the observations should be presented persuasively and supported by sufficient information and explanations for the reader to understand the extent and significance of these observations; and the conclusions and recommendations should follow logically from the facts and arguments presented. The information given should convince the reader of the validity of the findings, the reasonableness of the conclusions, and the benefit of implementing the recommendations.

Instructions

Conclusions argumentation

The overall conclusion of the audit should be supported by a pyramid of conclusions, based on a logical progression of the argument. A sound structure makes the arguments more powerful, and helps the reader understand them and be convinced of the content. Readers should never find themselves questioning why they are reading a particular sentence, paragraph or section, or how it fits in with the overall argument. The raw material for the pyramid of conclusions is evaluative statements, derived from the audit findings. Evaluative statements are complete sentences which give considered opinion about something. (The types of statements mirror the types of headings).
Types of statements
The evaluative statements should be organised in a logically convincing structure. Traditional reporting techniques often use a “funnel principle", starting with a wide range of points and narrowing down to a key message. The structure will make the argumentation more powerful by turning this on its head, starting with the overall message and opening out to develop the supporting messages. It is frequently the case that not all findings are needed for the conclusions and not all areas covered by the audit are material enough to be reported. Starting with the overall message will help to be ruthless in selecting issues for inclusion, and avoid the natural temptation to include as much of the findings as possible. Including findings that sit outside the main arguments – and therefore structure – of the report will dilute its impact. This is also how journalists – who are in the business of maximising impact – operate; the overall heading gives the key message of an article, expanded on in the first paragraph, and gradually including more detail as the article goes on. We are not journalists, of course, but we can learn from how journalists present their material. The advantage of a sound structure is that the high level statements can subsequently be used as headings within the report. They also provide a very convenient starting point for drafting the executive summary or press release later on.

Qualities of good pyramid of conclusions

Good pyramid of conclusions requires that
  • The overall conclusion is clearly stated and evaluative. All sub-conclusions are evaluative. The purpose is to convert factual evidence into judgements - typically about how well the Commission or other auditee did something.
  • Any higher level conclusion inevitably follows from the sub-conclusions. In other words each group of sub-conclusions obeys the principle.
  • The lowest level of the structure is firmly rooted in audit findings. There are no strict rules how deep the structure should go. As soon as you get down to a level where the statements are relatively uncontroversial statements of fact that do not need any further supporting evidence you can stop. Three levels represent a minimum; in practice it is generally helpful to go down at least one more level to provide evidence. It is perfectly acceptable to develop more levels in the hierarchy in some places than others, depending on the nature of the subject matter.
  • It answers an audit question, be it the original one stated in the task plan or a modified one.
The nature of our audits means that the subject matter is often messy and rarely fits naturally into the neat, precise pyramids, obeying all the rules that we are looking for. For example, we may find that there is an unexpected overlap between a conclusion down one “arm" of the pyramid at level 3, and a separate arm at level 2. Strictly speaking, this should not be allowable, but can be sometimes acceptable. As auditors, we aim to make our audit work and conclusions as objective as possible. When developing a sound structure, there is need for subjective judgement. There is no right or wrong. Several structures might be possible and comparably sound. The pyramid of conclusions can evolve as the report is drafted. The structure chosen should be changed if necessary, rather than broken. Including findings that sit outside the structure will dilute its impact. The corollary is that if a particular point, sitting outside the structure, is so significant that it must go in the report, then the structure and/or the key messages should be changed.

Tools to develop a sound structure

The audit team should initiate the drawing conclusions process thereby allowing those not directly involved in the audit to review the audit findings with the audit team and draw the appropriate conclusions. The structure can make the argumentation more understandable, if it replicates the thinking that takes place in a natural conversation. This is the idea behind the diner party concept.
The dinner party concept
The structure can be made more natural also by using the Situation-Complication-Question-Answer approach, where the answer is provided by the pyramid of conclusions. Looking for patterns – and so potential gaps – in the evaluative statements can help with developing a sound structure of conclusions.

Paragraph structure

We support each conclusion (typically in the form of an evaluative heading in the observation section) with a set of paragraphs, which are sequentially numbered. A good rule is: only one issue in each paragraph. In general, the first sentence or two in the paragraph should set the scene for the rest of it, by explaining what the paragraph is about and summarising the main point made. The remainder of the paragraph might then expand on this initial point – for example, by justifying it, or adding further details.

Integration of audit questions

The report should be structured around the audit questions (or sub-questions, if one main audit question was identified).The questions set out the purpose of the audit, and provide the focus for both the structure of the observations section and the questions to be answered by the conclusions. The report should set out the questions actually answered, i.e. those on which conclusions were reached, rather than those originally approved in the task plan but that were not answered. While the audit questions set out in planning might provide a good structure for the final report, they need not do so. Audit questions are formulated at the start of the audit on necessarily limited information. When the audit has been carried out, audit teams have much fuller knowledge of the subject matter. What seemed like an important question at the outset, once investigated, may not result in compelling findings. Conversely, when reviewing the evidence collected, auditors may take the view that a particular issue warrants a higher profile in the final report. And while audit questions will be very familiar to the audit team and even to the Commission, they are unknown to the wider readership of our reports. If the report adopts a different structure from the one the auditee is familiar with, in the interests of no surprises approach, the auditee should be informed in advance. One option – which has been used successfully by some audit teams – is to discuss the outline report with the auditee; any feedback received can then be taken into account, as appropriate, when drafting the full report. Where questions need to be set out in the text and where the answer should be given depend to some extent on the nature of the material being reported, and there is therefore scope for discretion. One possible way of handling this issue is as follows:
  • put the relevant audit question in each section of the observations, and make sure that you also provide an answer to that question in that section (rather than not answering it at all, or not answering it until the conclusions and recommendations section); and
  • answer the overall audit question in the Conclusions and recommendations section (as well as the Executive Summary). The Conclusions and recommendations section also provides an opportunity to repeat, as necessary, the answers to the level 2 questions.

Resources

 
Last Modified: 27/04/2023 15:56   Tags: