[toc-this]
Principles
The interview is a data gathering technique that needs to be corroborated to confirm that facts are correctly understood. It can be supported by evidence from other data collection methods (e.g. auditee's documents and data, observation, inspection, [link title="external%20confirmations" link="%2Faware%2FFA%2FPages%2FExamination%2FExternal-confirmations.aspx" /]
). Moreover, it can be difficult to draw general conclusions from the evidence.
Definition
An interview can either be face to face or remote, usually by telephone. Likewise, it can either be directed to an individual or a group. Through the interview data are gathered on various complex issues in an efficient way. Furthermore, it takes into account situational and individual factors and provides information that is not available from other sources.
Purpose of the interview
The interview has three main purposes: orientation, examination and confirmation. It can have one or two purposes, but normally not all three at the same time.
- Orientation is normally part of the audit team's learning process during the planning phase. It aims at exploring and giving an overview of a specific area or function, e.g., by asking for presentations of activities, explanations of formal or informal networks or interpretation of documents (reports, instructions or, budgets). The objective could be to identify possible audit subjects or to find out about other available sources of information, such as key persons or documentation.
- Examination aims at more specific issues with a view to establishing new information, often to be used as audit evidence. In some cases, such information has not been previously recorded at all but is embodied in the interviewee through personal experiences, particular references, opinions, etc. In other cases, the knowledge can be retrieved for example by (joint) interpretation of internal documents, reports or records.
- Confirmation, finally, often goes together with either orientation or examination, but deserves to be mentioned as a separate purpose because of its fundamental importance. Confirmation, by definition, is typically based on information that has already been gathered. However, in this context the information can also be gathered and confirmed simultaneously. Not least in the planning phase, it is important to have basic conditions and facts explicitly confirmed by stakeholders. However, in the examination phase there might also be a need to confirm facts and findings. If data is incorrectly understood, the quality of the whole audit may suffer and a lot of work may be in vain.
Types of interviews
Unstructured interview
An unstructured interview usually highlights a small number of themes and examples of questions for the interviewer to follow when needed. The interviewer is allowed maximum flexibility to explore themes that have not previously been considered and to follow up the responses that are given. Such an approach is well suited to obtaining rich data, often related to complex or sensitive subjects, and as a means to building theory. However, this type of interview can be time intensive and it is difficult to quantify data.
Structured interview
The structured interview can be defined as an interview in which a standard set of pre-set questions is established, which can be asked in the same order from one interview to another. The wording of questions and the order in which they are asked are consistent throughout, and there is very limited flexibility in the approach for exploring issues that have not been included in advance. These limited options can bias the responses and it is difficult to reach deep understanding. This approach is highly suited to capturing simple, factual data, covering a range of issues, pinpointing or confirming certain points and for hypothesis testing. The results are good for purposes of comparability and are receptive to coding and subsequent analysis. The evidence through this type is more easily quantified and generates breadth of data.
A mixture of the two interview type
In practice, most interviews contain a mixture of the two interview types. For example, the interviewer can allow for the interviewee to steer the dialogue, following the flow of the conversation, and use the questions/themes as a checklist to make sure all topics are covered. This is generally called a semi-structured interview.
Instructions
Initial preparations
Objective of the interview
Generally, the first step of the preparations is to define the objectives of the interview. What is the purpose of the interview, and what are the priority areas of information needed:
In the planning phase, initial preparations would normally focus on two things:
- gather facts, which are normally of the type which cannot be contested, e.g., which regulations are applicable, organigrammes, changes which are foreseen. The interviewer needs to strive to develop a rather exhaustive list. These listed facts will need to be confirmed, or rejected, by the interviewee.
- search for different options and gather all relevant information on opinions about the subject. This kind of information is normally subjective. Here it is not really possible to establish an exhaustive list. It is more important that the interviewer tries to find opinions from different sources and preferably with different views - e.g. pros and cons. This should form the basis for getting an orientation on the subject. The objective is often quite open and the interview used to inform the development of an audit programme.
In the examination phase, the interview will normally be narrower and the information needed is often indicated by the detailed audit questions. It is therefore necessary to consult these in order to acquire a thorough understanding of what information will be needed.
It is equally important for the interviewer to reflect on the different roles he/she would take on, while working for the different [link title="purposes" link="%23Purpose-of-the-interview" /]
. In pursuing the orientation purpose, and perhaps also for confirmation, the interviewer would need more to approach the task with the mind of a pure researcher, whereas in carrying out the examination task the role is that of the traditional auditor.
Information needs
No matter at what stage the interview takes place, the auditor should examine what information is available on the area. Advice could, e.g., be sought from colleagues or outside experts. Being updated will make the interview more efficient and effective.
It is often useful to consider how the information is to be used. The structure of the interview will determine what type of analysis can be done afterwards. It is considered good practice to prepare for most of the data analysis at this early stage. This includes considering e.g. how to group and codify the data and which methods to use for analysis, what questions the analysis should answer.
Thorough preparation will help later in drafting the interview guide, particularly in deciding which questions are necessary and which are superfluous for the later analysis and presentation of the data.
Interviewee(s)
Normally, before drafting the detailed questions, it should be decided who to interview. In some cases it will be evident, for example Commission officials responsible for specific programmes. In other cases, it might be relevant to talk to several people, and a selection must be done. The selection should be made on the basis of the knowledge acquired of the area, and involve some criteria to make sure relevant areas/programmes, hierarchical levels are sufficiently covered. The number of interviews depends on the subject of the audit, and the resources available.
Having selected the interviewee, it is important to find out information, if possible, about the person, but certainly about the organisation or programme he/she works with. This will facilitate the communication in the interview.
It is important to consider what might be the particular interests, or roles, of the interviewees in the interview situation. In different situations, the same interviewee can be regarded as an expert, stakeholder (with certain interests), or just an auditee. In fact, an interviewee can play different roles even within the same interview. For example, when interviewing an official in the Commission in the planning phase, he/she should be regarded as an expert when it comes to confirmation. In the case of orientation he/she can be regarded as either an expert or a stakeholder depending on whether answers are factual or opinions. In all other cases, he/she would normally be regarded as an auditee. This is important to have in mind when planning, conducting, and analysing the results of an interview.
Formulating questions
Open or closed questions
Before starting working with the detailed questions it is recommended to consider different styles of asking questions.
The objective of the open questions is to let the interviewee speak freely to develop their point of view. These questions allow the interviewees to express themselves in their own words, and are suitable for obtaining qualitative information. Key words/phrases are: why?, how?, what do you think of…? Open questions are suitable for exploring topics and collecting information in unstructured interviews, but require more work when it comes to coding and analysing the responses.
The objective of closed questions is to obtain precise, limited or quantifiable answers that can easily be categorised. The answers to these questions will often be more restricted, like a simple 'yes' or 'no', or they can take the form of short facts or a choice of defined alternatives. Key words/phrases are e.g.: when, where, how many and what.
[toggles]
[toggle title="Examples%20of%20open%20and%20closed%20questions"]
- Open: how does the system of internal control function?
- Closed: does the system of internal control of the project function well?
[/toggle]
[/toggles]
Different styles of questions
Different styles of questions provoke different types of answers, and it is important to be aware of the character of answer required before starting to develop the questions. Some audits are concerned with establishing the facts and chronology of events, and consequently knowledge based questions and time-frame questions are very useful.
In contrast, quality of service examinations focus on the opinions of interviewees in relation to the area studied. In this case, opinion, feeling or sensory based questions should be posed.
[toggles]
[toggle title="Examples%20of%20questions%20per%20question%20style"]
- Knowledge-based question: When was the grant awarded?
- Time-frame-based question: How were these grants awarded last year?
- Opinion-based question: Do you think the grant scheme is fair?
- Feeling-based question: Do you feel comfortable about the administrative arrangements for the grant scheme?
- Sensory-based question: What do you see when you visit the sites where grant money is used?
- Experience-based question: How long have you been awarding grants?; What is your background?
- Behaviour-based question: Do the grant recipients treat the attached conditions seriously?
[/toggle]
[/toggles]
Developing an interview guide
Questions should be organised into an interview guide, which is a sort of questionnaire, where all the questions and all other important information for the interviews should be listed. It should usually be sorted into sections, whose order is set out so as to focus the interviewees’ attention. The format of the guide depends on the type of the interviews - a more structured interview demands the drawing up of a detailed interview guide, while a more unstructured interview entails a sort of checklist of essential topics/questions to cover in conversations with key informants.
Ordering of questions
As a piece of general advice, it is useful to divide the interview guide into different topics or areas. It can be useful to start out with some general and more open questions and then move on to specific topics addressed by more specific questions. Any sensitive issues, such as potentially difficult personnel issues, should be placed towards the end of the interview, when the interviewee is a bit ‘warmed up’ on the subject, and feels more comfortable in the interview situation.
In the planning phase of the audit
For an interview in the planning phase, the guide should normally contain elements for both [link title="confirmation%20and%20orientation" link="%23Purpose-of-the-interview" /]
. The decision as to whether it should be divided into two such sections, or divided in another way, based on different subjects, is up to the interviewer.
The confirmation part should be very structured. For example, for the policies – applicable regulations and stability of the situation should be confirmed; for the organisation – responsibilities, reporting hierarchy and stability; for procedures – the requirements, etc. In many cases, several or all of these examples are vital to include.
For the orientation part, questions should allow for flexibility but not be too unfocused. Therefore, it is recommended to identify a number of themes to explore. Preferably, certain preliminary hypotheses could be developed and included in the list of questions. This technique requires the auditor to have a wide general knowledge of the area in order to be able to quickly assimilate information received during the interview, and thereby open up new lines of thinking which can be directly pursued during the session.
It is recommended to include in the guide, perhaps as an annex, any material that might be helpful, such as organigrammes, details about different sources from which information has been gathered. The development of the discussion is hard to foresee and it is easier to have a fruitful discussion if one can be very precise in references to, e.g., regulations, documents, reports.
In the examination phase
For an interview in the examination phase, the starting point for developing interview questions is most often the audit question in the [link title="evidence%20collection%20plan" link="%2Faware%2FPA%2FPages%2FPlanning%2FTask-plan.aspx%23Principles" /]
and/or the [link title="audit%20programme" link="%2Faware%2FGAP%2FPages%2FCA-FA%2FPlanning%2Ftask-plan-audit-programme.aspx%23Audit-Programme" /]
and then develop a hierarchy of sub-questions that allow each overall question to be answered. The next step is to consider what kind of information is needed to answer these. The questions must be translated into measurable elements, and the different data and information needed to answer these should be listed.
In the process of developing the interview guide, the questions should be tested on colleagues, experts or other advisors and it is advisable to run a pre-test of the competed interview guide on a limited number of people. These could be potential interviewees that are not included in the sample to be interviewed. A pre-test like this will provide a critical view of the form and content of the interview guide, and makes it possible to assess its relevance. In addition, it gives useful practice in running the interviews, and can also give a preliminary indication of the likely answers and reactions to the questions. Furthermore, it gives an opportunity to test methods of analysis.
When the pre-test is completed and relevant conclusions are drawn, the final version of the interview guide can be drawn up and the methods of analysis can, if necessary, be adjusted.
Organising the interview
The entire interview situation needs to be planned and practical arrangements need to be taken care of.
Organisation of the interview team
The size of the team can vary, but teams of two persons are frequently used. Single interviewing should be avoided as well as a team of too many people, which can be intimidating for the interviewee.
Ensure common approach
If there is more than one team of interviewers covering a series of interviews in the audit, it is essential for the different teams to work together in the initial phase. It can be useful for the teams to go through the interview guide together, rehearsing the question session and the different tasks to perform, in order to secure a mutual platform and a common approach.
Division of tasks
It is also necessary to discuss the interaction between the interviewers and the division of tasks. As a rule, it can be useful that one of the team gives attention to the interviewee, asking the questions, while another person focus on taking notes, keeping an eye on the interview guide to make sure all major topics are covered, and where necessary contribute with follow-up questions.
Make a time plan
How much time to plan for the interviews will vary depending on the type of interviews and the particular situation. The steps in the interview preparations are not necessarily sequential; many of them can be overlapping. Although the activities are not additive, all of the steps are time-consuming and it is important to consider all the individual stages when making an estimation of the time needed.
The planning must, e.g. allow time for potential re-scheduling of appointments, and for writing out the minutes from each of the interviews. This aspect is particularly important when several interviews are planned for the same day. If possible, it should be ensured that there is sufficient time between interviews for the audit team to sit down undisturbed and reflect on the outcome of each interview. If the interviews are carried out in the same building it is advisable to explain to the organisation visited that a separate meeting room is needed for the audit team.
Communication with the auditee
It is important to notify the interviewees in good time, and give a brief overview of the topics that will be covered in the interview and give an estimation of time needed for the interview. It can often be constructive to send the questions in advance, so they have time to prepare for the interview. This can be particularly useful if there is a need to gather information from documents or other sources so as to be able to answer the questions, and also if the questions concern events in the past that could be difficult to recall instantly.
It can be useful to keep in mind that, in certain instances, especially when facing a challenging interview, sending the complete interview guide to the interviewee in advance can pose the risk that the interviewee may use this to take the lead at the interview session.
Interview situation
For the interview to be successful, it is important to make an effort to establish and maintain a good climate for [link title="communication" link="%2Faware%2FGAP%2FPages%2FCommunication-in-audit.aspx" /]
. Cooperation and a positive attitude from the auditee significantly increase the prospect of getting the required information in the interview.
Opening the interview session
Always open the interview session by thanking the interviewee for the reception and introduce each team member and their responsibilities. Take the opportunity at this stage to exchange business cards with the interviewee. Knowing and using names correctly will facilitate communication. An attendance list should be drawn containing the information about processing of personal data. Use privacy statements in EU languages. The interviewer should also note down who participated in the interview (interviewers, interviewees), location, date and time of the interview. The interviewees contact details could also be useful to note, in case there is a need to get back to the interviewee to clarify specific points from the interview.
Set the background
Make sure that the auditee has a clear understanding of what the ECA is and the role it plays within the European Union. Take the time to present the audit, including the objectives, scope and tasks. Also explain the purpose of the interview and how it is planned to be conducted. This is also important since it will clarify the course of the meeting, who is leading the meeting and what information is expected from the interview. Having an understanding of the purpose of the interview and its context might help the interviewee to give better and more relevant answers.
Also guarantee confidential treatment of the answers where necessary or possible, or else explain how the data will be used.
Steering the interview
There will often be considerable variation between interviewees when it comes to how much they talk, whether they keep to the topic, or easily get distracted or diverted. It is the interviewer’s task to maintain control and steer the conversation to get the required information. Normally one of the team members should be the interview leader. During the interview there can be a change of leader with one of the other team members present taking on the role. Preferably such changes should be foreseen in the initial presentation and clearly announced when it takes place.
Take time to carefully note down key words and important facts, and avoid being stressed. A lone interviewer should not be afraid of taking pauses in the questioning to be able to note down the answers. Make sure the interviewee understands the importance of this. Do not try to record everything being said in the interview, but make a synthesis of opinions and conclusions. It is also wise to note down, on a separate list, documents or other evidence that the interviewee, during the session, agrees to supply at a later stage.
Summarise and confirm
Refer back to the interview guide throughout the session and lead the interviewee back on track if necessary. A good way to keep control is to summarise the answers from the auditee at the end of each section of the interview and get confirmation that the information has been properly understood before moving on to the next step. This may also be a way to build up "the case" and detect inconsistencies later on in the session.
Clarify answers
Often answers needs to be clarified by follow-up questions. If the answers are vague, the interviewer should ask questions to make the answer more specific; if the answers are too complex, or involve terminology or jargon that are difficult to comprehend, the interviewer should ask the interviewee to clarify. If the answers are off the subject, the interviewer should ask again.
[toggles]
[toggle title="Examples%20of%20useful%20approaches%20to%20follow-up%20questions"]
- Clarify important/vague points: Is this different from that…?
- Pursue inconsistencies: Didn't you say that…?
- Agree understanding: So, what you mean is…
- Contrast with different perspectives: The Delegation thinks this...
- Deconstruct opinions: Why do you say that? Could you clarify?
- Get examples: On what occasion did this happen? Can you give an example of…?
- Exploit complaints: How does the new system cause you problems?
- Make assumptions visible: What is the basis for that...?
- 'Unpack' organisational rhetoric: What do you mean by 'cascade'?
[/toggle]
[/toggles]
Facts or opinion?
In cases where it is unclear, it will also be necessary to clarify if given information is factual or the expression of an opinion. If it is an opinion, it should further be clarified as to whether it is a personal opinion or if it represents an official view of the organisation.
Attention to non-verbal language
It is also important that the interviewer is aware of characteristic non-verbal cues such as change in voice, facial expressions, or gestures, since as much as half of the communication that takes place during the interview is conveyed by these modes of expression. Failure to understand these cues may result in miscommunication.
Handling difficult situations
Some interviews can have a particularly challenging context, e.g. when the aim is to confirm or document negative findings. This may require extra careful consideration from the interviewer’s side. A few important points can be especially useful to keep in mind in those cases:
- Always show a respectful attitude towards the auditee.
- It can be constructive to recognise the auditee’s difficulties, and mention their strong points or previous success.
- Recognise disagreement, but do not get caught in unconstructive discussions, trying to convince the auditee.
- Refrain from using sarcasm or irony.
- Do not impose opinions on the auditees.
- As far as possible, avoid interrupting the auditee.
- Be aware of linguistic difficulties or ambiguities.
- Always make an effort to end the interview on a positive note.
When dealing with audit interviews, suspicion of [link title="fraud" link="%2Faware%2FGAP%2FPages%2FFraud.aspx" /]
or other illegal matters may emerge. In such case, auditors should follow the [link title="established%20procedure" link="%2Faware%2FGAP%2FPages%2FFraud.aspx%23Suspected-fraud" /]
.
Closing the interview session
Therefore, before closing the interview, it is recommended to check that all questions have been answered. The interviewer may sometimes find it useful to return to the main questions to get confirmation of the answers given. It is good practice to round off the interview with a short summary of the main conclusions from the interview. The interview team leader shall make sure that any comments or additional questions from the team members are heard. The interview leader may also need support to ensure that all questions are covered.
It is also recommended, if applicable, to agree with the interviewee how and when further contacts should be made for supplying additional information, etc. Then the interviewee should be offered the opportunity to add any comments. After that it is polite to inform the interviewee of future plans for the audit. Finally, it is time to thank them once again for their time and declare the formal interview session closed.
Documentation
In order to be able to make use of the data arising from interviews, it is essential that the answers are properly recorded and [link title="documented" link="%2Faware%2FGAP%2FPages%2FDocumenting-an-audit.aspx" /]
. The final phase consists of interpreting, and analysing the information given by the interviewee and comparing with information obtained from other interviews.
Minutes
The minutes are the tangible evidence obtained from the interview. It is important that sufficient notes are taken and they are typed out in comparable format, adapted to the later data analysis. Notes should be typed out soon after the interview, putting the answers into relevant categories.
To avoid controversies at a later stage, it is advisable to have the interviewee confirm the minutes from the meeting. Having the possibility to correct and approve the minutes afterwards may also make the interviewee feel less restrained during the interview.
Audio or video
Using audio or video recording of the interviews is an option, but this should be approached with great caution and only done with permission from the interviewee. This might make interviewees feel uncomfortable, and will in any case involve a lot of work after the interviews, as notes and a summary of the interviews will still have to be made for data analysis purposes.
Analysis
The main challenge in analysing data from interviews is to interpret what has been said, and sort this out into different categories or topics. Usually, the approach will be some form of qualitative data analysis. One should look for common aspects of information, things that fit together or examples of the same underlying problem, issue or concept in the interviews.
Identify topics of interest
The process starts with reading and re-reading a substantial number of the written down responses in order to interpret the text and identify particular topics of interest. The data should be sorted into themes, developing some scheme to categorise the answers. This process involves a transformation of the data from the particular to the more general, and similarly a movement from the descriptive, summarising what the interviewee says, to the interpretative, involving making an attempt to identify what the information actually means.
In essence, the elaborated data can be embodied in very broad categories, like positive-negative-neutral, or they can resemble a set of alternative responses to the question. Rules must then be developed for assigning responses to the different categories. Even with a set of rules, people may categorise answers differently. Therefore, ideally several people should go through each completed interview and categorise the answers, ensuring comparability.
It is important that the interpretations being made are transparent, so the reader can see very clearly how the interviewer's interpretations of the data relate to the data. It is also important that the interpretations are plausible. There is no totally 'right' interpretation of any qualitative data set, but some interpretations may be more persuasive than others.
Important considerations in interpreting data
When interpreting interview data, it is important to be aware that all responses to interview questions are influenced by preconception and contextual factors that need to be unravelled. What the interviewee expresses will be influenced e.g. by their background, their expectations, their experience of things discussed. A lot of things frequently distort communication between people, and a few facts are important to keep in mind when analysing/using the data:
- Not everything said by the respondent is heard by the interviewer;
- Not everything heard by the interviewer is understood;
- Not everything said by the respondent will be remembered or recorded;
- Not everything the respondent says can be corroborated by an independent source.
Resources
[icons-list icon-size="2" separator="line" icon-vertical-alignment="middle" vertical-alignment="middle"]
[icon-list-item title="Checklist%20for%20audit%20interview" description="to%20be%20used%20when%20preparing%20for%20an%20interview." link="%2Faware%2FDocuments%2FAudit-interview-checklist.docx" icon="file-word-o" /]
[/icons-list]
[/toc-this]
