Can we improve the questions we ask? Performance Managing Scrutiny's questions

On the surface, the question seems ludicrous. Why, or how, would we performance manage questions at a scrutiny or a select committee?  We all know how to ask questions so it would be a waste of time. Who would want to record the questions and answers and review how effectively they produced useful information?

At a deeper level, though the question seems to be so obvious that one wonders why it has never been done.  In sports and others areas of our professional lives, We constantly assess and analyze a team or an individual's performance. In sport, a player's technique are reviewed and assessed to improve performance.  Yet, the one place where it is vitally important for democracy, we seem to ignore it.

We might ask how can we performance manage questions? We may not have the capacity to record and analyze our performance before the next meeting.  In response, I would suggest what has been happening in sport for years, looking at the tapes of other performers to see how they do.

For people working in scrutiny, there is a ready resource for such performance information. The Parliamentary service provides a video and a transcript of all the select committee meetings.  I have blogged extensively on this topic here and here.  Thus, one could take a meeting and work through a few of the questions to see how well they work. The participants could then consider how they would have asked the same question or whether they would have asked something different.  At the same time, they could consider how they would have planned the questions in advance, the lines of enquiry, to get the most out of the occasion. 

 

The Leveson Inquiry provides a master class in good questioning technique.  Robert Jay QC showed us how to ask good questions and why good preparation is key to success. You can find my blog post on his approach here as well as a mild critique here.

How the Treasury Select Committee approach their questions with Bob Diamond, the now former Barclays CEO, shows some lessons regarding questions. First, avoid closed questions.  For example, yes or no questions allow one-word answers.  By asking these, the line of inquiry is closed. Instead, only use these questions to guide the session or to shut down a line of enquiry.

Second, avoid putting words in the other person’s mouth.  For example, do not give them an answer for which they can then agree with your statement or restatement of their position.

Third, listen; always listen, to the answer, before moving on to the next question. Each answer invites a different follow up question. For example, Tyrie focuses on the regulators calling Barclays over the weekend, but not asking whether Mr. Diamond spoke with the Chancellor, Mr. Osborne.

If we use the same analytical skills we reserve for sports on select or scrutiny committee questions, we could go a long way to improving their content and their effect.  A healthy democracy requires that we train ourselves to ask good questions so that we can explore the common good and hold our representatives to account.

Here is an analysis of questions with my commentary. 

 

Q6 Chair: Why are you so reluctant to tell us what may have transpired with those regulators over the weekend? We are going to have them before us.

Bob Diamond: I am trying to think if I had any conversations with regulators over the weekend.

Q7 Chair: You didn’t but Marcus Agius did, didn’t he?

Here is an example of putting the words in the mouth of Mr. Diamond. It is also an example of asking a closed question.  Moreover, it asks Mr. Diamond to speculate on something he cannot know. Instead, he should have asked “Mr. Agius had discussions with regulators and others over the weekend, what did he explain to you from those conversations.”

Bob Diamond: Chairman, I think it is as simple as this. If Marcus had conversations with regulators, that is a conversation for him to have with you. I did not discuss that with him; I just discussed my reasons.

Here is an example of a missed opportunity for a follow up question.  Instead of focusing on regulators, the question should be about widening to other people that were contacting Mr. Agius.  As it stands, there is little to cross-reference against Mr. Agius when he comes to the Committee.

Q8 Chair: It is widely held that regulators have lost confidence-and it’s not just LIBOR-in your leadership of Barclays. Why do you think that is?

This is an example of a good question. It is open ended, it forces Mr. Diamond to respond and explain his position, which allows for further follow up questions.

Bob Diamond: I think there has been an unfortunate series of events in the past week around Barclays being identified as the first bank in a report that clearly showed very, very bad behaviour by groups of people. How we dealt with that was, I think, appropriate and that was a sign of the culture at Barclays, but that is not coming out.

Q9 Chair: The answer you are giving is that it is the “the first mover disadvantage“.

This is a poor follow up question. The Chair accepts the response at face value. Then he summarizes it in his own words and puts those words in the mouth of the witness. up questions. Why did he overlook the emphasis on “unfortunate series of events?” Is the scandal unfortunate? No. Fortune has nothing to do with the willful intent to manipulate the LIBOR for profit. Alternatively, is the coverage what makes it unfortunate? Why not explore his leadership instead of accepting the first mover theory. The two are not connected. Instead, what Mr. Diamond accepts is that the report vindicates the view that his leadership was not capable of reforming Barclays. He was responsible for the unit that was operating inappropriately, yet we do not have that follow up question.

Bob Diamond: Yes.

Q10 Chair: But it is true, isn’t it-at least I have been told-that the FSA were concerned about your appointment as chief executive? They sought assurances from the board at the time of your appointment that there would be a change of culture at Barclays. Is that not correct?

This is a poor question.  The question is closed. No further response is required.  Questions that start “It is true, isn’t it” may provide a good soundbite, but they are not good questions. If the question is true, and the Chair has that evidence, then the whole line of questioning should been changed.  The questions should have been on the culture that Mr. Diamond inherited *and* to which he contributed as a senior manager. Instead, the Chair fails to listen and ask probing follow up questions.

Bob Diamond: That’s the first I’ve ever heard that there was any question about my appointment as chief executive. I certainly went through, as a chief executive appointment would, interviews with the Financial Services Authority, and I got very strong support for my appointment to chief executive.

Q13 Chair: And you know nothing at all about the suggestion that you were asked to provide assurances that you would challenge your long-term colleagues at BarCap not to take excessive risks?

This is an example of a leading question that is poorly phrased and constructed.  A closed question allows the response to be Yes or No.  A better question would have been to ask, “Why were you seen as being unable to challenge your colleagues to the extent that the FSA even commented on your perceived inability?”

Bob Diamond: I don’t remember any specific comments, but I am sure there were discussions with the regulators during the process of my succession. My memory is more around whether, having been associated with the investment bank for a number of years, I would be able to disassociate myself so, as a group chief executive, I would be able to leave the running of the investment bank to-at the time-Rich and Jerry.

Q14 Chair: Is it true that, in February this year, the FSA came to the board and expressed their concerns?

This is a poor question. The answer is obvious, as Mr. Diamond points out. The FSA come to the board every year as part of their normal regulatory work. Someone as experienced as the Chair would or should know that the FSA meets regularly with the Board.

Bob Diamond: I think it’s every year, Chairman, in that February meeting that the FSA comes, so-

Q15 Chair: What was said?

                This is a good question to get Mr. Diamond to talk about the situation.

Bob Diamond: The context of the discussion when it got to controls, which I think is what you are asking about-I should call it the control environment-was that the focus and the tone at the top was something that they were specifically happy with. In particular, they talked to the board about Chris and I and our relations with the regulators, how we dealt with any situation that came up. I am thinking of PPI-

Q16Chair: Isn’t it a bit more specific than that, Mr Diamond? Didn’t they tell you that trust had broken down between the FSA and Barclays?

This is a poor question. The Chair appears to want to avoid the answer to the original question or move the line of questions to something else. The question is leading, it is closed and it does not lead us to anything new.

Bob Diamond: I don’t recall that in the February meeting.

Q17 Chair: Didn’t they tell you that they no longer have confidence in your senior executive management team?

Another poor question by the Chair, it closes the line of questioning. A better question would have been to ask Mr. Diamond what he understood from the letter and how he acted after receiving the meeting. How Mr. Diamond acted after the meeting would have shown more about what he took from the meeting than asking him.

Bob Diamond: No, sir.

Q18 Chair: And wasn’t all this followed up with a letter?

Again, a poor question unless the Chair is trying to show that the meeting was followed up with a letter.  However, the Chair allows Mr. Diamond to avoid the question without asking about the letter.

Bob Diamond: There was a discussion that, as it got down into the organisation, they felt that there were some cultural issues-that people sometimes push back; that some of the push-back wasn’t always filtered up to the top-so there was an overall discussion on culture. We took some of this as, “This is the annual review from the FSA”, and-

Here is the answer that opens the door to Barclays’ corrupt culture. The Chair walks past it.  He does not wait for a follow up or listen to what is said. The FSA, as Mr. Diamond is admitting, is saying the culture does not allow critical upwards communication. The culture of good news reporting upward calls into question whether there is a healthy internal culture.  It suggests a top down culture in which dissent or bad news is punished, which means any bad news is dismissed or dealt with like the “media management” around the initial LIBOR fixing allegations.  The Chair misses these questions.

If the Chair had listened and used follow up questions, he could have opened up the issue. Instead, he walked past several open doors because he was unwilling or unable to ask the necessary follow up questions.

What should you do to have better questions?

First, ask open-ended questions, then use closed ended questions to change the line of enquiry. The goal is to get the person to tell their story.  When they start to tell you their story, then you can ask the probing follow up questions to get them to reveal more about their story. Otherwise, you remain on the surface.

For example, ask, “Why did you take a note of this telephone call” Avoid asking “Do you take notes of telephone calls?” Unless you are trying to determine whether they take notes.  If they take notes, you can assume that in your first question, or force them to deny they take notes.

The focus should be on “why” and “how” something was done because these allow the person being questioned to explain more about what they were doing and thinking.  The goal is to take them off the prepared script in their mind towards something you want to achieve. 

Third, listen to the answers so that you can ask follow up questions. On three occasions, in the questions that were sampled, a strong follow up question would have allowed us insight into what was happening at Barclays.

Fourth, prepare a question plan for the committee so that questions can be cross referenced. Moreover, a plan allows you to see where follow up questions would have worked better.

Fifth, asking good questions is difficult. The questions have to be prepared and possible answers have to be considered so follow up questions can be prepared.  This is why it is good to consider Robert Jay QC’s technique.

What you ask is what you will find.

 

 

 

 

Security level: Public

More Blog Entries

Is Augmented reality and Augmented decision making coming soon to local government?

  Dan Slee’s excellent post on Augmented Reality and the future of local government...

Miracle data protection cure?: Do tablets stop data breaches?

I was wonder if your organization was using more tablets or other electronic devices for staff...

0 Comments