Evaluating scrutiny

Posted on 20/07/2016 by Ed Hammond. Tags: governance and scrutiny, impact

This is the third in a series of blogposts reflecting on how the work of scrutiny might be evaluated. In the first two posts (here and here), we reflected on recent evaluations of the work of select committees by the Constitution Unit and the Institute for Government. In this post, we’ll develop some of those ideas a bit further and explain more about the way we are rethinking our own framework for evaluating scrutiny in local government.

Just as we have seen in our last couple of blogposts, demonstrating “influence and impact” is difficult. Study inevitably leads to more questions; the process of evaluating scrutiny itself can seem resource-intensive and complex.

In local government, things are more complicated still. A method for evaluating scrutiny which is developed nationally (by an organisation like ours) has to be equally relevant to the scrutiny functions of hundreds of councils. Scrutiny differs markedly from council to council. How are we meant to make this work?

Our first evaluation framework, which we launched in 2006, sought to take an approach which recognised the basic building blocks of scrutiny. We developed this with our second iteration in 2012, which highlighted the culture and values of those involved, and recognised that structures aren’t everything when it comes to evaluating scrutiny, and improving it.

Now, in 2016, we’re seeking to refine that message even further. We’re using the detailed research we mentioned in our earlier blogposts to do that. We’re also looking at learning developed since 2012, such as the Wales Audit Office’s fifteen “characteristics of effective scrutiny”.

The basic components of the exercise remain the same. The focus remains on impact and influence.

• First comes the need to reflect on existing practice and performance. Context is key to this kind of improvement; scrutiny is a political process which sits at the heart of a council’s decision-making processes, and it’s difficult to talk about its operation too much in the abstract. This part of the process involves scrutiny looking at its existing systems and processes and how they work practically to hold the executive to account, and make a difference to local people. But it also involves looking outwards, at the broader context for the local area – what’s changing? What are the big challenges? How are those challenges being addressed? These questions can help scrutiny to clarify its role.

• Second comes the identification of scrutiny’s key role – what scrutiny will do in future to add value. This is about more than saying that scrutiny is there to act as a “check and balance” – it is about challenge broad-brush statements of the scrutiny role and challenging you to give it some focus. This expands on some of the research we did that culminated in “The change game”, our work on the scrutiny of transformation and major change, in 2015;

• Third is about the identification of new ways of working that will help scrutiny to do its work. Scrutiny is about more than committee agendas and a smattering of task and finish reviews – there are a wide range of methods to gather evidence, and this part helps to identify, review and sanction to adoption of those methods;

• Fourth is the agreement of a new structural model. You’ll notice that this part pretty much comes at the end. It’s only when scrutiny’s role is clear, and the ways it will work have been agreed, that you will know what structures are necessary to support that work. The mistake that many make it to start and end with the structural discussion – which invariably ends up being fraught and difficult – instead of developing a common understanding amongst members and officers about scrutiny’s role and purpose, which can be used to reach consensus on structural points;

• Fifth is the need to evaluate and review. Evaluation is a continuing process and needs to be embedded in the way that scrutiny works. Annual reports present a useful way to do this.

What, though, does this mean for impact and influence? Well, reflection on your performance can help you to understand how and where this might be improved. The research into select committees on which I dwelt in previous posts highlighted a number of things that are important in securing influence – having a clear sense of objectives, being prepared to be flexible, recognising the need to liaise with Cabinet and other decision-makers. This comes down to fundamental issues like the status of scrutiny, the relationship between scrutiny and others, and scrutiny’s recognised expertise and credibility to carry out its role.

Making scrutiny effective requires a return to first principles, to consider how influence might be maximised by changing the focus and function of scrutiny overall. Things may have worked well for a long term – but the pressure is now on to change. How can that change be an opportunity – a chance for scrutiny to be seen much more as an integral part of a council’s governance arrangements?

The plan is that this framework for evaluating scrutiny will provide the tools necessary to answer that question, and others. Publication is planned for the end of the summer.