Fast policy, slow evaluation…

… be in no doubt – we are in a hurry, we mean to deliver. You don’t need to talk about it or show empathy. I want you to get on with it. And I know local government can get results. But understand – this isn’t either or. We are going to deliver on this. So get moving. 

Eric Pickles, 17 October, 2011

I’ve written before (and here) on how the Troubled Families Programme (TFP) can be understood as a good example of the phenomenon that Peck & Tickell call ‘fast policy’ whereby an off-the-shelf policy fix (‘family intervention’ in this case) can be quickly dusted off, thereby shortening any pilot or implementation stages, and smoothing the way for rapid expansion.

One area where DCLG appear to be taking their time, however, is with the official evaluation of the first phase of the programme. This, of course, is no bad thing and I don’t want to suggest that these things should be rushed. But given the speed at which the TFP was established and then expanded, the length of time it is taking to publish results from the evaluation appears at odds with the prorgamme’s overall approach to ‘getting on with it’.

In October last year, I wrote to the DCLG asking them if they could provide me with the titles of any reports they had received as part of the official evaluation and the dates they had received them. They initially refused to provide me with this info, but, following a review, I was provided with the following dates and titles of reports that had been received from Ecorys, the lead partner in the evaluation consortium:

  • National Evaluation of the Troubled Families Programme: Final Synthesis report [DRAFT] – 18 September 2015
  • National Evaluation of the Troubled Families Programme: – Technical report: impact evaluation using survey data [DRAFT] – 15 October 2015
  • National Evaluation of the Troubled Families Programme: – Findings from the Analysis of National Administrative Data [DRAFT] – 28 August 2015
  • National Evaluation of the Troubled Families Programme: Report on the Process evaluation [DRAFT] – 1 September 2015
  • National Evaluation of the Troubled Families Programme: Family experiences report [DRAFT] – 7 October 2015
  • National Evaluation of the Troubled Families Programme: Survey Technical report [DRAFT] – 9 October 2015
  • National Evaluation of the Troubled Families Programme: Family Monitoring Data final report [DRAFT] – 9 October 2015

The original response to my initial request did state that:

The department will be aiming to publish the evaluation within 12 weeks of final sign-off in accordance with Government Statistical Research guidelines, and current expectations are that we will achieve sign-off of the evaluation materials by the end of this calendar year.

According to the government’s own expectations, then, the evaluation should have been published by now (twelve weeks from the start of the New Year would have been around the 21 March). The final synthesis report has been with the DCLG for nearly seven months now and still we don’t know what the official evaluation has to say about the programme. The initial response also stated that the evaluation would be published once ‘the necessary quality assurance processes’ had been completed, which, again, is entirely understandable.

However, the delay does raise some interesting questions. I wonder, for example, if the evaluation was positive and produced some robust justification for the programme, would there have been such a delay in publishing it? Given that the ‘quality assurance processes’ are taking such a long time, does this indicate a problem or problems with the data collected during the evaluation? And given that nearly seven months (and counting) has passed between submission of the final report and its publication, how different will the published version be to the initial draft, after seven months of revising and redrafting?

If anyone has any answers to these questions, please do let me know.