It’s been an uncomfortable few weeks for the concept of impact, so dominant nowadays in thinking about youth work and the youth sector.
Firstly the much trumpeted post-2011 Troubled Families Strategy is exposed as an exercise in wilful deceit. A scathing study undertaken by an independent research consortium including the National Institute of Economic and Social Research finds that after four years there was no clear evidence that the programme had any tangible effect, despite persistent claims by politicians that it had “turned around” the lives of tens of thousands of families and saved over a billion pounds. One of its authors going so far as to say, “The troubled families programme has no significant impact on any of the key outcomes it was designed to change. As far as we can tell, there’s no evidence at all to suggest the programme had more than zero impact on any of the key findings it was designed to change.”
We drew attention to this state of affairs back in February, ‘Troubled Families is a fraudulent scam’- some thoughts from within’, drawing on a report by Stephen Crossley of the Centre for Crime and Justice Studies. Yet, despite the widespread unease about the programme, pursuing its outcomes agenda has been central to the restructuring of local government services and has impacted significantly on the role of the remnants of the Youth Service. For example in Lancashire the Youth Service is about to be submerged within the Wellbeing, Early Help and Prevention Service and ‘aligned’ to the TFU programme. By all accounts open access youth work will disappear.
This paper describes the implementation plan of the service offer proposals
presented to Cabinet in February 2015. It includes a description of the current state
of the services to be integrated and proposes a future service model to be delivered
within a revised financial envelope of £17,230,000. This represents a £7.4million
budget saving by 2017/18, based on current services spend (2015/16). The paper
outlines the service delivery model proposal to transform and fully integrate a range
of services within Wellbeing, Prevention and Early Help Service (WPEHS), which
will be implemented subject to consultation. The resultant integrated delivery model
will align existing core offers for Children’s Centres, Young People’s Provision,
Prevention and Early Help and Lancashire’s response to the national Troubled
Families Unit national programme.
Secondly Paul Oginsky, infamous in some circles for his role as an adviser to David Cameron and as an architect of the flagship National Citizen Service, has evidently seen the light. He declares,
“For many years, people in the youth sector have been looking for a definitive and universally accepted way of measuring the impact of their work.
So many people have tried and so many methods have been proposed that this quest has become known as the ‘Holy Grail’ of youth work. Let us kill this myth now (spoiler alert), there is no holy grail. There is no way of measuring impact on people that is definitive and universally accepted. There is no such thing as a unit of confidence, loyalty, honesty, motivation or any of the characteristics which this kind of work seeks to impact.”
He might have added with a touch of humility, that for many years many within youth work, including leading figures such as Batsleer, Davies, Jeffs, Smith, Bright and Yeung, together with In Defence of Youth Work and a host of practitioners rendered redundant, have been criticising the illusions of the outcomes and impact agenda. They have never got on the horse pursuing this particular Holy Grail.
Forget measurement, psychometric testing, pre and post course tick boxes, the best way of assessing the impact of this kind of work is to ask the person themselves to describe if and how they have changed, and then to ask for witnesses verification by those who know them.
Although this needs some serious unpacking we might discern a hint of the story-telling perspective advocated by ourselves – see our designated web site here. Certainly though we need to engage with the arguments contained in the paper, A Way Forward For Character Development: The missing piece of education , published by Personal Development Point, of which Paul is the Chief Executive.
Clearly Bethia McNeil of the Centre for Youth Impact and Phil Kerry of London Youth are of the same mind, finding themselves in agreement with much of Paul Oginsky’s thesis.
Bethia comments, Paul Oginsky is right that there is no ‘definitive and universally accepted way of measuring impact’. A ‘definitive approach’ implies agreement, a conclusion and a sense of authority. Impact measurement in the youth sector has none of those features. He’s also right that finding this definitive and universally accepted approach has at times seemed like the Holy Grail. This is in part because impact measurement has been connected with sustainability, and the suggestion has long been that the ability to ‘prove’ your impact or worth is associated with turning on the funding taps. A search for the Holy Grail is also motivated by the belief that it exists – that it is possible to find a definitive and universally accepted approach to measuring impact. Again, Paul is right – there isn’t one.
Phil Kerry writes, I read with interest Paul Oginsky’s recent white paper, A Way Forward for Character Development: The missing piece of education. Much of what Personal Development Point says rings true. There is no holy grail to character measurement within our sector and the sooner we all realise this the better. But attempting to condense all of this into one evaluation tool or definition would just be just as much of a fruitless mission. Our sector embraces a brilliantly wide church of practice and thinking and in this lies our strength and not our weakness.
Read their responses in full here.
To repeat the point made above, we can but suggest that a critical caution about imposed templates of universality has been around for ages. Indeed it is caught in youth work’s long-standing acknowledgement that it is a contested ideological space. It is fascinating that this fundamental insight is by and large ignored, even when notions of social change are allowed into the debate. We will endeavour to explain this further in a measured response to the PDP Oginsky pamphlet. Meanwhile there is reason to believe that the discussion about ‘what we’re up to’ is loosening up.