Rod Norton responds to Bethia McNeil’s pertinent musings

At the end of last week, we asked for responses to Bethia McNeil’s musings upon key issues in evaluating our practice, Here they are again.

Thoughts From The Centre – to be found in the very useful Centre for Youth Impact Newsletter
This month, Bethia has been mostly thinking about…

What is the most meaningful language in which to talk about provision for young people? Does it make sense to talk about different ‘fields’ of practice? Or approaches? Or even practices? And what about ethics, values and principles? To what extent is shared learning held back by a lack of common language and/or understanding?
What is the relationship between ‘organisational’ or ‘practice improvement’ and improvement in outcomes for young people? Are there common areas of improvement, or is it more nuanced? How much is about systems and processes, and how much about relationships?
How can we value the act of measurement as much as the data that we are gathering? It feels like there are fractures on both sides of this issue at the moment: there is widespread antipathy towards the act of measurement and its impact on youth provision, and similarly a scepticism that the data gathered can tell us anything meaningful about our engagement with young people. Where to start?
What do we actually mean when we talk about ‘what works’? To what extent is this in the eye of the beholder?

judgement

Thus we are really pleased to post a reply from Rod Norton, a long-time youth worker and the former Chief Officer of a local authority Youth Service.

 

Tony’s call for people to engage in the debate with the Centre for Youth Impact got me thinking that his distinction between judgement and measurement is fundamental to the discussion. Let me explain ….

 
One way to look at youth work is that it is about helping young people to engage with, and change, the world about them. It is therefore about values, relationships, debate, discussion and above all about the active engagement of young people in their communities – it is a process, not an outcome. Predefined outcomes are therefore problematic as, from this point of view, the very purpose of youth work is to help young people gain the insights, skills and confidence necessary to change the world in ways that have meaning for them and not necessarily in ways that have meaning for workers or funders. Youth work therefore inhabits the realm of politics (in the widest sense of the word, we’re not talking about party politics here) and had its heyday under the Social Democracy of the post-war period where a commitment to process driven, user-focussed services was fairly mainstream. Evaluating youth work in this context involves making judgements about the work that are based upon transparent and contestable moral and political values. In the end, the fundamental question is whether the work advances the common good – which makes it political.

 
The type of society that has developed over the past thirty years is very different to Social Democracy and is based upon very different foundations. The neoliberalism which now dominates our lives doesn’t value politics, it values economic efficiency. For neoliberals, what matters is success in the marketplace and here politics, morals and values are largely irrelevant. Neoliberalism therefore has a desire to turn social activities, such as youth work, into products that can be traded in the market in order to make a profit, or at least to save money spent elsewhere. As products always have to be sold, the key driver for the work under neoliberalism becomes the wants and needs of the buyer or funder – the views of young people are secondary. And, of course, funders always want proof that they have received what they have paid for, so the emphasis of evaluation shifts to the measurement of outcomes and to the generation of savings or profits. Youth work thus risks becoming a value free commodity, delivering only funder defined outcomes. The natural tendency under these influences is for youth work to move away from an open access, user led and process-based format, towards more individualistic and formal models delivering predetermined behavioural change to passive young people.

 
Of course, these two models of evaluation based on political judgement and economic measurement have been contrasted here in very stark terms, whereas in reality the two usually merge into each other in some form of messy compromise. But, in the exchanges between Tony and Bethia over the last couple of years it is Tony who inhabits the world of political judgement whereas it is Bethia who, even though she valiantly struggles against their worst economistic excesses, is more influenced by models based on the measurement of economic efficiency. It is therefore Tony who is trying to move beyond neoliberalism whilst it is Bethia who is trying to find some kind of progressive accommodation with it.

 

Bethia McNeil muses on the dilemmas of naming and measuring youth work

cyi-new-logo

Our relationship with the Centre for Youth Impact [CYI] remains inevitably delicate. We are deeply cautious about the consequences of its perspective for a process-led, person-centred youth work practice. However, we have sought to be in critical dialogue with CYI and this desire has been reciprocated. Hence it’s important to engage with Bethia McNeil’s New Year’s musings on what she sees as key issues in the ongoing debate about the relationship between something we call youth work and the heterogeneous layer of humanity we call young people.

Thoughts From The Centre – to be found in the very useful CYI Newsletter
This month, Bethia has been mostly thinking about…

  • What is the most meaningful language in which to talk about provision for young people? Does it make sense to talk about different ‘fields’ of practice? Or approaches? Or even practices? And what about ethics, values and principles? To what extent is shared learning held back by a lack of common language and/or understanding?
  • What is the relationship between ‘organisational’ or ‘practice improvement’ and improvement in outcomes for young people? Are there common areas of improvement, or is it more nuanced? How much is about systems and processes, and how much about relationships?
  • How can we value the act of measurement as much as the data that we are gathering? It feels like there are fractures on both sides of this issue at the moment: there is widespread antipathy towards the act of measurement and its impact on youth provision, and similarly a scepticism that the data gathered can tell us anything meaningful about our engagement with young people. Where to start?
  • What do we actually mean when we talk about ‘what works’? To what extent is this in the eye of the beholder?

 

If I can get my act together I’ll scribble some sort of response to these pertinent questions in the coming week, but it would be brilliant if other folk contributed.

Newsletter
The CYI newsletter collects news, events, research and blogs from the Centre, its networks and practitioners and organisations around the world. Sign-up here

Impact takes a bit of a troublesome knocking and from an unexpected quarter

It’s been an uncomfortable few weeks for the concept of impact, so dominant nowadays in thinking about youth work and the youth sector.

troubled

Firstly the much trumpeted post-2011 Troubled Families Strategy is exposed as an exercise in wilful deceit.  A scathing study undertaken by an independent research consortium including the National Institute of Economic and Social Research finds that after four years there was no clear evidence that the programme had any tangible effect, despite persistent claims by politicians that it had “turned around” the lives of tens of thousands of families and saved over a billion pounds. One of its authors going so far as to say, “The troubled families programme has no significant impact on any of the key outcomes it was designed to change. As far as we can tell, there’s no evidence at all to suggest the programme had more than zero impact on any of the key findings it was designed to change.”

We drew attention to this state of affairs back in February, ‘Troubled Families is a fraudulent scam’- some thoughts from within’, drawing on a report by Stephen Crossley of the Centre for Crime and Justice Studies. Yet, despite the widespread unease about the programme, pursuing its outcomes agenda has been central to the restructuring of local government services and has impacted significantly on the role of the remnants of the Youth Service. For example in Lancashire the Youth Service is about to be submerged within the Wellbeing, Early Help and Prevention Service and ‘aligned’ to the TFU programme. By all accounts open access youth work will disappear.

 Proposals for Transforming Wellbeing, Prevention and Early Help
Services for Children, Young People and Families in Lancashire

This paper describes the implementation plan of the service offer proposals
presented to Cabinet in February 2015. It includes a description of the current state
of the services to be integrated and proposes a future service model to be delivered
within a revised financial envelope of £17,230,000. This represents a £7.4million
budget saving by 2017/18, based on current services spend (2015/16). The paper
outlines the service delivery model proposal to transform and fully integrate a range
of services within Wellbeing, Prevention and Early Help Service (WPEHS), which
will be implemented subject to consultation. The resultant integrated delivery model
will align existing core offers for Children’s Centres, Young People’s Provision,
Prevention and Early Help and Lancashire’s response to the national Troubled
Families Unit national programme.

oginsky

Secondly Paul Oginsky, infamous in some circles for his role as an adviser to David Cameron and as an architect of the flagship National Citizen Service, has evidently seen the light. He declares,

“For many years, people in the youth sector have been looking for a definitive and universally accepted way of measuring the impact of their work.
So many people have tried and so many methods have been proposed that this quest has become known as the ‘Holy Grail’ of youth work. Let us kill this myth now (spoiler alert), there is no holy grail. There is no way of measuring impact on people that is definitive and universally accepted. There is no such thing as a unit of confidence, loyalty, honesty, motivation or any of the characteristics which this kind of work seeks to impact.”

He might have added with a touch of humility, that for many years many within youth work, including leading figures such as Batsleer, Davies, Jeffs, Smith, Bright and Yeung, together with In Defence of Youth Work and a host of practitioners rendered redundant, have been criticising the illusions of the outcomes and impact agenda. They have never got on the horse pursuing this particular Holy Grail.

He continues,

Forget measurement, psychometric testing, pre and post course tick boxes, the best way of assessing the impact of this kind of work is to ask the person themselves to describe if and how they have changed, and then to ask for witnesses verification by those who know them.

Although this needs some serious unpacking we might discern a hint of the story-telling perspective advocated by ourselves – see our designated web site here. Certainly though we need to engage with the arguments contained in the paper, A Way Forward For Character Development: The missing piece of education , published by Personal Development Point, of which Paul is the Chief Executive.

Clearly Bethia McNeil of the Centre for Youth Impact and Phil Kerry of London Youth are of the same mind, finding themselves in agreement with much of Paul Oginsky’s thesis.

 Bethia comments, Paul Oginsky is right that there is no ‘definitive and universally accepted way of measuring impact’. A ‘definitive approach’ implies agreement, a conclusion and a sense of authority. Impact measurement in the youth sector has none of those features. He’s also right that finding this definitive and universally accepted approach has at times seemed like the Holy Grail. This is in part because impact measurement has been connected with sustainability, and the suggestion has long been that the ability to ‘prove’ your impact or worth is associated with turning on the funding taps. A search for the Holy Grail is also motivated by the belief that it exists – that it is possible to find a definitive and universally accepted approach to measuring impact. Again, Paul is right – there isn’t one.

Phil Kerry writes, I read with interest Paul Oginsky’s recent white paper, A Way Forward for Character Development: The missing piece of education. Much of what Personal Development Point says rings true. There is no holy grail to character measurement within our sector and the sooner we all realise this the better. But attempting to condense all of this into one evaluation tool or definition would just be just as much of a fruitless mission. Our sector embraces a brilliantly wide church of practice and thinking and in this lies our strength and not our weakness.

Read their responses in full here.

To repeat the point made above, we can but suggest that a critical caution about imposed templates of universality has been around for ages. Indeed it is caught in youth work’s long-standing acknowledgement that it is a contested ideological space. It is fascinating that this fundamental insight is by and large ignored, even when notions of social change are allowed into the debate. We will endeavour to explain this further in a measured response to the PDP Oginsky pamphlet. Meanwhile there is reason to believe that the discussion about ‘what we’re up to’ is loosening up.

 

 

Management based on outcomes makes good people do the wrong thing!?

From the outset our Campaign has been critical of today’s obsession with targets and outcomes. Nevertheless the world of work with young people continues to be blighted by this fetish. For example in March – without a caution in sight – CYPN invites us to attend a ‘ crucial one day event at the Aston Business School Centre.

The Outcomes Spring Conference: Best practice in commissioning and delivery for children, young people and families

We will be looking afresh at the Young Foundation’s Framework of Outcomes, which is to be championed at the CYPN event, at our conference in Leeds on March 8.

In the meantime it is refreshing to read Toby Lowe’s analysis of the dangers of outcomes-based management and practice in this piece,

Payment by results – a ‘dangerous idiocy’ that makes staff tell lies

Controversially perhaps he suggests that ‘there’s a growing momentum behind the understanding that outcomes-based performance management in general – and payment by results, in particular – is dangerous idiocy. It makes good people do the wrong things, and then forces them to lie about it.’

This a deeply uncomfortable truth. And youth work is not be excepted from services, whose practice is distorted by deception.

In another excellent blog Helen Wilding, a self-confessed systems practitioner explores The use and abuse of measurement.

She concludes,

‘We need to remember there is nothing objective about measurement.  Choices are made about what is the best measurement or the best proxy.  And, evaluation of whether performance is good enough is a judgement made by a person.  Those choices are formed through our ideologies and what we value.  In a radio programme, Marmot said something like – “arguments about evidence are often arguments about ideologies” – so too arguments about measurements are arguments about what we choose to value, how we choose to measure that and how we choose to use them as ‘tools’ in helping us do what we do.  They can be instruments of learning or instruments of control.’