CYPN in a piece, Tool to help youth sector prove its worth reports that,
Moves to boost investment in youth work have taken a step forward with the publication of a guide to help the sector measure outcomes.
Think-tank The Young Foundation has created the document to help providers demonstrate evidence of their impact, by creating standardised measurements across young people’s services.
It forms part of the work of the government-funded Catalyst consortium, which is charged with helping the youth sector adapt to the changing policy environment.
Over on the Young Foundation site itself Bethia McNeil, co-author of the research, in a blog, When is self-evidently good not good enough? outlines the rationale for the focus on outcomes related to social and emotional capabilities and challenges us to take ‘ a collective breath and blink’.
Whatever our failings we never fight shy of ‘collective breath taking’. Thus several of us in the Campaign have immersed ourselves in the research and come up blinking with deep reservations. As a starter for debate we are pleased to post Bernard Davies’s initial critical thoughts.
A framework of outcomes for young people
The Young Foundation
The Young Foundation framework
The Executive Summary sets out the paper’s purposes and approach for the Framework of Outcomes for Young People it is proposing. It sees this as:
… designed to highlight the fundamental importance of social and emotional capabilities to the achievement of all other outcomes for all young people. It:
- proposes a model of seven interlinked clusters of social and emotional capabilities that are of value to all young people, supported by a strong evidence base demonstrating their link to outcomes such as educational attainment, employment, and health
- sets out a matrix of available tools to measure these capabilities, outlining which capabilities each tool covers, and key criteria that might be considered in selecting an appropriate tool – such as cost or the number of users
- outlines a step by step approach to measuring these capabilities in practice, that is illustrated in four case studies that exemplify how the Framework might be used by providers, commissioners and funders.
The Framework describes itself as aiming ‘to address the key challenges in measuring impact on the lives of young people’. It seeks to do this in order to ‘support progress towards a future in which providers are confident and able to evidence their impact, and commissioners are confident to supplement their focus on reducing negative outcomes with an equal or stronger focus on commissioning for positive and sustained personal and social development’
- These comments on the proposed Framework start from the overall proposition that this is important work; that is, that we do need to be trying to develop credible ways on gathering and presenting evidence of youth work’s value to young people.
- The paper is a careful and thoughtful attempt to do this – e.g:
- It makes clear that it does not see its approach as ‘stand alone’ (21).
- It acknowledges the difficulties and constraints of any exercise seeking to get beyond the ‘easy’ measures of numbers etc and to capture ‘soft’ outcomes. (7)
- It also acknowledges the difficulty of distinguishing cause and effect in these kinds of human processes. (14)
- It attempts to see how ‘intrinsic’ (personal) and ‘extrinsic’ (‘societal’) outcomes connect. (14-15)
- It stresses the need for any evaluation scheme to stay proportionate (eg to the size and aims of the piece of work) and to ‘fit’ the people and the tasks involved. (27)
- In seeking outcomes for young people, it recognises the need to focus on groups and group work and to get beyond targeted categories. (19-20, 27)
- It gives some recognition to the role of youth work in the services and provision it is considering with references to significant sources going back to the 1970’s (the Manchester Wincroft Project – 7) and including the Occupational Standards for Youth Work (6), the Merton et al ‘Impact’ study (11) and the recent London Youth ‘Hunch’ report (14). (This has to be seen however in the context of a later apparent conflation of youth work with youth development – 34).
Queries and cautions
- In some (key) ways this is a document of its time – for example:
- It offers only selective and uncritical quotes from Positive for Youth (6) which miss its constant refocusing on targeted work rooted in deficit models of young people.
- The audience addressed is repeatedly that of ‘providers’ – especially ‘funders, commissioners and investors’ (7) – so that, intentionally or not, definitions of what is needed emerge as primarily top-down. Though at one point the paper acknowledges that ‘the young people you work with, and how you work with them, will influence your practical approach to measurement’, (27)it is therefore hard to see most of the time where and how young people’s own starting points for the work – or indeed the strongly affirmed broader ‘youth voice’ – are going to influence its precise shape and direction.
- Despite the reference to the need for group work and a denial that the framework is seeking to ‘decontextualise work with young people’ (20), the only constraints on ‘empowerment’ explicitly recognised in the paper are the interpersonal (families, peers) or the institutional (schools) (14). (The word ‘power’ itself never appears in the document). Young people emerge as a monolithic undifferentiated group: indeed, tellingly, a reference to the effects of social class in a quote from the research by Feinstein (which so influenced New Labour youth policies) is completely ignored in the follow-up comment which again focuses on ‘personal and social development’. (32) No reference is made to the current ‘youth crisis’ in employment, income, housing or transport. The perspective adopted is overwhelmingly individualistic and psychological, repeatedly envisaging ‘empowered, resilient young people, who play an active role in navigating (their transitions)’, without any reference to the wider structural obstacles to such ‘navigation’. (4)
- Such ‘transitions’ are another of the paper’s repeating themes. Young people, it seems, cannot exist as people,now, for their own sake but only as individuals in the making for some future broadly pre-defined roles – efficient worker, good parent, law-abiding and contributing citizen. (38, 49). Moreover though the paper’s aspiration is for them to end up with a wide and fulfilling set of ‘capabilities’, these (even the ‘creative’ ones) emerge ultimately as largely conformist, offering no explicit encouragement to young people to play socially critical or social change roles in their society. (19, 21).
- Despite its stated commitment to viewing young people’s personal and social development in positive ways, it is striking how often the paper falls back on examples about and references to avoiding the undesirable (anti-social behaviour, poor employment outcomes). As I read them, three of the paper’s four case studies assume targeted work.
- The paper is concerned not to present itself as the only way of identifying and presenting outcomes, suggesting for example that it is implemented ‘alongside other approaches such as case studies and witness testimonies’ (21). Nonetheless, in these circumstances the risk exists that hard-pressed managers and practitioners will use it in simplistic and uncritical ways and as the way of demonstrating to ‘funders, commissioners and investors’ that they deserve their money. A critical and continuing commentary and debate on the paper, including on its limitations, therefore seems vital.
Perhaps understandably Susanne Rauprich, chief executive of the National Council for Voluntary Youth Services, which leads the Catalyst consortium is less cautious than Bernard. CYPN reports that she believes the framework “will succeed in its aim of underpinning social investment work by enabling providers and commissioners to demonstrate the difference they make”.
She added that the youth sector often struggles to provide quantitative evidence of its effectiveness. “By addressing this problem, the framework will open the gateway for new finance and entrepreneurial capacity,” she said.