Today, Bernard Davies responds to fellow IDYW steering group member Tania de St Croix’s new article in Pedagogy, Culture and Society. Tania’s article reflects on the IDYW storytelling process and its potential as an approach to evaluation, as part of a wider argument for participatory democratic accountability in youth work and beyond. Read the article here (now open access).

Though – deep into the Covid-19 era – it now feels like the distant past, in the decade after 2009 ‘story-telling’ workshops became something of a distinctive IDYW offer to the youth work field. Over those years 58 events were held attracting some 1400 participants – full- and part-time face-to-face workers (paid and voluntary), service managers from both the statutory and voluntary sector, youth work students and their tutors and on occasions other ‘people practitioners’. Though mainly in England, workshops were hosted in all the UK countries, and in Ireland, Finland, Argentina, Czech Republic, and Kazakhstan, and for a group of visiting Japanese practitioners and researchers who have now incorporated story-telling into their work. They were used, too, to gather ‘stories from practice’ for a book published in 2011 and for a web resource sharing story-telling experience and materials.[1]

Shaped initially by the ‘Socratic dialogue’ approach to group discussion and exploration developed by Sarah Banks[2], the sessions’ overall framework was implemented and periodically reviewed by a group of some ten volunteer IDYW ‘facilitators’. Over two-to-three hours, each workshop sought to ‘unpick’ an example of ‘the practice as a youth worker’ of one of the participants, aimed particularly at clarifying if and how (or not) this fitted with IDYW’s definition of ‘open youth work’.[3] As such, the workshop programme (if that it what it can be called) was a largely ‘bottom-up’ response to the dominant neo-liberal assumptions of the time which were constraining and indeed often diverting youth workers from this process-led conception of a practice self-chosen by young people.

As one of the workshop facilitators, I’d feel I was rewriting history if I claimed that we were consciously locating what we were doing in wider related ‘training’ methodologies or explicitly drawing on relevant academic research. Nor was our main focus to seek evidence from the stories of the ‘outcomes’ of the practice for the young people involved or its ‘impacts’ on them.

Now available for free access on the web, however, is an article which explicitly sets out to fill these gaps.[4] It is written by Tania de St Croix, a Lecturer in the Sociology of Youth and Childhood at King’s College London, who comes to the task also as a practising detached youth worker – and as a regular workshop facilitator.

In developing her commentary and analysis, Tania spells out some of the core features of that IDYW conception of ‘open youth work’. She for example explains it as

… not a time-bound intervention with predefined outcomes, but rather a long-term negotiated engagement in which young people participate in different and changing ways for a variety of reasons, in the context of a systematically unequal society.

She also offers insights into the workshop process itself and its relevance to such a practice, emphasising that

…it was vital to communicate the value of youth work as part of a campaign to defend this practice, while avoiding deficit discourses that position youth work as ‘preventing crime’, ‘reducing teen pregnancy’, ‘saving money on prison spaces’ or ‘keeping kids off the streets’.

Most positively, she records, too, how the workshops enabled ‘a focus on young people’s and practitioners’ grounded experiences and what they value about youth work’ . As such, she sees them as acting as ‘a form of prefigurative practice, embodying the change we want to see in the world, both in youth work and in the wider education sector’.

The brief Tania set herself, however, is much wider. Its starting point is to treat the IDYW workshops as an expression of resistance to the demands of that neo-liberal ideology – particularly to its ‘top-down measurement-based approaches to accountability’. These she describes as presenting a ‘dominant policy discourse of evaluation as scientific, factual, and value-free’ ; as inhibiting ‘contestation and dissent’; as preoccupied with demonstrating ‘value-for-money’; and as simplifying practice and separating it from its wider social context.

As well as referencing a wide range of original sources, Tania draws on evidence she and a colleague have been gathering over three years from their own research project. This is particularly focused on how ‘… evaluation, impact measurement and accountability systems play out in youth work settings, and how they are experienced and perceived by a range of actors: young people, youth workers, managers, and policy influencers’.[5] Amongst a range of such impacts, this evidence points particularly to

Young people and practitioners often experiencing evaluation and monitoring as oppressive, intrusive, and inauthentic, particularly when it is based on quantitative ‘before and after’ questionnaires, or attendance and outcomes data logged on spreadsheets.

In response to these damaging intrusions into the practice Tania advocates for

… democratic evaluation and monitoring (which) values the expertise of young people, practitioners, family and community, rather than the political or pragmatic interests of resource holders.

One suggested way of working towards this goal is to locate IDYW-style story-telling within wider ‘narrative’ forms of educational practice. This however, as the article makes clear, comes with its own complications. It points, for example, to how ‘despite narrative research being a respected method of gathering meaningful data in the social sciences and humanities, it is less widely recognised in policy and practice evaluation’. Acknowledged, too, are the restrictions that ‘neoliberal logics’ can place on developing and implementing narrative methods and bottom-up evaluation more widely.

Nonetheless, the article highlights how ‘narratives have long been used in liberatory activism, practice and research as a powerful challenge to top-down knowledge production’, including ‘as a way of centring minority views’. Specifically in relation to youth work, it points to how such methods ‘can function as both alternative and resistance to monitoring based on outcomes and metrics’. As an example of narrative approaches ‘gaining traction’, it points in the European youth work context to ‘… Transformative Evaluation, in which young people are asked to discuss the “most significant change” they have experienced as a result of their participation in youth work’.[6]

Starting from the proposition that, ‘as youth work emerges from a decade of spending cuts, there is an urgent need for new approaches to evaluation, the article goes on to argue for these kinds of narrative ‘dialogical’ processes. They’re seen as potentially ‘more participatory and inclusive than metric systems’ and so as offering ‘… conditions under which democratic evaluation techniques can thrive’. Moreover, within such a framework, story-telling could be capable of providing a more authentic form of ‘service evaluation and accountability’ not just for youth workers and their managers but also by funders – as illustrated by one project’s inclusion of material from IDYW storytelling workshops when reporting back on its five-year project.

The article, however, explicitly distances itself from what it calls ‘a romanticised view’ of how any particular method might be utilised in practice’. It is clear, for example, that

… while narrative forms of evaluation may tend to be more participatory and inclusive than metrics systems, if they are implemented prescriptively from above they will often be experienced as an imposition that neglects the realities of practice and become a burden on educational relationships.

It acknowledges, too, that as a ‘tool cannot be democratic if the conditions surrounding it are not also democratic’, in the current neo-liberal context ‘the widespread use of narratives in accountability is difficult to imagine’. It thus concludes that, beyond new methods for evaluating and monitoring educational practice, what are required are ‘radically different conditions from those that currently exist’.

More immediately it identifies three ‘challenges’ to developing narrative-based forms of evaluation:

‘Resisting deficit narratives’

Given that stories are ‘not inherently anti-oppressive and inclusive’ or that ‘the dominant use of narratives by youth organisations is not always positive’, those implementing these methods will need to ensure that ‘the process of inviting and discussing stories, and any communication of these stories, (are) sensitive, ethical, and attentive to power and inequalities’.

‘Avoiding top-down implementation’

As ‘narrative methods can be incorporated within managerial accountability systems, with their associated problems of bureaucracy, incoherence and inauthenticity’, recognition will be needed that storytelling specifically ‘is not immune from becoming a top-down mechanism’. Tensions can emerge, for example, when ‘storytelling workshops are inserted within an existing hierarchical accountability framework’ – as was illustrated by New Labour’s inclusion of ‘qualitative accounts of young people’s progress within a broader system of target-based accountability’.

‘Providing time, space and support for reflection’

Even though ‘few resources are needed to run a storytelling workshop’, what will also need to be taken into account is that all ‘alternative methods … are potentially constrained by factors that affect evaluation more broadly: inadequate time, space, and support’. This is likely to be especially true where ‘there is a legacy of cuts, redundancies, closed courses, sold off buildings and depleted trust amongst practitioners and community members’ resulting in ‘real constraints in terms of the time, energy and confidence needed for in-depth evaluation of any kind’.

With an aspiration of turning their current accountability procedures ‘on their head’, all this leads Tania to pose a final list of ‘transformative’ questions ‘for decision makers and budget holders’. Somewhat abbreviated, these are:

• Do we build meaningful relationships with the recipients of resources, including young people and those directly working with them, or do we mainly encounter them through a report or a spreadsheet?
• Do we build trust with those we are considering funding and supporting?
• Have we thought about what kinds of funding or support would be most appropriate for the educational practice that is being evaluated?
• Do our systems favour organisations that are rooted long-term in communities, or those with the largest fundraising and administrative infrastructures?
• Do we look at how our processes embed, reinforce, and/or challenge inequalities and stereotypes …?
• Do we encourage practitioners to evaluate practice in ways that are meaningful in their process as well as their results…?
• Do we support organisations to … support participatory decision making by practitioners, young people (including those from marginalised groups who may not be served by current practices), and (where relevant) parents, carers, family members and the wider community?

For agencies which still exert so much indirect and even, as Tania shows, direct power over the delivery of youth work, these remain very real challenges.

In her conclusion, Tania also reiterates that the methods she’s been discussing ‘have considerable potential in other educational settings such as schools, adult education, universities and arts-based learning’. Her article, however, surely also opens up a more immediate and grounded opportunity: to explore whether, as again a bottom up response to whatever new (perhaps digitally dominated) ‘normal’ might be emerging, there is a need, space and the means for developing an updated story-telling offer to youth workers and their organisations.

This article’s searching analysis and critique have for me put that question right back on the agenda.

Bernard Davies                                                                                                      January 2021

---

[1] IDYW, 2011, This Is Youth Work: Stories from Practice. London: Unison, story-tellinginyouthwork.com/; IDYW. 2014. Youth Work Story-telling: A Resource for Workers, Managers, Tutors and Students, https://story-tellinginyouthwork.com/

[2] Banks, S. 2013. “Socratic Dialogue and Co-inquiry: Exploring Cognitive, Affective and Embodied Ways of Knowing”, Ways of Knowing website, 19 May https://waysofknowingresearch.wordpress. com/2013/05/19/socratic-dialogue-and-co-inquiry-exploring-cognitive-affective-and-embodiedways-of-knowing/

[3] IDYW. 2009, ‘The open letter’, at https://indefenceofyouthwork.com/the-in-defence-of-youth-work-letter-2/

[4] Tan de St Croix, 2020, Re-imagining accountability: storytelling workshops for evaluation in and beyond youth work’, Pedagogy, Culture and Society, 2 December, pp ??, available at https://doi.org/10.1080/14681366.2020.1855231  

[5] See Doherty, L. & T. de St Croix. 2019, “The Everyday and the Remarkable: Valuing and Evaluating Youth Work”, Youth and Policy, https://www.youthandpolicy.org/articles/valuing-and-evaluatingyouth-work/

[6] Cooper, S. 2014, “Transformative Evaluation: Organisational Learning Through Participative Practice.” The Learning Organization 21 (2): 146–157