Evaluation Blog #4: An Evidence Review on involving young people in the design and delivery of programmes

The involvement of young people is a central principle of the Talent Match Programme: the programme seeks to encourage young people to design provision themselves, to give them control over this provision, and to promote new approaches.  Some Talent Match partnerships committed resources to involving young people before the project launch; whilst others have provided more support since the programme went live.

CRESR led an evidence review on the involvement of young people in partnerships. Whilst much is known about youth involvement, relatively little is known about the involvement of 18-24 year olds.  Three forms of involvement were identified: consultation with young people; co-production, where young people and staff work together; or mostly/entirely young- people-led projects.

In-depth case study work with three partnerships followed the evidence review, highlighting these key points:

  • there is a lot of good practice in terms of involving young people
  • the extent of involvement varies within, and across partnerships – there is no one size fits all
  • some approaches are more effective than others, particularly at engaging the hard-to-reach. These include: utilising staff with experience of engagement, outreach and youth work; building in resources for additional support; peer-to-peer approaches to recruitment etc.

However, there are various barriers and opportunities to involvement. These include: a lack of buy-in from staff for a youth-led approach; low levels of confidence among young people; the capacity of the organisations involved; training opportunities etc.

The evidence review raises the following questions for partnerships to assess how effective their approaches are:

  • What approach is being used: what facilitates involvement e.g. training, incentives, flexibility?
  • Is involvement meaningful?
  • What are the outcomes for those directly involved and wider beneficiaries?
  • How inclusive are the approaches to involvement?
  • Is the scale and scope of opportunities for involvement appropriate?

Underpinning these considerations, is honesty and clarity about the extent of, and limits to, young peoples’ involvement.  A major challenge for the programme will be about the management of expectations.

The evidence review can be found here: http://www.shu.ac.uk/research/cresr/sites/shu.ac.uk/files/tm-evidence-review-young-people.pdf

The literature on involving 18-24 year olds, and specifically NEET groups, is sparse. It is therefore even more important that evidence of good practice and lessons are systematically collected and shared.

Evaluation Blog #3: What Prior Evaluation Evidence tells us about Youth Employment programmes

Programmes to support young people gain in employment are not new. Many have been subject to extensive research and evaluation.  The main finding is that there are no simple or common solutions to youth unemployment. A range of interventions are required, some work better in certain localities than others, some work better with those closest to the labour market than others, and some may only bring benefits in the very long term. A key consideration in designing interventions is to take account of the local labour market contexts and the prevailing strength of the economy – whether it is in recession or close to recession with little job growth, or whether there is a strong upturn with job growth.

This opening paragraph may suggest that there are too many imponderables. However, there is consistent evidence around three areas:

  • the importance of high quality information, advice and guidance
  • effective and locally tailored outreach (to engage those furthest from the labour market) and
  • high quality engagement with employers, including the offering of some form of short term wage subsidy.

What there is here is a mix of supply-side (young people) and demand-side (employer) interventions coming together at a local level.

These issues are explored in more detail in an evidence review we prepared at the design stage of the Talent Match programme. More details can be found here:  http://www.shu.ac.uk/research/cresr/sites/shu.ac.uk/files/tm-evidence-review.pdf Much of this evidence informed the partnership’s delivery plans.

The evidence also points to some agendas which are best addressed nationally or at least at the level of regions or city-regions. Examples here would include the design of high quality progression routes into vocational education and training or initiatives to generate more jobs, such as from the European Structural and Investment Funds or the Regional Growth Fund.

However, there are also gaps in the evidence and this presents an important opportunity to inform future policies. A case in point is the engagement and involvement of young people aged 18-24 in the design and delivery of programmes. My colleague Nadia Bashir undertook an evidence of this area and whilst she found some evidence of good practice principles, often drawing on the literature on youth work, relatively little is known about what works effectively for engaging 18-24 year olds. This will be the subject of a later evidence review.

Evaluation Blog #2: The Talent Match Common Data Framework – the guidance and questionnaires are now online

Talent Match is an ambitious programme.  This includes the ambition of the Big Lottery Fund to collect detailed data systematically on all the young people supported by the programme. With this in mind BIG asked us to set up a Common Data Framework (CDF) as a requirement of the evaluation. Alongside other research undertaken by the Evaluation Team at CRESR (Sheffield Hallam University) and IER (University of Warwick) this will provide us and you with one of the most comprehensive insights into what works in supporting young people furthest from the labour market gain employment.

Talent Match is also an innovative programme. Each of the 21 partnerships designing interventions which are appropriate to the local context and to what young people asked for. As the Common Data Framework reflects this: partnerships are working with different groups of young people, in different ways and at this early stage with some different outcomes.

In future Blogs I will write more about these early lessons from the Common Data Framework data and also provide guidance to analysing the data further at a local level.

However, we recognise that the CDF is challenging and at times onerous for the staff and young people involved in the Programme. A particular challenge is the follow-ups after the baseline questionnaire is complete. We know that young people may engage in the programme at the start and then may no longer need the support or may return to the programme after several months. Whilst the ideal would be to have responses at every time point we know that this is not always possible. What is important is that attempts are made to follow up young people. Only through this will we be able to gain an overall insight into the journey to employment. As you will see from the latest versions of the follow-up questions there are now options for where young people are not contactable.

And finally, the following links are to the key documents you need to understand the CDF: the manual and quick guide; and the baseline and follow-up questionnaires.

Talent Match Common Data Framework
http://www.shu.ac.uk/research/cresr/sites/shu.ac.uk/files/tm-cdf-manual1.pdf

Talent Match Common Data Framework: Quick Guide
http://www.shu.ac.uk/research/cresr/sites/shu.ac.uk/files/tm-cdf-quick-guide.pdf

Talent Match Questionnaire – Baseline
http://www.shu.ac.uk/research/cresr/sites/shu.ac.uk/files/tm-baseline-questionnaire.pdf

Talent Match Questionnaire – Follow-up
http://www.shu.ac.uk/research/cresr/sites/shu.ac.uk/files/tm-followup-questionnaire.pdf

We welcome your continued feedback.

Elizabeth Sanderson and Peter Wells

Evaluation Blog #1: Talent Match Programme Evaluation: What have we discovered so far

Welcome to this first blog from the Talent Match evaluation team at CRESR, Sheffield Hallam University. Over the course of the programme myself and colleagues will be posting blogs reflecting on different aspects of the programme. Some of these will draw on evidence we’ve collected from you as part of the evaluation; some will be from other evaluation evidence; and some will be reflections of policy agendas. We hope these will be of use. Above all we want to use the blogs to stimulate discussion and feedback; whether you agree or disagree we’d welcome your input.

There’s a risk in a first blog to try to cover too much. All I will do here is draw a few reflections:

  • Innovation: the involvement of young people is genuinely innovative and challenging norms of working; not least because it is involving 18-24 year olds a group overlooked in previous involvement work
  • Partnerships took different forms, some built on established relationships whilst others involved organisations new to particular areas.
  • Interventions ranged from therapeutic activities to raise confidence and improve mental health through to wage subsidies. What underpinned most work though was outreach.

There are also many unanswered questions. Will the young people supported get better jobs than they would otherwise? Will these jobs be fulfilling? What other support have projects brought, such as improved confidence or addressing barriers to housing? And what will be left in place after Talent Match?

Feedback from many partnerships is that they are working with many young people who face multiple barriers in re-entering the labour market. Analysis of the Common Data Framework will allow us to understand these challenges in more detail and to an extent which has not been possible before.

All the research publications from the evaluation can be found here. http://www.shu.ac.uk/research/cresr/ourexpertise/talent-match-evaluation-and-learning-contract

Peter Wells