On league tables and what matters in universities

They are selling replica Leicester City shirts on market stalls in Beijing. They weren’t selling them a year ago. That’s what topping the Premier League does. It’s almost certain that Leicester’s two universities will now see a spike in international, and probably UK, applications: everyone loves a winner.

Leicester City shirts for sale

Leicester City shirts for sale (photo: Joe Giddens/PA Wire)

The Guardian published its 2017 University Guide two weeks ago. The data underlying this guide are a bit more complex, and perhaps less reliable, than the data underlying the Premier League table. My daughter, an avid supporter and season ticket holder at Norwich City, is pretty clear that opaque refereeing decisions explain their relegation this season. So it’s worth digging into the Guardian league table a little.

The Guardian table is made up of a series of indicators: some measure inputs such as spending on facilities or staff-student ratios, some measure performance including entry tariff, research intensity and student satisfaction, and some measure outcomes, including degree completion rates and employment levels after graduation.

To add to the complexity, these measures are compiled across different years. For the 2017 Guardian tables, the entry tariff is for 2014/15 undergraduate entrants but the value added score relates to 2012/13 entrants. These indicators are given different weights: entry tariff and value added scores count for more than some other scores.

If we were to apply this approach to football league tables, we’d take some results from 2015/16, some from 2014/15, some from 2012/13, and then give different weight to results against different teams. I suspect Norwich City would still have been relegated.

All league tables are imperfect. They all reduce higher education to simple measures, and apply arbitrary weightings. And there are different league tables which give different results.

Sheffield Hallam did not come well out of the Guardian 2017 guide. It fell twelve places from last year – from 74th to 86th. This is deeply frustrating; as I have written before, the University’s league table position is far from being a reflection of its quality. Of course, there are other league tables using different measures.

In the Complete University Guide, published earlier this year, Hallam maintained its 2015 position at 72nd. In the Times Higher Education student experience league table, Hallam is placed 46th. There is the 2015 People and Planet league table (essentially a measure of green-ness), where Hallam came fifteenth. Even so, we need to be honest about some of the issues that emerge from league tables because improvements in these areas will improve what we offer our students.

Let’s be clear: a university of this quality should not be ranked in the eighties. But even in the Guardian table there is mixed news. On graduate employment data, Hallam comes 85th: that is not something we should be content with. However, if you focus solely on student satisfaction scores (satisfaction with course, teaching and feedback), Hallam’s position rises to about 50th – that’s obviously better. And there is a key message in the individual subject tables: if we brought all subjects up to the level of our best performing subjects, our position would rise by about forty places.

These are things which matter – to us, and to our students – and that, not league tables, make them worth addressing. And they are going to become more important. The Teaching Excellence Framework, set out in the White Paper and the Higher Education Bill now going through Parliament, will make use of a range of metrics, including destination data, intensity measures and student survey results. The TEF will have funding and reputational consequences for universities, and, importantly, the TEF will produce a single institutional grade – so it will be all the more important to be able to address variations across the University.

I have a very simple rubric about external pressures: do they help to do things which are in the interests of students? If so, then you have more reasons for doing them. If they don’t, then you should focus on the things which really matter.

Improving the student experience, embedding teaching quality across the University, improving our students’ transitions to employment – these are core to Hallam’s mission and, by and large, they are within our capacity to influence. The league tables tell us that we do know how to excel: some parts of the University do stunningly well. We should, and will, do better at these things, not because they will help us rise in the league tables, but because they matter to us as a university.

The world around us is changing quickly. The new Higher Education Bill is just one example of that. As we move into a more competitive environment, it’s all the more important that we review and focus our strategy as a university, being much clearer about what we want to achieve and – just as significant – how we are going to organise ourselves to achieve it.

Leave a Reply

Your email address will not be published. Required fields are marked *