A fine romance

An interesting item on the BBC website today – the University of Sheffield has abolished library fines for students.

The BBC rightly identifies this as stemming in part from the OFT’s advice against academic penalties for financial debts. And it also speaks to the growing recognition that students are customers (see also Goldsmiths’ decision to have a student member of its Remuneration Committee.)

But there’s a small sting in the tail – students have to bring the book back when someone else wants it, and won’t be allowed to borrow any more books until they do. So it isn’t quite a free-for-all, and the students will still need to learn to share. Although students in plural may be king (or queen), a student in the singular still needs to do what they’re told. The student contract isn’t quite dead yet.

I’m filing this under ‘straws in the wind’. There’s clearly a changing relationship between a university and its students, and some benign changes like this will occur, as well as some which may seem more threatening.

Key Performance Indicators

Look on the webpages of almost every university, tucked away sometimes in the ‘About Us’ pages, and you’ll find a section on KPIs – Key Performance Indicators. I thought it might be useful to write a bit about what they are, and why they matter. In this post I’ll also go into a bit of detail about one common KPI – staff costs as a percentage of income.

A performance indicator simply means a measure of how you’re doing. So, if I’m typing, words per minute might be a performance measure. But so might typos per sentence. Or proportion of screen-breaks properly taken. Performance indicators are quite easy to come up with (although, as a game, making up performance indicators does get pretty tedious after a while).

Now think about all the things that go on in a university (or any other organisation, for that matter), and that can be measured, and you’ll see that a full list of university performance indicators would be huge. And so Key Performance Indicators are those chosen for particular monitoring.

Chosen is an important point here. There’s no absolute list of what matters to an organisation: it is a judgment about strategy and circumstances. There are KPIs which are measured by university regulators (funding councils, research councils etc), and they do tend to get measured by universities, but it is a choice that universities make. Sometimes a pretty obvious choice, but a choice nonetheless.

The choice is made by governing bodies, on the advice of the VC and colleagues in the senior management team. And the choice tells you about what’s important to the University. If a university is trying to improve its student experience, it might have staff:student ratio as a KPI. Or proportion of space of the highest standard. Or NSS results. But if it has a financial focus in its strategy (for investment, or to ensure sustainability) it might choose income per student (or cost per student).

The other jargon term is metric. That’s the actual value associated with a Key Performance Indicator. So for my words-per-minute typing KPI, the metric at the moment is about twenty. And the typos-per-sentence is about two. Could do better! And that’s the point – there’s often a target (or a range) associated with a KPI and metric. I’d like to type with fewer errors and more words per minute. So using a KPI I can measure my progress.

So that’s the theory. Let’s take a common university KPI – proportion of income spent on staff – and look at that in more detail.

The definition is straightforward. Take a university’s total income and its total spend on staff, both of which are in the annual financial statement, and divide the latter by the former. So if my university’s income is £100m per year and staff costs are £55m per year, then the proportion of income spent on staff KPI has a value of 55/100, or 55%.

Here’s two charts which show the KPI. They’re both drawn from 2012-13 HESA data. The first is a scatter plot, where each university is represented by a single dot.

This shows that for most universities, staff costs are in the range of forty-something to sixty-something percent of income, with an average around fifty-something percent. It also shows there are some outliers – some very large institutions, with turnovers of over one billion pounds (and I do hope that the Vice-Chancellors report that to their governing bodies in the style of Dr Evil); and some smaller institutions with much smaller (3% and 14%) spend on staff. These latter are instructive: one is a small consortium institution, and doesn’t directly employ many staff at all; the other is a university with a huge geographical spread and lots of costs associated with teaching over this area. It shows why the choice of KPIs depends on the specific circumstances of the institution. This probably isn’t a very useful KPI for those particular institutions

Now let’s see a different chart, using the same data. This is a bar chart where the institutions are arranged from left to right in ascending order of turnover. That is, the bar furthest to the left is the institution with the smallest turnover, the bar furthest to the right is the institution with the largest turnover. The height of each bar shows the proportion of income spent on staff.

This is more like a range of mountains – a few really high peaks and a few deep valleys, but most are around the 45%-55% range. What’s interesting here, I think, is that there’s no obvious economy of scale – larger institutions don’t seem to have a significantly lower proportionate spend on staff. This is counterintuitive.

So why does it matter? Staff costs are unlike other costs for a university. Firstly, they are long term. Once you’ve employed someone then you need to keep paying them, month after month, and to stop paying them involves effort and more expense. Secondly, they increase on a regular basis. Pay increases incrementally for many, simply by the passage of time as you go up the pay scale; there are pay increases every year through negotiation with Trades Unions; and on-costs – pension contributions and national insurance contributions – go up not down, particularly at the moment. A pound spend on employing someone this year becomes more like £1.05 next year, and so on. To employ someone is to make a commitment. 

And if you’re managing an institution, you need to know about this. If you have to find money each year for a rising staff bill, then you have fewer opportunities for other things – equipment, buildings, training costs, IT, library books, student bursaries and scholarships. So an institution which can manage its staff costs has more opportunities. And when there’s a large increase – a hike in pension contributions, or a big pay award (they did used to happen!) then institutions with a large staff cost as a proportion of income get hit hardest.

There can’t be an absolute target, of course. Some subjects cost more to teach than others – for instance clinical subjects. Some subjects need a lot of equipment, which keeps the proportion of spend on staff low because lots of money is spent on kit. And institutions make choices about how they operate which make an impact – for instance, having small tutorial groups.

There’s no moral to this tale: if you have a high proportion of staff spend, and know why, and it fits with your strategy, maybe that’s fine. And too low a proportionate spend and maybe your institution isn’t doing things as well as it might – universities are people businesses. But the one number does help you understand quite a lot about an institution’s plans and how sustainable it is (that is, how resilient to shocks, or how much scope for investment it has). And if you’re accountable for the institution, or deciding who to give more resource to, those are good things to know.

Counting them out and counting them back in again

One of my current clients is preparing for a graduation ceremony, which got me thinking. It’s been my view for some time now that many features of academic administration in universities can be explained by graduation: that is, they all work towards the task of making sure that the right student walks across stage with the right name called out to get the right degree certificate, and dressed in the right robes for a good photo.

Two proud graduates ...

Let’s take this in parts.

Firstly, the right student. So that’s about recruitment and admissions, that’s about enrolment, module selection and assessment, and that’s about keeping tags through a student record system on who a person is and whether they’ve passed the exams.

The right name called out. Well, that is also about a student record system, but it’s about knowing the student – which name do they actually use; how do you pronounce it; how do you make sure that the person reading the name out sounds like they’ve actually heard of them and aren’t surprised to see them there. So it ties in with things done to make sure that a student actually feels at home in the university and that someone knows them. Often the departmental office...

The right degree certificate. This ties into programme approval; any professional or statutory accreditation; making sure that the programme and the modules are all set up on the record system so that the transcript is easy to produce; and making sure that the credentials of the degree are all correct. Although a degree is really about the education, the validity of the certificate and transcript are really important in the instrumental uses of an education which students do care about.

The right gown and photo. This is about making the day memorable, so that the student leaves with fond memories; so that their family and friends get a good impression; so that there’s a tangible reminder to show proud grandparents; and so that the student (no, the graduate!) feels a lasting connection, and will stay in touch. With the alumni office. So that they might come back to give talks about their career, mentor current students, give money, leave legacies, and also say good things about the university whenever they get the chance.

I think graduations are great, as they’re one of the few days when everybody in the room is glad to be there. They can be wonderfully uplifting, with sudden displays of love and happiness and all manner of human frailty revealed. They are also, more prosaically, a significant milestone in the journey of a student.

And perhaps that’s the point I’m really getting at here: all of the different processes and functions in a university add together to make a student journey, which is a big part of their life story. It’s pretty important stuff, and seeing and recognising the joins and the connections makes it a better journey. So go to graduation and enjoy it; help out in marshalling the procession. But also think about all the steps necessary to get the right student the right degree with the right name read out and the right photo. And think what could be done to make it better.

QAA? Armageddon outta here!

Yesterday’s announcement – on the HEFCE webpages but on behalf of HEFCW and DELNI and in parallel with the Scottish Funding Council – has caused more than a few ripples of excitement.

What’s the news? A request for feedback from the sector, to inform the specification of a tender for quality assurance for higher education. If that doesn’t sound significant, let me tell you that it is. First, some background.

The 1992 Further and Higher Education Act provides (para 70) that the Funding Councils “shall … secure that provision is made for assessing the quality of education provided in institutions for whose activities they provide, or are considering providing, financial support …”. Each Funding Council has a Quality Assurance Committee, comprising people with sector experience, to advise on how to undertake this work.

If my memory serves me right*, in the early stages HEFCE did some assessment of standards work in-house (the Academic Audit Unit), and a sector body – the Higher Education Quality Council (HEQC) – developed methodology for assessing and assuring quality at a subject level. And in 1997 the QAA was established to bring it all together.

An opportunity for a new logo?

And so the consultation and tender exercise is giving notice that the status quo is up for renegotiation. No doubt with pressure from some such as the Russell Group for a ‘risk-based’ approach (‘leave us alone, the metrics show that we’re fine!’) and, with an equal lack of doubt political pressure from different quarters to address perceived issues such as contact hours and value for money

And so this is a fight about university autonomy and about marketization. Part of the rationale for establishing HEQC and QAA (both bodies owned by the HE sector, like UCAS and HESA) was that if the sector didn’t do something, government would impose an OFSTED style inspectorate. And since one of the crowning glories of UK higher education is that our academic standards are high, this would be a problem. Universities need autonomy to set and maintain these standards, so the argument goes.

And marketization? Well, applicants to universities, almost by definition, don’t know much about the content or value of the programme they’re seeking to join. In the OFT’s view, they are much less sophisticated as consumers than the universities with which they are making a contract. So an assurance regime which is more like consumer protection – guarding against rogue traders – might be considered appropriate. There’s a lot of public money and private individual debt at stake, so these aren’t concerns which can be dismissed flippantly.

There’ll be a lot of debate and discussion. The Wonkhe post by Mark Leach is a good resource with links to various statements and also in the comments. Which is where, by the way, the Armageddon word came from: #QAmageddon being the hashtag of choice …

It pays to read the legislation carefully: “assess the quality of education.” Who defines quality and its indicators, that’s the question. And the outcomes will matter for the sector.

*Edit 9 October. My memory didn't serve me right! Academic Audit Unit became HEQC; HEFCE continued with TQA until QAA took over. Thanks to Mike Ratcliffe @Mike_Rat for the correction. We need the equivalent of rock family trees ...