NSS Decoded

The results of the National Student Survey for 2016 were released yesterday, and there’s much jubilation at those institutions which have done well, accompanied by a judicious silence from others. All final year undergraduate students are invited to complete the study, which asks a range of questions about their experience at university and their learning. It also includes one question to sum up their overall experience, and it is this question which drives many published rankings. Students are asked to say how much they agree or disagree with the following statement:

 “Overall, I am satisfied with the quality of the course”

The possible answers are “Definitely agree”, “Mostly agree”, “Neither agree nor disagree”, “Mostly disagree” and “Definitely disagree”. The outcomes are reported as the sum of the percentage of respondents who definitely or mostly agree, this being reported as the overall satisfaction.

I’ve posted before about how this oversimplifies things. For example, the following scores from two institutions would get the same overall satisfaction rating, but how different they are:

	Definitely disagree	Mostly disagree	Neither agree nor disagree	Mostly agree	Definitely agree
University A	20%	0%	0%	50%	30%
University B	0%	0%	20%	20%	60%

Both A and B have an overall satisfaction of 80%. But 20% of those who go to A are very unsatisfied; and of those who are satisfied, three in four at B are really satisfied, compared to only three in eight at A. University B definitely has greater student satisfaction than University A, but the simple overall measure doesn’t show it.

But there is value in simplicity. I prefer a different measure, which also has the virtue of reporting one number per institution, but which gives more nuance: the Grade Point Average (GPA). This gives a numerical weight for each different category of answer (that is, 1 for “Definitely disagree”, 2 for “Mostly disagree” and so on, up to 5 for “Definitely agree”) and multiplies these by the percentage reporting the relevant response.

So, using the examples above, University A scores 20% of 1 (Definitely disagree) plus 50% of 4 (Mostly agree) plus 30% of 5 (Definitely agree), which gives a GPA of 0.2+2.0+1.5=3.7.

University B scores 20% of 3 (Neither agree nor disagree) plus 20% of 4 (Mostly agree) plus 60% of 5 (Definitely agree), giving a GPA of 0.6+0.8+3.0=4.4.

You can see that with GPA it’s easier to compare the two universities: both universities score above 3, showing that overall students are positive; and B’s score of 4.4 is more than A’s score of 3.7, which reflects the more positive response of students at B.

So what happens if we use GPA to rank universities in 2016? I’m glad you asked, as I’ve done a comparison. The table shows UK universities ranked by the NSS method and by GPA. I’ve excluded from the list FE Colleges and alternative providers, in line with other NSS rankings you might see, so we have 159 institutions. (There's a bit more blog after the table, so please do keep scrolling down!)

NSS method			GPA method
University	%	Rank	GPA	University
The University of Buckingham	97%	1	4.72	Brighton and Sussex Medical School
Brighton and Sussex Medical School	95%	2	4.70	St Mary's University College
St Mary's University College	95%	3	4.61	University of St Andrews
Courtauld Institute of Art	94%	4	4.59	The University of Buckingham
University of Keele	94%	5	4.51	University of Keele
University of St Andrews	94%	6	4.49	Liverpool Hope University
Aberystwyth University	92%	7	4.48	Rose Bruford
Bishop Grosseteste University	92%	8	4.43	Central School of Speech and Drama
Harper Adams University	92%	9	4.42	The Royal Veterinary College
Liverpool Hope University	92%	10	4.41	University of Dundee
Coventry University	91%	11	4.40	Harper Adams University
Heythrop College	91%	12	4.40	Leeds College of Art
Stranmillis University College	91%	13	4.40	University of Lincoln
The Royal Veterinary College	91%	14	4.39	Conservatoire for Dance and Drama
The University of East Anglia	91%	15	4.39	University of Exeter
The University of Lancaster	91%	16	4.39	University of Oxford
University of Dundee	91%	17	4.38	Coventry University
University of Exeter	91%	18	4.37	Aberystwyth University
University of Lincoln	91%	19	4.37	Bangor University
Bangor University	90%	20	4.37	Loughborough University
Medway School of Pharmacy	90%	21	4.37	The University of East Anglia
Queen's University of Belfast	90%	22	4.37	The University of Essex
Rose Bruford	90%	23	4.37	The University of Lancaster
Swansea University	90%	24	4.37	University of Cambridge
Conservatoire for Dance and Drama	90%	25	4.36	The University of Bath
Central School of Speech and Drama	90%	26	4.35	The Royal Academy of Music
The University of Bath	90%	27	4.35	The University of Kent
The University of Essex	90%	28	4.35	The University of Surrey
The University of Kent	90%	29	4.34	Medway School of Pharmacy
The University of Leeds	90%	30	4.34	University of York
The University of Surrey	90%	31	4.33	Swansea University
University of Cambridge	90%	32	4.33	The Open University
University of Newcastle upon Tyne	90%	33	4.33	University of Newcastle upon Tyne
University of Oxford	90%	34	4.32	Queen's University of Belfast
Heriot-Watt University	89%	35	4.32	University of Portsmouth
Loughborough University	89%	36	4.32	Courtauld Institute of Art
Nottingham Trent University	89%	37	4.31	Stranmillis University College
Royal Holloway, University of London	89%	38	4.31	The University of Chichester
The Open University	89%	39	4.31	Bishop Grosseteste University
The Royal Academy of Music	89%	40	4.31	The University of Nottingham
The University of Birmingham	89%	41	4.31	University of Durham
The University of Nottingham	89%	42	4.30	The University of Sheffield
The University of Sheffield	89%	43	4.30	The University of Warwick
University of Durham	89%	44	4.30	University of the West of England
University of Glasgow	89%	45	4.29	Heriot-Watt University
University of Portsmouth	89%	46	4.29	Nottingham Trent University
University of St Mark & St John	89%	47	4.29	The University of Huddersfield
University of Stirling	89%	48	4.29	University of St Mark & St John
University of Sussex	89%	49	4.29	The University of Leeds
University of York	89%	50	4.28	Royal Welsh College of Music and Drama
Aston University	88%	51	4.27	The University of Leicester
Leeds College of Art	88%	52	4.27	University of Aberdeen
The Royal Agricultural University	88%	53	4.27	Birkbeck College
SOAS	88%	54	4.27	Heythrop College
The University of Leicester	88%	55	4.27	Liverpool Institute for Performing Arts
The University of Warwick	88%	56	4.27	SOAS
University of Aberdeen	88%	57	4.27	University of Glasgow
University of Abertay Dundee	88%	58	4.27	University of Ulster
University of Ulster	88%	59	4.26	Aston University
Birkbeck College	87%	60	4.26	Royal Holloway, University of London
Cardiff University	87%	61	4.26	University of Abertay Dundee
Edge Hill University	87%	62	4.26	University of Sussex
Oxford Brookes University	87%	63	4.26	University of the West of Scotland
Teesside University	87%	64	4.25	Edge Hill University
Liverpool Institute for Performing Arts	87%	65	4.25	Royal College of Music
The University of Chichester	87%	66	4.24	Falmouth University
The University of Huddersfield	87%	67	4.24	Royal Northern College of Music
The University of Liverpool	87%	68	4.24	University of Northumbria at Newcastle
The University of Reading	87%	69	4.24	Wrexham Glyndwr University
University of Chester	87%	70	4.23	The University of Reading
University of Strathclyde	87%	71	4.23	University College Birmingham
University of the West of England	87%	72	4.23	University of Gloucestershire
University of the West of Scotland	87%	73	4.22	Cardiff University
University of Winchester	87%	74	4.22	The City University
Anglia Ruskin University	86%	75	4.22	The University of Birmingham
Queen Margaret University, Edinburgh	86%	76	4.22	University of Southampton
Royal College of Music	86%	77	4.22	University of Strathclyde
Royal Welsh College of Music and Drama	86%	78	4.21	Anglia Ruskin University
The University of Hull	86%	79	4.21	Plymouth College of Art
University College Birmingham	86%	80	4.21	The University of Hull
University of Bristol	86%	81	4.21	University of Winchester
University of Derby	86%	82	4.20	University of Chester
University of Gloucestershire	86%	83	4.20	University of Stirling
University of Northumbria at Newcastle	86%	84	4.20	University of Wales Trinity Saint David
University of Southampton	86%	85	4.19	University of Derby
De Montfort University	85%	86	4.19	University of Bristol
Falmouth University	85%	87	4.18	Liverpool John Moores University
Guildhall School of Music & Drama	85%	88	4.18	St Mary's University, Twickenham
Hull and York Medical School	85%	89	4.18	Staffordshire University
Manchester Metropolitan University	85%	90	4.18	The Royal Agricultural University
Newman University	85%	91	4.18	The University of Bolton
St Mary's University, Twickenham	85%	92	4.18	The University of West London
Staffordshire University	85%	93	4.18	University for the Creative Arts
The City University	85%	94	4.18	University of Plymouth
The Robert Gordon University	85%	95	4.17	Oxford Brookes University
The University of Manchester	85%	96	4.17	Queen Margaret University, Edinburgh
University for the Creative Arts	85%	97	4.17	Queen Mary University of London
University of Plymouth	85%	98	4.17	Teesside University
University of Worcester	85%	99	4.17	University of Bedfordshire
Wrexham Glyndwr University	85%	100	4.17	University of East London
Bath Spa University	84%	101	4.16	De Montfort University
Birmingham City University	84%	102	4.16	The University of Northampton
Canterbury Christ Church University	84%	103	4.16	The University of Salford
Edinburgh Napier University	84%	104	4.16	University of Worcester
Leeds Beckett University	84%	105	4.15	Bath Spa University
Liverpool John Moores University	84%	106	4.15	Brunel University London
Queen Mary University of London	84%	107	4.15	Edinburgh Napier University
Sheffield Hallam University	84%	108	4.15	The Robert Gordon University
The University of Northampton	84%	109	4.14	Guildhall School of Music & Drama
The University of West London	84%	110	4.14	Imperial College London
The University of Wolverhampton	84%	111	4.14	Roehampton University
University College London	84%	112	4.14	Royal Conservatoire of Scotland
University of Bedfordshire	84%	113	4.14	The Arts University Bournemouth
University of Wales Trinity Saint David	84%	114	4.14	Middlesex University
Brunel University London	83%	115	4.13	London Metropolitan University
Cardiff Metropolitan University	83%	116	4.13	Manchester Metropolitan University
Glasgow Caledonian University	83%	117	4.13	The University of Liverpool
Goldsmiths' College	83%	118	4.13	York St John University
Imperial College London	83%	119	4.12	Cardiff Metropolitan University
Middlesex University	83%	120	4.12	Leeds Beckett University
Royal Conservatoire of Scotland	83%	121	4.12	The University of Wolverhampton
The University of Bradford	83%	122	4.11	The University of Manchester
University of Central Lancashire	83%	123	4.11	Goldsmiths' College
University of East London	83%	124	4.11	Newman University
University of Greenwich	83%	125	4.11	The University of Bradford
University of Sunderland	83%	126	4.11	University College London
York St John University	83%	127	4.11	University of Central Lancashire
Bournemouth University	82%	128	4.10	Buckinghamshire New University
Buckinghamshire New University	82%	129	4.10	Glasgow Caledonian University
King's College London	82%	130	4.09	Birmingham City University
Roehampton University	82%	131	4.09	Canterbury Christ Church University
Royal Northern College of Music	82%	132	4.09	University of Greenwich
The Arts University Bournemouth	82%	133	4.08	Hull and York Medical School
The University of Bolton	82%	134	4.08	Norwich University of the Arts
The University of Salford	82%	135	4.08	Sheffield Hallam University
University of Hertfordshire	82%	136	4.08	The University of Westminster
Leeds Trinity University	81%	137	4.08	University of Hertfordshire
London Metropolitan University	81%	138	4.07	King's College London
London South Bank University	81%	139	4.07	London South Bank University
Plymouth College of Art	81%	140	4.06	Southampton Solent University
Southampton Solent University	81%	141	4.05	Kingston University
The University of Westminster	81%	142	4.05	Leeds Trinity University
University of the Highlands and Islands	81%	143	4.05	University of Brighton
Kingston University	80%	144	4.04	University of Suffolk
St. George's, University of London	80%	145	4.03	Bournemouth University
University of Brighton	80%	146	4.03	University of Sunderland
University of Cumbria	80%	147	4.02	University of South Wales
University of Edinburgh	80%	148	4.02	University of the Highlands and Islands
University of South Wales	80%	149	4.01	University of Cumbria
University of Suffolk	80%	150	4.01	University of Edinburgh
Norwich University of the Arts	79%	151	3.98	St. George's, University of London
SRUC	76%	152	3.94	Trinity Laban
University of London Institute in Paris	76%	153	3.88	SRUC
LSE	75%	154	3.85	LSE
Glasgow School of Art.	74%	155	3.84	Glasgow School of Art.
Trinity Laban	72%	156	3.82	University of the Arts, London
University of the Arts, London	72%	157	3.80	University of London Institute in Paris
Ravensbourne	71%	158	3.73	Ravensbourne
Writtle College	68%	159	3.71	Writtle College

Made it this far? Well done!

Although many institutions stay in roughly the same place in the rankings, there are some big differences in both directions: Stirling, for instance, ranks 48^th by the NSS satisfaction measure, but 83^rd on GPA. And Bolton, for another example, ranks 134^th on the NSS measure, but 91^st using GPA. So the GPA measure is more revealing and richer as a single score than the satisfaction percentage alone.

GPA isn’t a perfect measure, of course. There’s an arbitrariness about assigning a consistent numerical difference between the five points on the scale: we don’t know how much individual respondents would weight the difference between the answers. And the simple percentage satisfaction score is more intuitive, and links more obviously directly to the raw data. This makes it easier to use.

So what’s the moral? Simply this: if you’re going to use the NSS to compare universities, look beyond the simple satisfaction measure. The data, publicly available, holds a wealth of information which you can use to better understand how universities are perceived by their students.

Caveat emptor!

Where does 2.8% fee inflation come from?

So we know that the government intend – certainly for English universities – to permit undergraduate fees for Home/EU students to grow to £9,250 for new and continuing students in 2017-18. This was set out in the BIS report ‘Teaching Excellence Framework: Provisional list of eligible providers – Year One’ published on 7 July 2016 and confirmed by Jo Johnson, Minister for Universities and Science, in a written statement to parliament on 21 July 2016:

For all new students and eligible continuing students who started their full-time courses on or after 1 September 2012 and are undertaking courses at publicly funded higher education providers that have achieved a TEF rating of Meets Expectations, maximum tuition fee caps will be increased by forecast inflation (2.8%) in 2017/18. 

The TEF rating relates to the new regime set out in the Higher Education and Research Bill, under which providers who meet certain standards of teaching excellence will be allowed to charge higher fees. The Bill is midway through its parliamentary journey, and my expectation is that at some point it will become a matter of controversy. But for now I want to focus on a different question. What is the basis for the forecast of 2.8% inflation in 2017-18? 

The Bank of England

The Bank of England is responsible for managing inflation in the UK, and in its most recent inflation report (table 5B on page 31) forecasts CPI of 1.5% in the second quarter 2017 (ie April-June 2017) and 2.0% in the second quarter 2018 (ie April-June 2018). The Bank’s model gives probability estimates for inflation differing from this forecast. For Q2 2017 the Bank places a probability of 10% that the actual rate of CPI is between 2.5% and 3.0%; for Q2 2018 the equivalent probability is 12%. (These data come from the excel file available to download from the Bank of England). So we can see that the Bank of England is not forecasting inflation of 2.8% for 2017-18.

The Bank also reports other forecasts of inflation (page 44 of the report), from over twenty other forecasters. The average of these was 1.6% for Q2 2017 and 2.0% for Q2 2018 – not a great deal different from the Bank of England’s own forecast. The key point – other forecasters are not forecasting inflation of 2.8% in 2017-18.

Is the government separately forecasting this rate of inflation? The Office for Budgetary Responsibility publishes an ‘Economic and Fiscal outlook’, the most recent of which was March 2016 (to co-incide with the budget). This reports CPI forecasts of 1.6% for 2017 and 2.0% for 2018 (page 12, table 1.1). So, the government does not forecast inflation of 2.8% for 2017-18.

This is a bit of a puzzle. Is the government using a forecast other than CPI? I don’t think that any other policy area has a separate calculation and forecast of inflation. Universities UK maintained an inflation series – the Higher Education Pay and Prices Index – but this was discontinued a few years ago. Most of the (positive) difference in HE inflation related to pay costs, and it would seem odd for the government to publicly acknowledge that a sector was likely to have higher pay than inflation. So I think we can discount the idea that the government has a separate HE cost inflator.

This leaves another possibility. Perhaps the amount of the increase - £250 – was agreed on, and then the inflation forecast adjusted to match. Universities have in the past been effective lobbyists of government, and both Universities UK and the mission groups are well connected. £250 is fractionally less than 2.8% of £9,000. So maybe that’s where the 2.8% forecast came from.

Is there another explanation? I’d like there to be one, and perhaps a more learned reader can supply it. Until then, I think I’ve got a bit more cynical about HE policy-making.

Widening participation - what do the data really say?

Earlier this month HEFCE published a monitoring report on the Student Opportunity Allocation and National Scholarship Programme for 2014-15, which contained some fascinating data on spend on widening participation by English universities and HEIs. Included was a table and graph showing how annual spend on WP had increased between 2010-11 and 2014-15 by over £150m: a good news story, surely.

Here’s the data that HEFCE presented: I’ve put it in a table rather than the graph on page 11 of the report, to make it easier to compare. It’s separated into categories; these have changed over time, with some categories added, and a miscellaneous category lost – they are the cells in the table with nothing in them. Note also that the early years’ don’t add up, and by more than just rounding errors. Consistent data collection on WP was in its infancy at this point, so some errors aren’t too surprising.

	2010-11	2011-12	2012-13	2013-14	2014-15
Outreach work with Schools and young people	82.7	67.2	108.1	122.2	124.7
Outreach work with communities and adults	31.2	28.9	32.3	34.4	35.5
Outreach work with disabled students			4.2	5.7	6.3
Strategic partnerships with schools					8.1
Support for current students (academic and pastoral)	435.2	444.2	425.1	434.2	447.0
Support for disabled students	40.5	49.9	47.5	48.4	55.6
Support for progression into employment or pg study	17.1	19	40.9	59.2	68.9
Support for progression of disabled students				4.9	5.2
WP staffing and administration	78.8	70.5	84.1	93.4	90.8
Other	2.9	0.9	0.8
Total	690.7	681.6	743.0	802.5	842.2

So it looks like, across the board, universities are spending more on widening participation. Hooray!

But hang on a minute, what about inflation? The data HEFCE report look like they are cash sums. What is it in real terms?

I calculated inflation to match University reporting years using ONS CPI data. Although inflation isn’t much now, it was higher in this period – 4.0% in 2010-11, 3.6% the following year, for instance. When the data is presented at 2010-11 constant prices, this is what the table looks like:

	2010-11	2011-12	2012-13	2013-14	2014-15
Outreach work with Schools and young people	82.7	64.9	101.6	112.7	114.6
Outreach work with communities and adults	31.2	27.9	30.4	31.7	32.6
Outreach work with disabled students			3.9	5.3	5.8
Strategic partnerships with schools					7.4
Support for current students (academic and pastoral)	435.2	428.8	399.5	400.5	410.6
Support for disabled students	40.5	48.2	44.6	44.6	51.1
Support for progression into employment or pg study	17.1	18.3	38.4	54.6	63.3
Support for progression of disabled students				4.5	4.8
WP staffing and administration	78.8	68.1	79.0	86.1	83.4
Other	2.9	0.9	0.8
Total	690.7	657.9	698.3	740.2	773.7

Now some real patterns start to emerge.

Three of the categories have seen real increases: outreach work with schools and young people; support for disabled students; and support for progression from HE into employment or postgraduate study.  Spend on outreach work with communities and adults, and WP staffing costs, have stayed pretty constant. And spend on academic and pastoral support for current students has fallen.

It seems to me that spending on support for current students – academic and pastoral – is also related to the number of students. Over the period the HE sector has grown. It’s possible to identify the total number of UK-domiciled undergraduates at English universities, so that spend per student can be calculated.

Support for current students	2010-11	2011-12	2012-13	2013-14	2014-15
Spend in 2010-11 prices (£m)	£435.2m	£428.8m	£399.5m	£400.5m	£410.6m
UK domiciled students	873,910	919,990	915,710	928,365	929,200
Spend per student, 2010-11 prices (£)	£498	£483	£464	£468	£481

The data shows that per student, spend on academic and pastoral support has reduced from £498 in 2010-11 to £481 in 2014-15.

So in conclusion, we see that rather than spend increasing in all categories, there’s actually been a focus on outreach with schools, support for students with disabilities and with post-HE progression. These are things which make a big difference to life-chances, and that’s what it’s all about.

Does this mean that HEFCE wasn’t being accurate?  I think that would be going too far, but it’s always worth looking at data carefully to see what it really means, and what it really says. If spend per student on actual support is really declining, not increasing, isn’t that worth knowing? 

Brexit - bad news for conservatoires

A lot of people within the HE sector are shocked by the Brexit vote, and the impact is beginning to be felt. Individually, people on both sides of the issue will take time to adjust, but it’s important for institutional leaders to get quickly beyond the shock and think about how Brexit will affect them.

One obvious concern relates to students from other EU countries. Presently, such students are able to access student loans company funding, and are treated the same as students in the home nation for fees purposes. (This latter point gives some curious results: Scottish universities charge zero fees to Scottish domiciled students, and £9k for students from England, Wales and Northern Ireland. Students from other EU countries are treated the same as Scottish students, paying zero fees.)

There’s obviously big uncertainty about what happens in the future. In the short term, current EU students, and those who start in 2016-17, will continue on the same terms and conditions. Beyond this, there isn’t certainty yet. And so universities need to start contingency planning. (It’s all in the negotiations. An EEA-type outcome may include special dispensation for students, but that is simply an unknown.)

At some point it seems possible that EU students will be treated the same as any other overseas student. If that were to happen, we could expect EU student numbers to decline. Universities will need to respond, and there are fundamentally two choices – replace the ‘lost’ EU students with others, or admit fewer students. So what is the scale of the challenge, and who is most affected?

EU undergraduates comprise over 5% of the current (2014-15 HESA data) Home/EU undergraduate population. Scotland has the highest proportion – perhaps unsurprisingly as EU students pay no tuition fees – but London is not far behind.

It is reasonable to assume that some universities at least will try to make up the shortfall with UK students, and that means greater competition. Universities with stronger recruitment profiles will admit students who may have gone elsewhere, and universities which recruit less strongly will have to admit students they otherwise would not have, if they wish to remain the same size. This may not always be possible. In London this competition will possibly be intense – the number of EU students in the region - almost 17000 - is larger than most universities’ total student intakes.

It’s also useful to look at the impact on individual universities. In absolute terms, the ten universities with the most EU undergraduates are:

The three Scottish universities are unsurprising, and the differences in Scottish university finance probably mean that in purely financial terms the loss will be less significant for them than if they were in England. But the English universities in the list vary in character, some being very strong recruiters, others less so.

In proportionate terms, another issues arises. Here’s the ten institutions with the greatest percentage reduction, if EU students are lost:

This list includes the four London conservatoires – specialist music institutions – and the broader performing arts conservatoire which is the Conservatoire for Dance and Drama. These institutions don’t have a normal recruitment pattern. To gain admittance you have to already be a very capable musical performer. The number of lost students is greater than the total intake at two of these institutions. It is hard to see how the conservatoires could remain at their current size without EU students. And they probably aren't big enough to easily survive such large changes in tuition fee income.

Here is, I suspect, the first unanticipated and unintended consequence of Brexit: there’ll be fewer conservatoires. To avoid this, increased governmental support will be needed. Which won’t be straightforward in the more straitened times we face.