NSS Decoded

The results of the National Student Survey for 2016 were released yesterday, and there’s much jubilation at those institutions which have done well, accompanied by a judicious silence from others. All final year undergraduate students are invited to complete the study, which asks a range of questions about their experience at university and their learning. It also includes one question to sum up their overall experience, and it is this question which drives many published rankings. Students are asked to say how much they agree or disagree with the following statement:

 “Overall, I am satisfied with the quality of the course”

The possible answers are “Definitely agree”, “Mostly agree”, “Neither agree nor disagree”, “Mostly disagree” and “Definitely disagree”. The outcomes are reported as the sum of the percentage of respondents who definitely or mostly agree, this being reported as the overall satisfaction.

I’ve posted before about how this oversimplifies things. For example, the following scores from two institutions would get the same overall satisfaction rating, but how different they are:

	Definitely disagree	Mostly disagree	Neither agree nor disagree	Mostly agree	Definitely agree
University A	20%	0%	0%	50%	30%
University B	0%	0%	20%	20%	60%

Both A and B have an overall satisfaction of 80%. But 20% of those who go to A are very unsatisfied; and of those who are satisfied, three in four at B are really satisfied, compared to only three in eight at A. University B definitely has greater student satisfaction than University A, but the simple overall measure doesn’t show it.

But there is value in simplicity. I prefer a different measure, which also has the virtue of reporting one number per institution, but which gives more nuance: the Grade Point Average (GPA). This gives a numerical weight for each different category of answer (that is, 1 for “Definitely disagree”, 2 for “Mostly disagree” and so on, up to 5 for “Definitely agree”) and multiplies these by the percentage reporting the relevant response.

So, using the examples above, University A scores 20% of 1 (Definitely disagree) plus 50% of 4 (Mostly agree) plus 30% of 5 (Definitely agree), which gives a GPA of 0.2+2.0+1.5=3.7.

University B scores 20% of 3 (Neither agree nor disagree) plus 20% of 4 (Mostly agree) plus 60% of 5 (Definitely agree), giving a GPA of 0.6+0.8+3.0=4.4.

You can see that with GPA it’s easier to compare the two universities: both universities score above 3, showing that overall students are positive; and B’s score of 4.4 is more than A’s score of 3.7, which reflects the more positive response of students at B.

So what happens if we use GPA to rank universities in 2016? I’m glad you asked, as I’ve done a comparison. The table shows UK universities ranked by the NSS method and by GPA. I’ve excluded from the list FE Colleges and alternative providers, in line with other NSS rankings you might see, so we have 159 institutions. (There's a bit more blog after the table, so please do keep scrolling down!)

NSS method			GPA method
University	%	Rank	GPA	University
The University of Buckingham	97%	1	4.72	Brighton and Sussex Medical School
Brighton and Sussex Medical School	95%	2	4.70	St Mary's University College
St Mary's University College	95%	3	4.61	University of St Andrews
Courtauld Institute of Art	94%	4	4.59	The University of Buckingham
University of Keele	94%	5	4.51	University of Keele
University of St Andrews	94%	6	4.49	Liverpool Hope University
Aberystwyth University	92%	7	4.48	Rose Bruford
Bishop Grosseteste University	92%	8	4.43	Central School of Speech and Drama
Harper Adams University	92%	9	4.42	The Royal Veterinary College
Liverpool Hope University	92%	10	4.41	University of Dundee
Coventry University	91%	11	4.40	Harper Adams University
Heythrop College	91%	12	4.40	Leeds College of Art
Stranmillis University College	91%	13	4.40	University of Lincoln
The Royal Veterinary College	91%	14	4.39	Conservatoire for Dance and Drama
The University of East Anglia	91%	15	4.39	University of Exeter
The University of Lancaster	91%	16	4.39	University of Oxford
University of Dundee	91%	17	4.38	Coventry University
University of Exeter	91%	18	4.37	Aberystwyth University
University of Lincoln	91%	19	4.37	Bangor University
Bangor University	90%	20	4.37	Loughborough University
Medway School of Pharmacy	90%	21	4.37	The University of East Anglia
Queen's University of Belfast	90%	22	4.37	The University of Essex
Rose Bruford	90%	23	4.37	The University of Lancaster
Swansea University	90%	24	4.37	University of Cambridge
Conservatoire for Dance and Drama	90%	25	4.36	The University of Bath
Central School of Speech and Drama	90%	26	4.35	The Royal Academy of Music
The University of Bath	90%	27	4.35	The University of Kent
The University of Essex	90%	28	4.35	The University of Surrey
The University of Kent	90%	29	4.34	Medway School of Pharmacy
The University of Leeds	90%	30	4.34	University of York
The University of Surrey	90%	31	4.33	Swansea University
University of Cambridge	90%	32	4.33	The Open University
University of Newcastle upon Tyne	90%	33	4.33	University of Newcastle upon Tyne
University of Oxford	90%	34	4.32	Queen's University of Belfast
Heriot-Watt University	89%	35	4.32	University of Portsmouth
Loughborough University	89%	36	4.32	Courtauld Institute of Art
Nottingham Trent University	89%	37	4.31	Stranmillis University College
Royal Holloway, University of London	89%	38	4.31	The University of Chichester
The Open University	89%	39	4.31	Bishop Grosseteste University
The Royal Academy of Music	89%	40	4.31	The University of Nottingham
The University of Birmingham	89%	41	4.31	University of Durham
The University of Nottingham	89%	42	4.30	The University of Sheffield
The University of Sheffield	89%	43	4.30	The University of Warwick
University of Durham	89%	44	4.30	University of the West of England
University of Glasgow	89%	45	4.29	Heriot-Watt University
University of Portsmouth	89%	46	4.29	Nottingham Trent University
University of St Mark & St John	89%	47	4.29	The University of Huddersfield
University of Stirling	89%	48	4.29	University of St Mark & St John
University of Sussex	89%	49	4.29	The University of Leeds
University of York	89%	50	4.28	Royal Welsh College of Music and Drama
Aston University	88%	51	4.27	The University of Leicester
Leeds College of Art	88%	52	4.27	University of Aberdeen
The Royal Agricultural University	88%	53	4.27	Birkbeck College
SOAS	88%	54	4.27	Heythrop College
The University of Leicester	88%	55	4.27	Liverpool Institute for Performing Arts
The University of Warwick	88%	56	4.27	SOAS
University of Aberdeen	88%	57	4.27	University of Glasgow
University of Abertay Dundee	88%	58	4.27	University of Ulster
University of Ulster	88%	59	4.26	Aston University
Birkbeck College	87%	60	4.26	Royal Holloway, University of London
Cardiff University	87%	61	4.26	University of Abertay Dundee
Edge Hill University	87%	62	4.26	University of Sussex
Oxford Brookes University	87%	63	4.26	University of the West of Scotland
Teesside University	87%	64	4.25	Edge Hill University
Liverpool Institute for Performing Arts	87%	65	4.25	Royal College of Music
The University of Chichester	87%	66	4.24	Falmouth University
The University of Huddersfield	87%	67	4.24	Royal Northern College of Music
The University of Liverpool	87%	68	4.24	University of Northumbria at Newcastle
The University of Reading	87%	69	4.24	Wrexham Glyndwr University
University of Chester	87%	70	4.23	The University of Reading
University of Strathclyde	87%	71	4.23	University College Birmingham
University of the West of England	87%	72	4.23	University of Gloucestershire
University of the West of Scotland	87%	73	4.22	Cardiff University
University of Winchester	87%	74	4.22	The City University
Anglia Ruskin University	86%	75	4.22	The University of Birmingham
Queen Margaret University, Edinburgh	86%	76	4.22	University of Southampton
Royal College of Music	86%	77	4.22	University of Strathclyde
Royal Welsh College of Music and Drama	86%	78	4.21	Anglia Ruskin University
The University of Hull	86%	79	4.21	Plymouth College of Art
University College Birmingham	86%	80	4.21	The University of Hull
University of Bristol	86%	81	4.21	University of Winchester
University of Derby	86%	82	4.20	University of Chester
University of Gloucestershire	86%	83	4.20	University of Stirling
University of Northumbria at Newcastle	86%	84	4.20	University of Wales Trinity Saint David
University of Southampton	86%	85	4.19	University of Derby
De Montfort University	85%	86	4.19	University of Bristol
Falmouth University	85%	87	4.18	Liverpool John Moores University
Guildhall School of Music & Drama	85%	88	4.18	St Mary's University, Twickenham
Hull and York Medical School	85%	89	4.18	Staffordshire University
Manchester Metropolitan University	85%	90	4.18	The Royal Agricultural University
Newman University	85%	91	4.18	The University of Bolton
St Mary's University, Twickenham	85%	92	4.18	The University of West London
Staffordshire University	85%	93	4.18	University for the Creative Arts
The City University	85%	94	4.18	University of Plymouth
The Robert Gordon University	85%	95	4.17	Oxford Brookes University
The University of Manchester	85%	96	4.17	Queen Margaret University, Edinburgh
University for the Creative Arts	85%	97	4.17	Queen Mary University of London
University of Plymouth	85%	98	4.17	Teesside University
University of Worcester	85%	99	4.17	University of Bedfordshire
Wrexham Glyndwr University	85%	100	4.17	University of East London
Bath Spa University	84%	101	4.16	De Montfort University
Birmingham City University	84%	102	4.16	The University of Northampton
Canterbury Christ Church University	84%	103	4.16	The University of Salford
Edinburgh Napier University	84%	104	4.16	University of Worcester
Leeds Beckett University	84%	105	4.15	Bath Spa University
Liverpool John Moores University	84%	106	4.15	Brunel University London
Queen Mary University of London	84%	107	4.15	Edinburgh Napier University
Sheffield Hallam University	84%	108	4.15	The Robert Gordon University
The University of Northampton	84%	109	4.14	Guildhall School of Music & Drama
The University of West London	84%	110	4.14	Imperial College London
The University of Wolverhampton	84%	111	4.14	Roehampton University
University College London	84%	112	4.14	Royal Conservatoire of Scotland
University of Bedfordshire	84%	113	4.14	The Arts University Bournemouth
University of Wales Trinity Saint David	84%	114	4.14	Middlesex University
Brunel University London	83%	115	4.13	London Metropolitan University
Cardiff Metropolitan University	83%	116	4.13	Manchester Metropolitan University
Glasgow Caledonian University	83%	117	4.13	The University of Liverpool
Goldsmiths' College	83%	118	4.13	York St John University
Imperial College London	83%	119	4.12	Cardiff Metropolitan University
Middlesex University	83%	120	4.12	Leeds Beckett University
Royal Conservatoire of Scotland	83%	121	4.12	The University of Wolverhampton
The University of Bradford	83%	122	4.11	The University of Manchester
University of Central Lancashire	83%	123	4.11	Goldsmiths' College
University of East London	83%	124	4.11	Newman University
University of Greenwich	83%	125	4.11	The University of Bradford
University of Sunderland	83%	126	4.11	University College London
York St John University	83%	127	4.11	University of Central Lancashire
Bournemouth University	82%	128	4.10	Buckinghamshire New University
Buckinghamshire New University	82%	129	4.10	Glasgow Caledonian University
King's College London	82%	130	4.09	Birmingham City University
Roehampton University	82%	131	4.09	Canterbury Christ Church University
Royal Northern College of Music	82%	132	4.09	University of Greenwich
The Arts University Bournemouth	82%	133	4.08	Hull and York Medical School
The University of Bolton	82%	134	4.08	Norwich University of the Arts
The University of Salford	82%	135	4.08	Sheffield Hallam University
University of Hertfordshire	82%	136	4.08	The University of Westminster
Leeds Trinity University	81%	137	4.08	University of Hertfordshire
London Metropolitan University	81%	138	4.07	King's College London
London South Bank University	81%	139	4.07	London South Bank University
Plymouth College of Art	81%	140	4.06	Southampton Solent University
Southampton Solent University	81%	141	4.05	Kingston University
The University of Westminster	81%	142	4.05	Leeds Trinity University
University of the Highlands and Islands	81%	143	4.05	University of Brighton
Kingston University	80%	144	4.04	University of Suffolk
St. George's, University of London	80%	145	4.03	Bournemouth University
University of Brighton	80%	146	4.03	University of Sunderland
University of Cumbria	80%	147	4.02	University of South Wales
University of Edinburgh	80%	148	4.02	University of the Highlands and Islands
University of South Wales	80%	149	4.01	University of Cumbria
University of Suffolk	80%	150	4.01	University of Edinburgh
Norwich University of the Arts	79%	151	3.98	St. George's, University of London
SRUC	76%	152	3.94	Trinity Laban
University of London Institute in Paris	76%	153	3.88	SRUC
LSE	75%	154	3.85	LSE
Glasgow School of Art.	74%	155	3.84	Glasgow School of Art.
Trinity Laban	72%	156	3.82	University of the Arts, London
University of the Arts, London	72%	157	3.80	University of London Institute in Paris
Ravensbourne	71%	158	3.73	Ravensbourne
Writtle College	68%	159	3.71	Writtle College

Made it this far? Well done!

Although many institutions stay in roughly the same place in the rankings, there are some big differences in both directions: Stirling, for instance, ranks 48^th by the NSS satisfaction measure, but 83^rd on GPA. And Bolton, for another example, ranks 134^th on the NSS measure, but 91^st using GPA. So the GPA measure is more revealing and richer as a single score than the satisfaction percentage alone.

GPA isn’t a perfect measure, of course. There’s an arbitrariness about assigning a consistent numerical difference between the five points on the scale: we don’t know how much individual respondents would weight the difference between the answers. And the simple percentage satisfaction score is more intuitive, and links more obviously directly to the raw data. This makes it easier to use.

So what’s the moral? Simply this: if you’re going to use the NSS to compare universities, look beyond the simple satisfaction measure. The data, publicly available, holds a wealth of information which you can use to better understand how universities are perceived by their students.

Caveat emptor!