Thursday, 11 August 2016

NSS Decoded

The results of the National Student Survey for 2016 were released yesterday, and there’s much jubilation at those institutions which have done well, accompanied by a judicious silence from others. All final year undergraduate students are invited to complete the study, which asks a range of questions about their experience at university and their learning. It also includes one question to sum up their overall experience, and it is this question which drives many published rankings. Students are asked to say how much they agree or disagree with the following statement:
 “Overall, I am satisfied with the quality of the course”
The possible answers are “Definitely agree”, “Mostly agree”, “Neither agree nor disagree”, “Mostly disagree” and “Definitely disagree”. The outcomes are reported as the sum of the percentage of respondents who definitely or mostly agree, this being reported as the overall satisfaction.

I’ve posted before about how this oversimplifies things. For example, the following scores from two institutions would get the same overall satisfaction rating, but how different they are:


Definitely disagree
Mostly disagree
Neither agree nor disagree
Mostly agree
Definitely agree
University A
20%
0%
0%
50%
30%
University B
0%
0%
20%
20%
60%

Both A and B have an overall satisfaction of 80%. But 20% of those who go to A are very unsatisfied; and of those who are satisfied, three in four at B are really satisfied, compared to only three in eight at A. University B definitely has greater student satisfaction than University A, but the simple overall measure doesn’t show it.

But there is value in simplicity. I prefer a different measure, which also has the virtue of reporting one number per institution, but which gives more nuance: the Grade Point Average (GPA). This gives a numerical weight for each different category of answer (that is, 1 for “Definitely disagree”, 2 for “Mostly disagree” and so on, up to 5 for “Definitely agree”) and multiplies these by the percentage reporting the relevant response.

So, using the examples above, University A scores 20% of 1 (Definitely disagree) plus 50% of 4 (Mostly agree) plus 30% of 5 (Definitely agree), which gives a GPA of 0.2+2.0+1.5=3.7.

University B scores 20% of 3 (Neither agree nor disagree) plus 20% of 4 (Mostly agree) plus 60% of 5 (Definitely agree), giving a GPA of 0.6+0.8+3.0=4.4.

You can see that with GPA it’s easier to compare the two universities: both universities score above 3, showing that overall students are positive; and B’s score of 4.4 is more than A’s score of 3.7, which reflects the more positive response of students at B.

So what happens if we use GPA to rank universities in 2016? I’m glad you asked, as I’ve done a comparison. The table shows UK universities ranked by the NSS method and by GPA. I’ve excluded from the list FE Colleges and alternative providers, in line with other NSS rankings you might see, so we have 159 institutions. (There's a bit more blog after the table, so please do keep scrolling down!)

NSS method
GPA method
University
%
Rank
GPA
University
The University of Buckingham
97%
1
4.72
Brighton and Sussex Medical School
Brighton and Sussex Medical School
95%
2
4.70
St Mary's University College
St Mary's University College
95%
3
4.61
University of St Andrews
Courtauld Institute of Art
94%
4
4.59
The University of Buckingham
University of Keele
94%
5
4.51
University of Keele
University of St Andrews
94%
6
4.49
Liverpool Hope University
Aberystwyth University
92%
7
4.48
Rose Bruford
Bishop Grosseteste University
92%
8
4.43
Central School of Speech and Drama
Harper Adams University
92%
9
4.42
The Royal Veterinary College
Liverpool Hope University
92%
10
4.41
University of Dundee
Coventry University
91%
11
4.40
Harper Adams University
Heythrop College
91%
12
4.40
Leeds College of Art
Stranmillis University College
91%
13
4.40
University of Lincoln
The Royal Veterinary College
91%
14
4.39
Conservatoire for Dance and Drama
The University of East Anglia
91%
15
4.39
University of Exeter
The University of Lancaster
91%
16
4.39
University of Oxford
University of Dundee
91%
17
4.38
Coventry University
University of Exeter
91%
18
4.37
Aberystwyth University
University of Lincoln
91%
19
4.37
Bangor University
Bangor University
90%
20
4.37
Loughborough University
Medway School of Pharmacy
90%
21
4.37
The University of East Anglia
Queen's University of Belfast
90%
22
4.37
The University of Essex
Rose Bruford
90%
23
4.37
The University of Lancaster
Swansea University
90%
24
4.37
University of Cambridge
Conservatoire for Dance and Drama
90%
25
4.36
The University of Bath
Central School of Speech and Drama
90%
26
4.35
The Royal Academy of Music
The University of Bath
90%
27
4.35
The University of Kent
The University of Essex
90%
28
4.35
The University of Surrey
The University of Kent
90%
29
4.34
Medway School of Pharmacy
The University of Leeds
90%
30
4.34
University of York
The University of Surrey
90%
31
4.33
Swansea University
University of Cambridge
90%
32
4.33
The Open University
University of Newcastle upon Tyne
90%
33
4.33
University of Newcastle upon Tyne
University of Oxford
90%
34
4.32
Queen's University of Belfast
Heriot-Watt University
89%
35
4.32
University of Portsmouth
Loughborough University
89%
36
4.32
Courtauld Institute of Art
Nottingham Trent University
89%
37
4.31
Stranmillis University College
Royal Holloway, University of London
89%
38
4.31
The University of Chichester
The Open University
89%
39
4.31
Bishop Grosseteste University
The Royal Academy of Music
89%
40
4.31
The University of Nottingham
The University of Birmingham
89%
41
4.31
University of Durham
The University of Nottingham
89%
42
4.30
The University of Sheffield
The University of Sheffield
89%
43
4.30
The University of Warwick
University of Durham
89%
44
4.30
University of the West of England
University of Glasgow
89%
45
4.29
Heriot-Watt University
University of Portsmouth
89%
46
4.29
Nottingham Trent University
University of St Mark & St John
89%
47
4.29
The University of Huddersfield
University of Stirling
89%
48
4.29
University of St Mark & St John
University of Sussex
89%
49
4.29
The University of Leeds
University of York
89%
50
4.28
Royal Welsh College of Music and Drama
Aston University
88%
51
4.27
The University of Leicester
Leeds College of Art
88%
52
4.27
University of Aberdeen
The Royal Agricultural University
88%
53
4.27
Birkbeck College
SOAS
88%
54
4.27
Heythrop College
The University of Leicester
88%
55
4.27
Liverpool Institute for Performing Arts
The University of Warwick
88%
56
4.27
SOAS
University of Aberdeen
88%
57
4.27
University of Glasgow
University of Abertay Dundee
88%
58
4.27
University of Ulster
University of Ulster
88%
59
4.26
Aston University
Birkbeck College
87%
60
4.26
Royal Holloway, University of London
Cardiff University
87%
61
4.26
University of Abertay Dundee
Edge Hill University
87%
62
4.26
University of Sussex
Oxford Brookes University
87%
63
4.26
University of the West of Scotland
Teesside University
87%
64
4.25
Edge Hill University
Liverpool Institute for Performing Arts
87%
65
4.25
Royal College of Music
The University of Chichester
87%
66
4.24
Falmouth University
The University of Huddersfield
87%
67
4.24
Royal Northern College of Music
The University of Liverpool
87%
68
4.24
University of Northumbria at Newcastle
The University of Reading
87%
69
4.24
Wrexham Glyndwr University
University of Chester
87%
70
4.23
The University of Reading
University of Strathclyde
87%
71
4.23
University College Birmingham
University of the West of England
87%
72
4.23
University of Gloucestershire
University of the West of Scotland
87%
73
4.22
Cardiff University
University of Winchester
87%
74
4.22
The City University
Anglia Ruskin University
86%
75
4.22
The University of Birmingham
Queen Margaret University, Edinburgh
86%
76
4.22
University of Southampton
Royal College of Music
86%
77
4.22
University of Strathclyde
Royal Welsh College of Music and Drama
86%
78
4.21
Anglia Ruskin University
The University of Hull
86%
79
4.21
Plymouth College of Art
University College Birmingham
86%
80
4.21
The University of Hull
University of Bristol
86%
81
4.21
University of Winchester
University of Derby
86%
82
4.20
University of Chester
University of Gloucestershire
86%
83
4.20
University of Stirling
University of Northumbria at Newcastle
86%
84
4.20
University of Wales Trinity Saint David
University of Southampton
86%
85
4.19
University of Derby
De Montfort University
85%
86
4.19
University of Bristol
Falmouth University
85%
87
4.18
Liverpool John Moores University
Guildhall School of Music & Drama
85%
88
4.18
St Mary's University, Twickenham
Hull and York Medical School
85%
89
4.18
Staffordshire University
Manchester Metropolitan University
85%
90
4.18
The Royal Agricultural University
Newman University
85%
91
4.18
The University of Bolton
St Mary's University, Twickenham
85%
92
4.18
The University of West London
Staffordshire University
85%
93
4.18
University for the Creative Arts
The City University
85%
94
4.18
University of Plymouth
The Robert Gordon University
85%
95
4.17
Oxford Brookes University
The University of Manchester
85%
96
4.17
Queen Margaret University, Edinburgh
University for the Creative Arts
85%
97
4.17
Queen Mary University of London
University of Plymouth
85%
98
4.17
Teesside University
University of Worcester
85%
99
4.17
University of Bedfordshire
Wrexham Glyndwr University
85%
100
4.17
University of East London
Bath Spa University
84%
101
4.16
De Montfort University
Birmingham City University
84%
102
4.16
The University of Northampton
Canterbury Christ Church University
84%
103
4.16
The University of Salford
Edinburgh Napier University
84%
104
4.16
University of Worcester
Leeds Beckett University
84%
105
4.15
Bath Spa University
Liverpool John Moores University
84%
106
4.15
Brunel University London
Queen Mary University of London
84%
107
4.15
Edinburgh Napier University
Sheffield Hallam University
84%
108
4.15
The Robert Gordon University
The University of Northampton
84%
109
4.14
Guildhall School of Music & Drama
The University of West London
84%
110
4.14
Imperial College London
The University of Wolverhampton
84%
111
4.14
Roehampton University
University College London
84%
112
4.14
Royal Conservatoire of Scotland
University of Bedfordshire
84%
113
4.14
The Arts University Bournemouth
University of Wales Trinity Saint David
84%
114
4.14
Middlesex University
Brunel University London
83%
115
4.13
London Metropolitan University
Cardiff Metropolitan University
83%
116
4.13
Manchester Metropolitan University
Glasgow Caledonian University
83%
117
4.13
The University of Liverpool
Goldsmiths' College
83%
118
4.13
York St John University
Imperial College London
83%
119
4.12
Cardiff Metropolitan University
Middlesex University
83%
120
4.12
Leeds Beckett University
Royal Conservatoire of Scotland
83%
121
4.12
The University of Wolverhampton
The University of Bradford
83%
122
4.11
The University of Manchester
University of Central Lancashire
83%
123
4.11
Goldsmiths' College
University of East London
83%
124
4.11
Newman University
University of Greenwich
83%
125
4.11
The University of Bradford
University of Sunderland
83%
126
4.11
University College London
York St John University
83%
127
4.11
University of Central Lancashire
Bournemouth University
82%
128
4.10
Buckinghamshire New University
Buckinghamshire New University
82%
129
4.10
Glasgow Caledonian University
King's College London
82%
130
4.09
Birmingham City University
Roehampton University
82%
131
4.09
Canterbury Christ Church University
Royal Northern College of Music
82%
132
4.09
University of Greenwich
The Arts University Bournemouth
82%
133
4.08
Hull and York Medical School
The University of Bolton
82%
134
4.08
Norwich University of the Arts
The University of Salford
82%
135
4.08
Sheffield Hallam University
University of Hertfordshire
82%
136
4.08
The University of Westminster
Leeds Trinity University
81%
137
4.08
University of Hertfordshire
London Metropolitan University
81%
138
4.07
King's College London
London South Bank University
81%
139
4.07
London South Bank University
Plymouth College of Art
81%
140
4.06
Southampton Solent University
Southampton Solent University
81%
141
4.05
Kingston University
The University of Westminster
81%
142
4.05
Leeds Trinity University
University of the Highlands and Islands
81%
143
4.05
University of Brighton
Kingston University
80%
144
4.04
University of Suffolk
St. George's, University of London
80%
145
4.03
Bournemouth University
University of Brighton
80%
146
4.03
University of Sunderland
University of Cumbria
80%
147
4.02
University of South Wales
University of Edinburgh
80%
148
4.02
University of the Highlands and Islands
University of South Wales
80%
149
4.01
University of Cumbria
University of Suffolk
80%
150
4.01
University of Edinburgh
Norwich University of the Arts
79%
151
3.98
St. George's, University of London
SRUC
76%
152
3.94
Trinity Laban
University of London Institute in Paris
76%
153
3.88
SRUC
LSE
75%
154
3.85
LSE
Glasgow School of Art.
74%
155
3.84
Glasgow School of Art.
Trinity Laban
72%
156
3.82
University of the Arts, London
University of the Arts, London
72%
157
3.80
University of London Institute in Paris
Ravensbourne
71%
158
3.73
Ravensbourne
Writtle College
68%
159
3.71
Writtle College

Made it this far? Well done!

Although many institutions stay in roughly the same place in the rankings, there are some big differences in both directions: Stirling, for instance, ranks 48th by the NSS satisfaction measure, but 83rd on GPA. And Bolton, for another example, ranks 134th on the NSS measure, but 91st using GPA. So the GPA measure is more revealing and richer as a single score than the satisfaction percentage alone.

GPA isn’t a perfect measure, of course. There’s an arbitrariness about assigning a consistent numerical difference between the five points on the scale: we don’t know how much individual respondents would weight the difference between the answers. And the simple percentage satisfaction score is more intuitive, and links more obviously directly to the raw data. This makes it easier to use.


So what’s the moral? Simply this: if you’re going to use the NSS to compare universities, look beyond the simple satisfaction measure. The data, publicly available, holds a wealth of information which you can use to better understand how universities are perceived by their students.

Caveat emptor!

No comments:

Post a comment