Copy that, Minister

Jo Johnson, Universities minister, has taken time from piloting the Higher Education and Research Bill through Parliament, to taking strong action – or at the very least issuing a strongly worded press release – enjoining universities and the QAA to do more to prevent plagiarism. Specifically, to take action against essay mills.

Properly cited lyrics, so it isn't plagiarism

This is an issue which has been bubbling around for a while. The government brought together stakeholders last summer to consider the problems; in August 2016 the QAA published a pretty thorough analysis of the issues and made recommendations about what could be done.

The problem is simple to describe. There is a concern that some students are buying essays written by others and passing them off as their own. This is plagiarism, and is cheating. It isn’t clear how many students are doing this, but a few businesses which supply such essays are obviously doing decent business, and by implication the number of students doing this is non-negligible.  The Telegraph cites 20,000.

The QAA recommend a three-pronged approach:

• Partnership working by the sector to tackle the problem
• The possibility of legislation, perhaps like that in New Zealand
• Using regulators to prevent essay mills from advertising.

Jo Johnson’s statement focuses on the first and third approaches: the sector should sort it out, by means of guidelines and tougher penalties for students; the QAA should also go after the essay mills’ advertising. Legislation isn’t ruled out as a possibility, but is definitely kicked into the long grass. And, gratifyingly for the minister, some proper outraged press coverage about those naughty students.

I’m torn as to the best approach on this.

Early in my career I handled cases of student cheating, and plagiarism was by far the most common ‘irregularity’. This was in the very early days of the internet; pre Turnitin, and spotting plagiarism was down to eagle-eyed markers. And it was often pretty obvious: passages copied from books without editing, meaning that ‘as I argued on page 34’ for instance, showed up as out of place in a 6 page essay. Often the source material was cited in the bibliography. Whilst there was no doubt that the work had been copied, and without proper attribution, it was difficult with examples like this to see it as a deliberate attempt to deceive, as opposed to a complete lack of understanding about what the essay was trying to test. Often coupled with poor self-organisation meaning that at the last minute the student panicked.

In such cases throwing the book at the student would have been wrong, and that was often the academic judgement. A clear fail for the essay; a requirement to resubmit; and work to help the student understands what the problem was: this was a good remedy.

In the case of a student submitting as their own an essay which they’ve bought online, it is harder to be so forgiving. There seems to be more agency involved in committing the exam irregularity. Buying an essay and passing it off really isn’t a failure to understand the practices of academic referencing; it’s a straightforward attempt to cheat. And on that argument, the minister is right. Sort it out, HE sector.

But legislation surely can’t be a bad thing to get on with. Even if the ‘crime’ is committed by the student who submits the purchased essay, there really isn’t an argument for leaving the essay providers untouched. The notion that these are bought for ‘research’ purposes is laughable. The idea that tailored exemplars are a good way to learn is not a strong one. (The use of model answers afterwards to help feed back to students on their performance is a different question.) And the New Zealand law, which empowers the higher education quality regulator to prosecute essay mills, does not look like a bad law.

Would it solve the problem entirely? No – essay mills may be based outside the UK, and beyond the scope of the law. Would it help to convey the message that the practice is wrong? Yes. Would it help to demonstrate that the UK takes academic standards seriously? Yes. Is there a convenient current parliamentary vehicle to achieve this? Unbelievably, yes there is.  The minister could do more to make this happen now.

There’s another angle here. A student who submits a plagiarised essay may get caught, and face a penalty, or may not. But what is absolutely certain is that they will not learn as much about the essay topic as a student who reads and writes and submits honestly, even if they fail or get a poor mark. The plagiariser knows less than the honest student. And this cost is borne throughout their life. To plagiarise is to fail.

If universities could convey this message better to students, we might be getting somewhere important.

Where does 2.8% fee inflation come from?

So we know that the government intend – certainly for English universities – to permit undergraduate fees for Home/EU students to grow to £9,250 for new and continuing students in 2017-18. This was set out in the BIS report ‘Teaching Excellence Framework: Provisional list of eligible providers – Year One’ published on 7 July 2016 and confirmed by Jo Johnson, Minister for Universities and Science, in a written statement to parliament on 21 July 2016:

For all new students and eligible continuing students who started their full-time courses on or after 1 September 2012 and are undertaking courses at publicly funded higher education providers that have achieved a TEF rating of Meets Expectations, maximum tuition fee caps will be increased by forecast inflation (2.8%) in 2017/18. 

The TEF rating relates to the new regime set out in the Higher Education and Research Bill, under which providers who meet certain standards of teaching excellence will be allowed to charge higher fees. The Bill is midway through its parliamentary journey, and my expectation is that at some point it will become a matter of controversy. But for now I want to focus on a different question. What is the basis for the forecast of 2.8% inflation in 2017-18? 

The Bank of England

The Bank of England is responsible for managing inflation in the UK, and in its most recent inflation report (table 5B on page 31) forecasts CPI of 1.5% in the second quarter 2017 (ie April-June 2017) and 2.0% in the second quarter 2018 (ie April-June 2018). The Bank’s model gives probability estimates for inflation differing from this forecast. For Q2 2017 the Bank places a probability of 10% that the actual rate of CPI is between 2.5% and 3.0%; for Q2 2018 the equivalent probability is 12%. (These data come from the excel file available to download from the Bank of England). So we can see that the Bank of England is not forecasting inflation of 2.8% for 2017-18.

The Bank also reports other forecasts of inflation (page 44 of the report), from over twenty other forecasters. The average of these was 1.6% for Q2 2017 and 2.0% for Q2 2018 – not a great deal different from the Bank of England’s own forecast. The key point – other forecasters are not forecasting inflation of 2.8% in 2017-18.

Is the government separately forecasting this rate of inflation? The Office for Budgetary Responsibility publishes an ‘Economic and Fiscal outlook’, the most recent of which was March 2016 (to co-incide with the budget). This reports CPI forecasts of 1.6% for 2017 and 2.0% for 2018 (page 12, table 1.1). So, the government does not forecast inflation of 2.8% for 2017-18.

This is a bit of a puzzle. Is the government using a forecast other than CPI? I don’t think that any other policy area has a separate calculation and forecast of inflation. Universities UK maintained an inflation series – the Higher Education Pay and Prices Index – but this was discontinued a few years ago. Most of the (positive) difference in HE inflation related to pay costs, and it would seem odd for the government to publicly acknowledge that a sector was likely to have higher pay than inflation. So I think we can discount the idea that the government has a separate HE cost inflator.

This leaves another possibility. Perhaps the amount of the increase - £250 – was agreed on, and then the inflation forecast adjusted to match. Universities have in the past been effective lobbyists of government, and both Universities UK and the mission groups are well connected. £250 is fractionally less than 2.8% of £9,000. So maybe that’s where the 2.8% forecast came from.

Is there another explanation? I’d like there to be one, and perhaps a more learned reader can supply it. Until then, I think I’ve got a bit more cynical about HE policy-making.

Research and Development

So the review of the Research Councils by Sir Paul Nurse has been published, and what a report it is. A much better read than many government reports – it felt like the innocent scientific positivity in some of John Wyndham’s novels, or even better, in The Black Cloud by Sir Fred Hoyle.

I studied philosophy at the LSE, and the focus was very much – unsurprisingly for a department built upon Karl Popper’s work on epistemology and scientific method – around the philosophy of science. So it isn’t every day that the reading lists given to us crop up in government policy documents. But here we are: Popper’s Conjectures and Refutations, and TS Kuhn’s Structure of Scientific Revolutions. I was hoping for a bit of Imre Lakatos or Paul Feyerabend too, but perhaps that was asking too much.

It’s also got a lovely opening section into the whys and wherefores of research. And a historical gem: the funding of the Medical Research Council from National Insurance contributions at a rate of “a penny per working person per year” in the 1911 National Insurance Act. By my calculations – and using the historical data set available from the Bank of England – that would equate now to 3 shillings and 11 pence  per working person, or just under 18 pence per head in post-decimal coinage. By comparison, the total science budget of £2.6bn in 2014-15 equates to £68.82 per working person per year. So it’s fair to say that government investment in research has grown over time.

So, from my reading, what are the key points of the review?

• Research in all disciplines is really important to a nation, and can be focused on discovery, on applying knowledge to a problem, or translating basic research to applied problems (translational). And a great line: “To rush into translation may result in becoming lost in translation.”
• Research Councils waste a lot of time engaging with bureaucracy, and bringing back control of support in house rather than through the UK Shared Business Services is necessary.
• RCUK should be given more strength as a co-ordinator of the efforts of the seven research councils, acting as the interlocutor with government and enabling the research councils to focus on research funding. It should become a new body – Research UK – with a single Accounting Officer replacing those in the individual research councils.
• Management of the funding process needs to improve, with better arrangements for international peer review, panel discussions, diversity, transparency and speed.
• There needs to be better collaboration with other research funders, and especially business, charities, devolved administrations, and Europe.
• Relationships with government need to be clarified, and a new structure to work with Research UK is identified.

Sir Paul’s report contains a curious mixture of realism and optimism.

On the realist front: “Given the many and varied demands made on the public purse which Government will need to balance, it is probably more likely the funding level will be set too low rather than too high.”. You're telling me.

On the optimist front: “The changes proposed through this report are not complex and could be easily adopted without disrupting on-going research activities”. Up to a point, Lord Copper.

Will it do the trick? It doesn’t actually reduce the number of BIS quangos, although it does create a single infrastructure, so costs should go down. It does reduce complexity, by having a single Accounting Officer. And it certainly makes sense about funding high quality research.

But ... Jo Johnson’s response included an ominous last paragraph:

I encourage everyone with an interest in the future of our research and innovation landscape to consider this review alongside the proposals set out in the Higher Education Green Paper we published recently.

Sounds like a watch this space to me…

It's not all about the TEF, you know

There’s been much speculation about the details of the Teaching Excellence Framework, but the forthcoming green paper on Higher Education will contain much more than that. A discussion yesterday at the AUA Partnerships network conference prompts me to take a closer look at one aspect of this.

Jo Johnson’s speech to UUK on 9 September contained the following:

"The green paper will cast a critical eye over the processes for awarding access to student support funding, Degree Awarding Powers and University Title.

We have already made a start by providing a new route for trusted new and smaller providers to grow their student numbers. We are also beginning to link student number controls to the quality of the provider, through a “performance pool” which will operate for 2016 to 2017.

But the green paper will consult on options to go further. Success in higher education should be based on merit, not on incumbency. I want to fulfil our aim of a level playing field for all providers of higher education.

Many of you validate degree courses at alternative providers. Many choose not to do so. I know some validation relationships work well, but the requirement for new providers to seek out a suitable validating body from amongst the pool of incumbents is quite frankly anti-competitive. It’s akin to Byron Burger having to ask permission of McDonald’s to open up a new restaurant.

It stifles competition, innovation and student choice, which is why we will consult on alternative options for new providers if they do not want to go down the current validation route."

The basic point here is that without the power to award degrees, it’s tricky to be a provider of higher education. The practice of validation – a university agreeing that a curriculum and education designed and delivered by another organisation can lead to the award of a degree from the university – is a solution to this. (Note that validation is different from franchising – under franchising, the university owns the curriculum but outsources the teaching.)

There are 75 UK universities which are members of the Council of Validating Universities (CVU), which gives an idea of the scale of the practice. There isn’t (yet) a single definitive register of validated providers, but research conducted for BIS in 2013 counted 674 privately funded HE providers, most of which will have a validation relationship with one of more universities. So the ministers claim that “Many of you [remember, he was talking to Vice-Chancellors] validate degree courses at alternative providers” looks true. And it can be a decent business for universities, helping a faculty to balance its books.

The underlying cause of the minister’s ire – the need to find a way to empower new colleges in times of expanding higher education - isn’t new. The University of London fulfilled this function, via its external degree, in the nineteenth and first half of the twentieth century: many UK and overseas universities can trace their origins back to colleges offering tuition for the London external degree. The Council for National Academic Awards (CNAA) fulfilled this function for the polytechnic sector until it was wound up in 1993, after the polytechnics had been made into universities.

Can we tell what approach might be in the green paper? If we take the CNAA model, then the establishment of a new body to do this could be possible. But the creation of a new quango at a time when they are set to be culled seems counter to the spirit of the times (O tempora! O mores!). So the University of London model is the other alternative from history: perhaps designating an existing university as having a 'duty to validate', or creating another university which is only a validator?

There’s a moral hazard here, just as the minister perceives an anti-competitive hazard in current arrangements. The key to validation is the maintenance of academic standards, and if you’ve a duty to validate, then an important element of the validation relationship – that of judging the capacity of another institution to meet the right standards – is put at risk. Some of the outcomes of the QAA’s review of alternative providers show the problems here: some alternative providers are good, but there are also some very shoddy ones.

Is the minister’s argument rooted in specific concerns? It would interesting to know which alternative providers have complained, and which university has refused to validate. What were the specifics? Was the refusal justified? Or perhaps the validation fee was simply seen as too high? Unless we can see that it’s a market that is broken and can’t be fixed by regulation, then the creation of a new entity seems premature.

I’ll be interested to see what the green paper has to propose.

A curate's egg

Higher Education Minister Jo Johnson’s speech to Universities UK this morning presages an interesting few months. He set the scene for a forthcoming green paper, with four broad themes.

Firstly, teaching excellence. The idea of a Teaching Excellence Framework was set out in July; we now know a bit more, but I’m not certain that BIS have a clear idea yet. On the one hand, they know their target: it’s the idea of students’ workload, with the Minister comparing a busy engineering student at Bristol with a drop-out humanities student at ‘a prestigious London university’.  And there’s the notion of a disengagement contract.

The Minister quotes Palfreyman and Tapper; it’s worth looking at the full quote (from Reshaping the University: the Rise of the Regulated Market in Higher Education):

The last item paints a grim picture indeed of ‘limited learning on college campuses’ based on an extensive research project funded by the US Social Science Research Council (this is not some hysterical polemic to be brushed aside by the HE establishment): students’ ‘academic effort has dramatically declined in recent decades’ from some 40 hours per week in the 1960s to about 27 in the 2000s, and the ‘faculty cultures and orientations’ of ‘the college professoriate’ has much to answer for, since they have struck a ‘disengagement contract’ with their students (along the lines of ‘I don’t want to have to set and mark much by way of essays and assignments which would be a distraction from my research, and you don’t want to do coursework that would distract you from partying: so we’ll award you the degree as the hoped-for job ticket in return for compliance with minimal academic requirements and due receipt of fees’; and on the Party Pathway through HE as some HEIs come to resemble country clubs see Armstrong and Hamilton 2013).

The words quoted by the minister are italicised; the whole sentence shows that the authors were describing a US study of a US issue. There may well be issues with teaching in UK universities, but I’m not sure that it’s right to raise the temperature by scare stories from across the Atlantic. And increasing the marketization of HE is precisely moving us towards a US model, not away from it.

That aside, there’s also a little bit of muddle between what they’re seeking to encourage. There’s speak of excellence in teaching, assessment, feedback and employment skills. From students being busy, and the Minister’s recollections of university life (Oxford, Balliol, Modern History) we also get an implicit elision to contact hours. So do we care about students being busy, or being in the classroom? They’re not the same thing.

And there’s an emphasis on information to applicants so they can see what they get (presumably KIS hasn’t done the trick). It’ll be good to see what this actually looks like when the Green paper is published; but there’s also a clarity that it’ll matter. Success in TEF will enable a university to increase its £9k fee in line with inflation. Ouch!

Second up is Widening Participation. The Minister has two issues in his sights: participation by black and minority ethnic students with a Caribbean background, and participation by white British boys from disadvantaged backgrounds. Participation by both of these groups is low, and I think that the focus is spot on in this regard. And there’s talk of better data to enable this to be understood (be still my beating heart!)

Third up, a blast form the past. Having spotted that alternative providers need validation to be able to award degrees, and that this presents a potential conflict of interest, there’ll be a consultation on ‘alternative options for new providers if they do not want to go down the current validation route’. Which sounds an awful lot like the return of the CNAA. The old Gray’s Inn Road building looks like rental office space now – I wonder if that could be used …

Parts of the speech are excellent

And finally, a level playing field. There’s a recognition that the current regulatory environment is complicated, with different levels of scrutiny for different types of provider, and, now that HEFCE funds universities less and students themselves fund more, a more limited range of sanctions are available to regulators. If you only have a nuclear option, you never use it, which isn’t good for regulation. Wales has got this a bit better – by having HEFCW oversee access agreements, there’s a more nuanced approach possible. I wouldn’t be surprised if HEFCE and OFFA went the same way.

Another aspect of a level playing field is recognising that entry to the market implies the possibility of exit from the market, and the promise of consultation on ‘measures to require all providers to have protection measures in place so that students who benefit from greater choice and diversity do not lose out in the event of provider failure’. An insurance bond scheme for universities? It’s an idea from the travel market, but the costs of securing provision elsewhere would be potentially large: I’ll be interested to see how the economics of this play out.

So lots to come, and some good and necessary issues flagged, but there’s still woolly and ideological thinking in there.

Agenda setting

Not a minister

Not a Scottish footballer

Jo Johnson, Universities minister, gave a speech to UUK today.

When his appointment was announced back in May, I misread and thought they’d appointed Mo Johnston, former Scotland striker. Instead they went for a slightly blonder minister, whose speech shows that the government has an agenda for higher education.

The speech name-checked the government’s manifesto pledges, so it’s worth reminding ourselves about what they were:

We will ensure that if you want to go to university, you can

This year, for the first time, over half a million people have been admitted to our universities, including a record proportion of students from disadvantaged backgrounds. From September, we will go even further, abolishing the cap on higher education student numbers and removing an arbitrary ceiling on ambition. Our reforms to university funding mean you do not have to pay anything towards tuition while studying, and only start paying back if you earn over £21,000 per year. We will ensure the continuing success and stability of these reforms, so that the interests of both students and taxpayers are fairly represented. We will also introduce a national postgraduate loan system for taught masters and PhD courses. We will ensure that universities deliver the best possible value for money to students: we will introduce a framework to recognise universities offering the highest teaching quality; encourage universities to offer more two-year courses; and require more data to be openly available to potential students so that they can make decisions informed by the career paths of past graduates.

Conservative manifesto 2015, p35

Like every good to-do list, it includes two things which have already been done – removing the cap on places at English universities and introducing postgraduate loans (again in England only – HE is a devolved matter). That leaves four pledges:

1. Ensuring the continuing success and stability of the student loan system
2. Introducing a framework for teaching quality
3. Encourage university to offer more two-year courses
4. Require more data to be available to students

‘Value for money’ is the catch all for the last three of these.

And then the speech itself:

… my focus will be on implementing 3 key manifesto pledges, so that we consolidate and build on these achievements:
• Firstly, lifting the cap on student numbers and widening participation, so that we remove barriers to ambition and meet the PM’s commitment to double the proportion of disadvantaged young people entering higher education by 2020 from 2009 levels.
• Secondly, delivering a teaching excellence framework that creates incentives for universities to devote as much attention to the quality of teaching as fee-paying students and prospective employers have a right to expect.
• Thirdly, driving value for money both for students investing in their education, and taxpayers underwriting the system, so that we ensure the continuing success and stability of these reforms.

Teaching at the heart of the system, speech by Jo Johnson to UUK, 1 July 2015

And he had a lot to say about this, highlighting:

• Data that shows students are less happy about choices they have made (but see my post on the ComRes data)
• The possibility of assembling data showing student engagement and specific salaries from different universities, and the need for greater transparency about how fees are spent
• The government’s commitment to support an increase in the number of high quality universities
• His worry that the graduate premium has reduced, and the need to find out why and do something about it
• The need for partnership with business to influence curriculum design
• Plans for a Teaching Excellence Framework: 

"I expect the TEF to include a clear set of outcome-focused criteria and metrics. This should be underpinned by an external assessment process undertaken by an independent quality body from within the existing landscape."

• A green paper to be published in the autumn
• The need to tackle degree class inflation
• The need to use continual assessment (ie the GPA system) to push students harder throughout their programmes (with a dig at contact hours in the meantime)
• His intention to focus on WP in the more selective institutions (by means of two year programmes, apparently)

Four thoughts on this.

Firstly, who’ll run TEF? QAA looks like the obvious candidate, but let’s not forget the other “independent quality bod[ies] from within the existing landscape” –

• HEA – they know about teaching
• HESA – they have data on student progress
• OFFA – they know about student success
• OIA – they know what makes students cross
• Ipsos Mori – they know about student opinion through the NSS

Secondly, how do you do a light-touch TEF? You can use data on student achievement and value-added, which is about outcomes not about teaching; you can survey the taught, which is about perceptions not about outcomes; or you can inspect the teaching, which won’t be light touch. Answers on a postcard to Jo Johnson, please, before he has to sign off the green paper.

Thirdly, this stuff about the GPA. I’ll need to think about this but it seems to me that the argument goes that moving from degree classification to a 13 point scale will help student motivation because students currently don’t work hard because degree classes are an inexact measure of achievement. I’m not sure that this is an entirely flawless argument.

Fourthly, the graduate premium. The minister acknowledges that a university education is about more than salary, but argues that the decline in the graduate premium is a cause for concern, and promotes data that enable students to see what previous graduates have earned. Is this an argument that paves the way for raising the cap on fees, when the evidence, or sector action, supports the conclusion that it’s worth it? Or an argument for not raising the fee cap, as there’s work yet to be done?

There’s more to think about, and enough material for seminars and conferences, let alone some more blog posts. Interesting times.