Exams algorithm was not ‘mutant’ and contained ‘predictable’ errors
The marking system which threatened to downgrade the exam results was not a "mutant algorithm" as claimed by the prime minister, but contained errors that were "predictable" and "could have been foreseen", according to a leading statistician.
Boris Johnson told a group of schoolchildren on Wednesday that "I'm afraid your grades were almost derailed by a mutant algorithm," a reference to the exam marking system which was scrapped after almost 40% of A-level results were downgraded from teachers' predictions.
Stian Westlake, the CEO of the Royal Statistical Society (RSS) said the "colourful phrase" did reflect the fact that ministers, officials and students "were surprised by the results that it generated".
But, he said, the algorithm "wasn't a mutant or a freak of nature. This was something that could have been foreseen".
Mr Westlake, a former adviser to the Universities Minister, said the results were "a predictable surprise" because of the demand that it reduce grade inflation. "The fact that this could have produced a lot of inaccuracy, which translates into unfair grades for individuals, that was known," he said.
Education Secretary Gavin Williamson has blamed exam regulator Ofqual, claiming he "consistently asked a large number of challenging questions about the system" and was reassured about its "robustness and its fairness".
But Ofqual has said that it was instructed by ministers to keep grade inflation down, which meant students' results would have to be downgraded from teachers' predicted grades, which tend to be higher than actual results.
More from UK
Professor Jo-Anne Baird, a member of Ofqual's standing advisory committee, told the BBC: "The Secretary of State for Education asked them to produce a system which didn't allow exam results to go up from previous years. So effectively this government has been very good at controlling grade inflation and that's essentially what they asked for, a system that did that."
Asked whether or not the exam marking algorithm was "mutant", Downing Street declined to comment.
Leyton Sixth Form College student Wiktoria Sniadowska, who was predicted AAA by her school then given BBC by the algorithm, said she felt let down by the prime minister's "mutant algorithm" comments.
"Boris Johnson is prime minister, he's in charge. If it's a mutant, we knew it was going to be created this way for months in advance, so why didn't he stop it? Why didn't he say: 'This is wrong, we need to do it differently'?"
Mr Westlake said the row showed the need for transparency in the creation of algorithms, not only because it helped prevent errors, but also because it allowed a public debate about their aims.
"I think the really important lesson from this is that we can't treat statistics as a holy grail," he said.
"We need to prepare them with transparency, we need to make sure we understand the assumptions that are going into them and we need to make sure that we use them in the right way.
"If you've got statistics that only have a certain degree of accuracy, as for example the A-level adjustments did, we can't pretend they are more accurate than they really are."