GCSE Mathematics

Maths is not the easy, deterministic subject

It is tempting to assume marking Maths is simple because arithmetic has right answers. The data says otherwise. The two examiners agreed with each other less closely on Maths than on English - 0.771 against 0.811 on the agreement score - because GCSE Maths marks are mostly method marks, and whether a student's approach earned one is a judgement rather than a calculation.

Raw agreement rates point the other way, and that is the trap. The examiners gave the same Maths mark 72% of the time against 29% on English, but most questions here are worth one to three marks, where agreeing by chance is easy. The agreement score corrects for that; the raw percentage does not.

How it compares with a general AI model

On the measures a teacher would recognise, Marking.ai and the general model both sit at the examiner bar and the differences are small. On the agreement score we read below it: 0.741 against 0.795, with the examiners at 0.771. Compared answer by answer the difference is not statistically separable from zero, so the honest summary is that we are level with a bare model on this corpus rather than behind it. On English we were measurably ahead. Here we were not, and we would rather you read that from us than find it.

Four answers explain almost all of the gap, and the explanation is not flattering to the alternative. In those four the student's working was not in the image at all: the canvas holds a single figure such as 50.5 degrees, because the working sat on the question paper, which the corpus does not capture. Marking.ai awarded one mark out of five and recorded why - no working shown, so the method marks could not be evidenced. The general model awarded five out of five, for working it could not have seen.

Remove those four answers and the order reverses: 0.780 for Marking.ai against the general model's 0.770. We publish the unadjusted figure, because dropping four inconvenient answers to improve a number is the practice this page exists to argue against. But it is worth knowing which behaviour you are buying: a marker that will not award a mark it cannot evidence.

One question resists that explanation. On a three-mark constrained optimisation question both arms scored far below the examiners, and we were wrong in both directions on different answers. The examiners were unstable there too - their own agreement was the second lowest of the twelve questions, and two answers with identical working were given different marks. It is a contested question rather than a defect with a known fix, and we have not fixed it.

What the study covers

It measures marking: an answer that has already been read off the page, marked against a mark scheme. Each answer was supplied on its own page. A teacher uploads a scanned script with many questions on it, and this study does not measure reading one. Keeping thirty scripts attached to the right questions is a different job, and we have not measured it.

The answers were chosen to cover the full range of marks, not to look like a real class. And the corpus has been public since June 2026, so an AI model trained after that date may have seen it - which applies equally to every arm, so the comparison between them holds.

As with every study in this section, it measures agreement rather than correctness. There is no ground-truth mark here, and on this subject the examiners disagree with each other more than they do on English.

How we measured it

Every answer was marked through the production marking path - the same routing, prompts, models and parsing a teacher's marking runs through. Nothing was re-implemented for the study. Each arm was scored against both examiners and averaged, so no arm can score well by happening to match whichever examiner was listed first. The corpus is the Medly AI GCSE marking benchmark, published openly under a CC BY 4.0 licence. We did not mark the work and we did not choose the answers.