Why is teacher assessment biased?

 

I have previously written about how disadvantaged pupils do better on tests than on teacher assessments – and also about how many people assume the opposite is the case (Christodoulou, 2015a). It’s interesting that today, we seem to think that teacher assessment will help the disadvantaged. In the late 19th and early 20th century, the meritocratic advantages of tests were better understood than they are today, as were the nepotism and prejudice that often resulted from other types of assessment. In the 19th century, the civil service reformers Charles Northcote and Stafford Trevelyan fought a long battle to make entry to the civil service based on test scores, rather than family connections. In the early 20th century, Labour educationalists such as RH Tawney and Beatrice and Sidney Webb fought for an education system based around exams, because they believed that only exams could ensure that underprivileged children were treated fairly (Wooldridge, 1995). Since then, we have only gathered more evidence about the equalizing power of exams, but oddly, we seem to have forgotten these insights.

I have explained before why it is that tests are fairer – they treat every pupil the same, and every pupil has to answer the same questions (Christodoulou, 2015a). However, whilst I gave plenty of evidence that teacher assessment was biased, I didn’t fully explain why this bias happens. As a result, quite a few people have suggested that the solution to bias in teacher assessment is ‘better teacher assessment’, perhaps by introducing more moderation and CPD, as a group of teaching unions recommended doing in 2011. How realistic is that? Are the problems with teacher assessment really so insoluble that we have to resort to tests? And what exactly are the nature of these flaws?

Teacher assessment is biased not because it is carried out by teachers, but because it is carried out by humans. Tammy Campbell, the UCL Institute of Education researcher whose recent research showed bias in teacher assessments of 7-year-olds, is at pains to point this out. She says, ‘I want to stress that this isn’t something unique to teachers. It’s human nature. Humans use stereotypes as a cognitive shortcut and we’re all prone to it.’ (quoted in Adams, 2015) A growing body of research reinforces Campbell’s point. We all have difficulties making certain complex judgments and decisions, and we resort to shortcuts when the mental strain becomes too great (see Daniel Kahneman’s work for more on this). Indeed, it is plausible to speculate that the reason why teacher assessment is biased is because it is so burdensome: when we are faced with difficult cognitive challenges we often default to stereotypes.

And teacher assessment really is burdensome. I thought I had it bad as a secondary teacher marking coursework for GCSE pupils, but I recently spoke to a primary colleague who told me about the hours they spend gathering evidence for KS1 assessments and cross-referencing it against the level descriptors – a task which, as I’ve said before (Christodoulou, 2015b), is at the limits of human cognitive capacity. When faced with such a difficult challenge, defaulting to stereotypes is in many ways a sensible attempt by our unconscious minds to reduce our workload. We know that on average pupils on free school meals do not attain as well, we know that the essay we are marking isn’t great, but it isn’t terrible, we know it sort of meets some of the criteria on the mark scheme, we need some more evidence in order to reach a final judgment, we could reread the essay and mark scheme but the mark scheme is hard to interpret…we also know that the pupil who wrote the essay is from the wrong side of the tracks. Done: it’s a below average essay. None of us want to admit that this is how our minds work, and for most of us, our minds don’t consciously work like this, but there is plenty of evidence that this is how our reasoning goes. That’s the nice story: here, Adrian Wooldridge (2014) gives the less charitable interpretation of the mental processes of assessors. He says that when you base assessment around ‘Oxford dons who pronounce mystically on whether a candidate possesses “a trace of alpha”’, don’t then be surprised when ‘a large number of those who show up favorably on the alpha detectors turn out to be Old Etonians.’

Is it possible to counter such bias in any way? Is it possible to ‘do better teacher assessment’? After all, whilst we humans are susceptible to bias, we are also self-aware. We know we make these errors, and in most of the fields where we make these errors, we have found ways around them. I once heard the scientific method described as a collection of practices designed to counteract human bias. Are there practices we can introduce to teacher assessment that would function like this, and counteract human bias? Yes, there are. We could anonymize the work that’s being marked. We could standardize tasks, conditions and marking protocols. We could carry out some statistical analysis of the marks different pupils get and that different teachers give. And once we had done all that, we would find that we had eliminated many of the biases associated with teacher assessment, but that we had also pretty much eliminated the teacher assessment and replaced it with a test. The flaws with teacher assessment are inherent in its very nature. Doing teacher assessment better basically means making it more test-like. The whole point of the test is that it, like the practices that characterize the scientific method, is essentially a method for countering human bias. As the Bew report (2011) says, most of the attempts to reduce the bias of teacher assessment have failed, and those that have succeeded do so by making teacher assessment more test-like.

Teacher assessment discriminates against poorer pupils and minorities, and generates significant workload for teachers. Tests are fairer and less burdensome. They deserve a better reputation than they have, and a greater role in our assessment system.

This article was originally published as blog post on the author’s website.

References

Adams R (2015) Children from poorer families perceived by teachers as less able, says study. In: The Guardian, 9 June 2015. Available at: https://www.theguardian.com/education/2015/jun/09/teachers-poorer-children-education-primary-school (accessed 13 June 2020).

Bew P (2011) Independent review of Key Stage 2 testing, assessment and accountability. Available at: https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/176180/Review-KS2-Testing_final-report.pdf (accessed 13 July 2020).

Christodoulou D (2015a) Test are inhuman and that is what is so good about them. Available at: https://daisychristodoulou.com/2015/10/tests-are-inhuman-and-that-is-what-so-good-about-them/(accessed 13 July 2020).

Christodoulou D (2015b) Marking essays and poisoning dogs. Available at https://daisychristodoulou.com/2015/06/marking-essays-and-poisoning-dogs/ (accessed 13 July 2020).

Wooldridge A (1995) Bell Curve Liberals: How the left betrayed I.Q. The New Republic. Available at: https://newrepublic.com/article/119332/iq-excuse-egalitarianism-vs-meritocracy (accessed 13 July 2020).

Wooldridge A (2014) A True Test: In Defense of the SAT. The New Republic. Available at: https://newrepublic.com/article/119322/defense-standardized-tests-sat (accessed 13 July 2020).

 

Share your experiences with educators globally by joining the discussion below. How might assessment bias be addressed in your context?

0 0 vote
Article Rating
Subscribe
Notify of
1 Comment
Inline Feedbacks
View all comments
s
Ms Stephanie Hilder
1 month ago

Thank you Daisy, for putting this point of view. I hope your points are heard and understood. In addition to the points you have made, I would like to hear more discussion around the fundamental reasons why disadvantaged pupils do less well than their more advantaged peers. It is pointless expecting any assessment system to reduce inequality which arises not so much from assessment, as from birth. If the national discussion and investment were directed towards addressing the causes of disadvantage, assessment outcomes would likely take care of themselves. We know, don’t we, what children need to thrive and succeed… Read more »