By Bryan Greetham
In this period of COVID-19 it is essential that we understand risk and to do so means we need a basic understanding of the statistics involved in assessing risk.
On 23 rd August, Professor Chris Whitty, the UK’s chief medical adviser, claimed that children are more likely to be harmed by not returning to school next month than if they catch coronavirus. Although commonsense would suggest he’s probably right, you would hope that government policy is based on more than just commonsense; that it is backed by an objective, scientific assessment of risk. The problem is that trained professionals of all kinds struggle to understand information and assess risk. Judges, doctors, lawyers and jurors struggle to understand the evidence of probabilities laid before them, leading to confusion, misleading diagnoses and unsafe convictions that later have to be overturned.
trained professionals of all kinds struggle to understand information and assess risk.
In 1999 Sally Clark was found guilty of murdering her two infant children and given two life sentences. It was not until two other women had been convicted on the same basis that in January 2003 questions were asked about the figure given for the likelihood of two cot deaths occurring in one family. At the original trial this had been calculated as one in 73 million, which was arrived at by squaring 1 in 8,500, the figure for the chance of one event occurring. But it was based on the assumption that the two events were independent of one another, when in fact several studies had shown that there is an increased frequency of cot deaths in families where one had already occurred. This should have been challenged by the defence, but it wasn’t. Nobody seemed to understand the implications of the original assessment of the risk.
In Germany each year around 100,000 women have part of their breasts surgically removed after a positive test, when in fact most positive mammograms are false.
Unfortunately, their training has left professionals ill-equipped to make these sorts of calculations. Doctors often know the error rates of a clinical test and the base rate of a disease, but not how to infer from this the chances that a patient with a positive test has the disease. Consequently, many patients undergo unnecessary procedures, including surgery. In Germany each year around 100,000 women have part of their breasts surgically removed after a positive test, when in fact most positive mammograms are false.
By way of illustration, consider the following example. A prominent figure in medical research and teaching in Germany with over three decades of experience was given this problem. The probability that a woman has breast cancer is 0.8 per cent. If a woman has breast cancer, the probability is 90 per cent that she will have a positive mammogram. If she does not have breast cancer the probability is 7 per cent that she will still have a positive mammogram. Imagine a woman who has a positive mammogram. What is the probability that she actually has breast cancer?
David Eddy, former consultant to the Clinton administration on healthcare, gave essentially the same problem to 100 American doctors: 95 of them estimated the probability to be about 75 per cent, nearly 10 times the actual figure.
He studied it for about ten minutes and then guessed that it was around 90 per cent, although he wasn’t sure. The problem was also presented to 48 doctors with an average of 14 years experience, ranging from recent graduates to heads of departments. The estimates ranged from 1 per cent to 90 per cent; a third thought it was 90 per cent certain; a third estimated the chances to be 50 to 80 per cent; and a third estimated it to be lower than 10 per cent – half of these estimated it at 1 per cent. The median estimate was 70 per cent. Only 2 gave the correct estimate of around 8 per cent, but for the wrong reasons. David Eddy, former consultant to the Clinton administration on healthcare, gave essentially the same problem to 100 American doctors: 95 of them estimated the probability to be about 75 per cent, nearly 10 times the actual figure.
When information is presented in this form as a frequency, there is much less confusion and, as a result, much less variation in responses.
As with many problems the key to solving these lies in their representation: represent the problem differently and the solution seems so obvious that you wonder why you hadn’t seen it in the first place. Prior to the invention of probability theory in the 17th century, information was collected and processed as frequencies. For example, a doctor observes 100 people, 10 of whom have a new disease. Of these, eight display a symptom, while four of the 90 without the disease also show the symptom. The doctor, therefore, has four numbers to work with, four frequencies:
- Disease and symptom – 8
- Disease and no symptom – 2
- No disease and symptom – 4
- No disease and no symptom – 86
If she then observes a new patient with the symptom, she can easily see that the chances that this person has the disease is
Similarly, in a murder trial an expert witness might tell the jury that the DNA found at the murder scene matched the suspect’s DNA and there was only a 0.00001 probability or 0.001 per cent chance of being anyone else’s. Although it is difficult to get a clear sense of the significance of this, it does sound pretty convincing. But now present it as a frequency and things become a lot clearer. It means that out of every 100,000 people, one will show a match. So how many people are there who could have committed the murder? If the city in which this occurred has a 10 million adult population, there are 100 inhabitants, whose DNA would match the sample on the victim.
When information is presented in this form as a frequency, there is much less confusion and, as a result, much less variation in responses. There is also more likelihood that the public will have greater confidence in government statements and advice, if, along with ministers and advisers, we understand the calculations on which it is based.
1 Gerd Gigerenzer, Reckoning with Risk (London: Penguin, 2002), p. 229.
2 Gigerenzer, p. 41.
Bryan Greetham was born in Faversham, Kent, in England. He was educated at the University of Kent, where he gained a BA Hons in History, and at the University of Sussex, where he completed his MA in Intellectual History. He was awarded his PhD at the University of Newcastle in Australia for his work in moral thinking.
Bryan is the author of How to Write Better Essays, How to Write your Undergraduate Dissertation, both on writing and thinking skills, Philosophy, an introduction to philosophy for undergraduates, Thinking Skills for Professionals and his latest book, Smart Thinking, all published by Palgrave, Macmillan.
You must be logged in to post a comment.