Doctors are busy people and therefore tend to skim medical journals rather than read them line by line. They look at the abstract at the beginning of a paper and the conclusions at the end, and trust that the conclusions are justified by all that went before.
To ensure that this is so, after all, is the scientific editors’ job; moreover, peer review should already have ensured the elimination of obvious error. Therefore, a highly-respected journal such as the New England Journal of Medicine surely can be trusted? And if it cannot be, what journal can be?
Until my retirement from practice, I admit, I was a skimmer rather than a careful reader. Thus much of my supposed knowledge was really a kind of sophisticated rumour rather than real knowledge. True, super specialists who cultivate only a tiny corner of the vast medical field read the papers pertaining to that corner more carefully, but most doctors are not super specialists, nor can or should they ever be. They have therefore to know a little about a lot rather than a lot about a little. This in turn means that they cannot look very deeply into what they read but have to take a lot on trust.
After my retirement, I had more time to examine some of the journals a little more closely and I chose the NEJM in particular because, together with The Lancet, it is the most prestigious medical journal in the world, read, or at least glanced at, by hundreds of thousands. I was surprised by what I found.
Dr Marcia Angell, a former editor for 20 years, rather belatedly came to the conclusion that much of what is published as disinterested research—up to 60 per cent—is actually false, the work having been distorted by commercial interest. Certainly, I found examples in the NEJM of trials of new drugs in which the pharmaceutical developer seemed to have been responsible for the analysis of the results and the writing of the paper, which does not automatically mean that what is written is false or distorted, but which does raise suspicions.
But this is far from the only kind of distortion I came across. For example, it is quite often the case that a relative risk is given without an absolute risk. Let me illustrate what I mean by a fictional example: Suppose research showed that eating bananas during pregnancy doubled the risk of a child developing progeria, a horrible and fatal condition. This result, which sounds dramatic, would be trumpeted in the world’s press and no doubt cause a panic and a fall in the world price of bananas. Who would willingly double the risk of the child developing such a terrible condition merely by eating bananas?
But wait: the original risk of a child developing the condition is one in 8 million, so that a doubling of the risk means that it is now one in 4 million. If we worried about every risk that was one in 4 million we should soon be nervous wrecks and never eat or do anything. Moreover, the research which found the risk might either be nonreproducible or turn out to be a statistical artefact caused by poor design of the study or mistaken analysis of the results.
In other words, it is important not just to give relative risks but absolute risks too, and this is frequently not done. In my experience, where the reader has to work out the absolute risk for himself, it is usually trivial.
It is very rare that a statistical correlation, even in the best medical journals such as the NEJM, is not taken to indicate cause and effect. For example, a huge statistical study (of a complexity that not one in a thousand doctors would understand) analyzed the relationship between death rates and air pollution. The correlation was not extremely strong, but the authors nevertheless claimed that the way to reduce disparities in health was to reduce air pollution. In spite of the immense sophistication of their method, they made the elementary error of mistaking correlation with causation.
To show that this is so, let me point out that there was probably as strong a correlation with the homicide rate as with the death rate, possibly even stronger. No one would be against a reduction of air pollution, but not on the basis that it would reduce the homicide rate, or think that the way to reduce the homicide rate was to reduce air pollution.
Such simple errors are to be found in every issue—the NEJM is neither unique nor the worst in this respect. I could list several other type of simple errors—mostly avoidable or easily correctible.
In their social commentary, medical journals, not excluding the NEJM, almost always follow the moral fashions of the intellectual elite. For example, the legalization of assisted suicide and euthanasia are regarded as self-evidently enlightened policies. The Netherlands has long been one of the most “enlightened” countries in the world in this regard, and in 2017 the NEJM published a letter from that country illustrating just how enlightened it was.
The editors of the journal seemed not to have noticed that when the authors stated that, of 6000 patients a year assisted to die in the Netherlands last year, 92 per cent were deemed to have had “serious physical illnesses,” the other 8 per cent—480 persons—presumably had not. Surely any minimally inquiring editor would have asked for clarification on what they had, if not serious physical illnesses? Moreover, serious physical illness is not the same as closeness to death. Clarification, one might have thought, was necessary here too.
Error is inseparable from activity and not every error is culpable. Moreover, there is much that is good or even brilliant in medical journals. Nevertheless, close examination reveals an alarming amount of avoidable error, which is all the more regrettable because many doctors take the NEJM and The Lancet as holy writ.
Based on the book, False Positive, A Year of Error, Omission and Political Correctness in the New England Journal of Medicine ©2019 Encounter Books, by Theodore Dalrymple.
Theodore Dalrymple was born in London in 1949. He retired as a full-time physician in 2005. He is the author of many books including Life at the Bottom (Ivan Dee) and Romancing Opiates (Encounter), which challenges the notion that heroin addiction is a bona fide disease. His latest book is False Positive—A Year of Error, Omission, and Political Correctness in the New England Journal of Medicine.
