Recognizing REAL pseudo-science — Details, distinctions, and pundit Ben Goldacre
Posted by Henry Bauer on 2011/12/26
Quite some time ago I had occasion to lambaste columnist Ben Goldacre, MD, for his culpably uninformed comments about Christine Maggiore; see “Scientifically illiterate science pundit: Ben Goldacre”. I put his book, Bad Science, on my reading list but at low priority in the expectation that it would be an unpleasant chore necessary as a prelude to writing a critical review. Finally got around to reading the book, and find that I need to urge others to read it, in particular for its sound discussions of how the media’s coverage of science is generally misleading; how misleading too is the way in which statistics are disseminated by drug companies and the media; and how the drug companies are not to be trusted. And there are various interesting tidbits of information:
→ Goldacre is spot on about the mess that the media make of covering scientific matters, and how disastrous is the ignorance of those in power (Chapter 11: How the media promote the public misunderstanding of science). He cites a dictionable* word, churnalism, credited to journalist Nick Davies and referring to the uncritical rehashing of press releases as news.
→ Chapter 12, “Bad Stats”, is generally sound and informative about how statistics can be misused and misrepresented, though the presentation is not very well organized. I don’t agree, for example, that “natural frequencies” are the only sensible way to communicate risk, though I agree that they should always be included. I would also have liked prime emphasis on correlation never proving causation and high probability never equaling certainty.
→ The book acknowledges that serious flaws in clinical trials are quite common (pp. 44-5).
→ The drug companies are properly given short shrift, e.g. p. 184 ff.
→ Specific details show how clinical trials can be deliberately biased to favor drug approval, and other flaws in the process (pp. 189-206).
→ The detailed debunking of British charlatans, though directly pertinent only for British readers, are well worth reading because similar situations with similar characteristics are present on this side of the Atlantic.
→ There are interesting tidbits of information about early German research into smoking and lung cancer (footnote, p. 218) and data indicating that episodes of fear of vaccination have been regional.
On the negative side:
→ Goldacre appears to believe that mainstream medicine is evidence-based (pp. x, 316), when most of it isn’t. Later he cites anecdotal evidence that 50-80% of treatment decisions are evidence-based, but only 13% of the treatments themselves are evidence-based with another 21% are “likely to be beneficial” (p. 182). In other words, doctors who properly rely on the evidence available to them from drug companies and official agencies are, about half the time, relying on unsound evidence.
Throughout, the book insinuates that mainstream medicine can be trusted even when admitting that it often cannot be, e.g. at p. 99 when referring to the Cochrane Collaboration.
In this vein, Goldacre misleads about John Ioannidis’s work (p. 219), implying that it reveals the unreliability of brand new studies. But Ioannidis has actually shown that widely and long accepted mainstream treatments are based on flawed early trials presented by drug companies.
→ Goldacre is far too blithely dismissive of the harm done to “a very small number of people” by any medical intervention or “any human activity” (p. 298). “Whenever we take a child to be vaccinated”, Dr. Goldacre writes, “we’re aware that we are striking a balance between benefit and harm, as with any medical intervention” (p. 313). Nonsense. Statisticians and researchers may understand this, but most of the rest of us trust the advice our doctors give us — particularly when we have no choice but to sign the “informed consent” forms if we want to be treated. We just hope that our trust is warranted, we don’t balance benefits against risks. Goldacre may not understand this because he doesn’t deal with patients.
→ A pervasive strand of Goldacre’s bravado is denigration of “humanities graduates” by contrast to scientists, among whom he seems, wrongly, to include doctors; see “Doctors aren’t scientists, and medicine isn’t science”.
→ The book implies that glucosamine can do nothing against arthritis (p. 155). A judicious evidence-based assessment says otherwise **.
→ Goldacre is quite wrong about HIV/AIDS (p. 88) when even common sense ought to have warned him: if Botswana really has a 48% prevalence of what’s supposedly a fatal disease, the country ought to have been depopulated long ago. Antiretrovirals are described as life-saving (p. 184), but they are the opposite.
→ Goldacre denigrates Linus Pauling for cherry-picking (p. 98) — the Pauling who is widely regarded as the greatest chemist of the 20th century, the founder of molecular biology, winner of two Nobel Prizes, who urged the importance of dietary anti-oxidants (and was maligned for doing so) long before it became the conventional wisdom. This is only one example of Goldacre’s outsized ego, hubris, self-confidence, and inability to see himself as others see him. The book’s style may turn some people off for this reason; Goldacre is just too full of himself.
On several points, I’m not sure how sound the coverage is:
→ Goldacre parrots the mainstream condemnation of Andrew Wakefield who warned that multiple vaccinations might be a cause of autism. I’ve read only enough about this affair to conclude that legitimate questions remain. Goldacre notes the irony that there is a definite correlation between maternal rubella infection while pregnant and autism in the later-born child; but surely this makes plausible Wakefield’s belief that exposure to rubella vaccine at an early age might act similarly? In some babies at least?
Since most or all vaccines harm a few individuals, surely it is always worth keeping a mind open and studying possible reasons for that, looking for characteristics that might identify people particularly likely to react badly to a specific vaccine.
→ David Horrobin is described as marketing by dubious means remedies that turned out to be ineffective (p. 157 ff.). The details Goldacre presents seem sound, though he admits that Horrobin may not have been guilty of actual deceit. This picture does not jibe with the David Horrobin who founded Medical Hypotheses and published sensible articles about peer review and the like, as well as the fascinating book, The Madness of Adam and Eve (Bantam 2001).
Bad Science and the many columns Goldacre has written illustrate two absolute truths:
1. There are no general principles or guidelines that can serve as short cuts for deciding whether any given controversial claim is worth attending to. There is no sound way to pronounce something “good science” or “bad science” without digging comprehensively into the evidence and the arguments pro and con; see Science or Pseudoscience: Magnetic Healing, Psychic Phenomena, and Other Heterodoxies.
2. Any given individual or book can be sound on some general matters and on some specific instances and yet quite wrong about other instances and generalities. The degree to which Goldacre is sound on any given point correlates with the amount of detail with which he is familiar.
Quite generally, compendia of “pseudo-science” are likely to be wrong about some of the topics, because the compilers of such lists simply haven’t had the time to look in sufficient detail at all the topics they cover. Recent examples include science journalist Specter’s Denialism (2009), which is uninformed and wrong about HIV/AIDS among others. A much better book is physicist Friedlander’s At the Fringes of Science (1995), but it remains uninformed and wrong about UFOs and cold fusion, for instance. Shermer’s Why People Believe Weird Things: Pseudoscience, Superstition, and Other Confusions of Our Time, too, is good about some topics and biased on others. Various older compendia and debunkings of supposed pseudo-science are cited in Science or Pseudoscience: Magnetic Healing, Psychic Phenomena, and Other Heterodoxies. The mid-20th century classic in this genre is Martin Gardner’s Fads and Fallacies in the Name of Science (1952/57). It is detailed and instructive about some outlandish claims but fails to let the reader know that chiropractic and osteopathy have overcome their cranky birth in one person’s hunch and developed into pragmatically useful healing techniques, superior to mainstream medicine in handling lower-back pain, for example.
All compendia have one thing in common: the authorial claims to base judgments on general principles or assessment of specific evidence are rationalizations; what is labeled as sound is what happens — for whatever reason — to strike the author as sound, and anything the author finds unbelievable — for whatever reason — is labeled pseudo-science. Readers have this choice: accept the author’s personal opinions, or dig into the evidence for themselves and arrive at an informed opinion.
* “dictionable”: worthy of being in a dictionary but not yet included. Another worthy is “tritto”, to describe repetition beyond “ditto”.
** Donal O’Mathuna & Walt Larimore, Alternative Medicine — The Christian Handbook, Zondervan 2001. Although several early chapters are addressed specifically to Christians, the assessments of specific claimed remedies are entirely empirical and evidence-based.