Scientific illiteracy? Innumeracy? Lack of common sense?
Posted by Henry Bauer on 2011/11/06
We’re deluged with assertions about everything, typically via TV and newspapers and magazines bringing statements from experts and authoritative organizations. Those assertions are to be trusted only at your peril.
Assertions in advertisements are obviously deceitful to a greater or lesser extent, and thereby perhaps less dangerous than those coming from apparently independent, disinterested sources. However, those purportedly independent, disinterested sources can be ignorant in a frighteningly high proportion of cases.
That assertion of mine was prompted by an episode of the Daily Show with Jon Stewart. I mean it quite literally when I insist that the Daily Show and the Colbert Report offer nowadays the soundest commentary to be found anywhere in the media on matters of culture and politics, because they are not ideologically prejudiced nor snowed by experts; they make experts’ assertions run the gauntlet of common sense.
Sometimes, of course, I can’t be sure whether they’re making something up or reporting it. So it was when the Daily Show presented a news item to the effect that one could increase by 20% one’s chances of contracting colon cancer if one ate a hot dog every day. A clip showing a doctor representing the Physicians Committee for Responsible Medicine suggested that the Daily Show was indeed reporting and not creating this absurdity. My Google search revealed that the Physicians Committee for Responsible Medicine (PCRM) exists; and it does warn that one hot dog — and more generally “just one 50-gram serving of processed meat” — “consumed daily increases the risk of colorectal cancer, on average, by 21 percent”. To bring this warning to the general public, PCRM set up a billboard comparing the risk of hot dogs to that of smoking:
To what should one ascribe this idiocy about hot dogs and colon cancer? Are AICR and PCRM suffering from innumeracy, scientific illiteracy, or just a lack of plain common sense?
I think there are two ways of discerning that the assertion is idiotic, a less technical one — common sense — and a slightly more technical one — elementary statistics and snippets of general knowledge about oncology, toxicology, biology generally.
Common sense, by the way, is not something one has; it is something one ought to use: “having” common sense means thinking. With matters like hot dogs and cancer, common sense can usually recognize the absurdity by asking the question,
HOW COULD THAT POSSIBLY BE KNOWN? *
The sweeping claim about cancer-causing processed meat cannot be based on actual experience. It could only be validated by an enormous clinical trial in which the diets of two groups of people were controlled so that processed meat was the only difference between them; and the two groups — one eating the purportedly dangerous meat and the other avoiding it — would have to be matched for every conceivably relevant characteristic: age, sex, environment, lifestyle, genetic predispositions. Quite literally impossible.
On what are these claims based, if not on actual data?
On studies on animals and statistical associations.
Extrapolating effects on animals to human beings involves a number of obvious uncertainties. No non-human animals have exactly the same physiology, nor the same proneness to cancers. Except in extreme cases (something like cyanide, say), results on animals can only be suggestive. Translating critical dosage in animals to human beings is typically done with a ratio of respective body-weights, which presumes no significant differences in physiology or proneness to cancers.
On these common-sense grounds alone, claims like cancer-causing hot dogs are better described as sheer speculation than as scientific findings.
Some bits of general knowledge about technical matters can also help against being snowed by such assertions about technical matters.
ALWAYS bear in mind that associations, correlations, links, do not identify CAUSES. Almost all of what the media disseminate on matters of health and diet are mere statistical associations. Economic and social status are often found to be “linked” to diet and health; so anything else that’s “linked” to diet or health is likely to be linked also to economic or social status. Hot dogs and processed meat in general form a larger part of the diet at lower economic levels than at higher ones. So maybe colon cancer is “caused” by poverty?
Highly recommended reading: Darrell Huff, How to lie with statistics; Joel Best, Damned lies and statistics: untangling numbers from the media, politicians, and activists and More damned lies and statistics: how numbers confuse public issues.
Furthermore, “statistically significant” never means “real”, “correct”, “true”. It merely states an arbitrary probability of being so.
That arbitrary probability, in social science and medicine, is typically “p<0.05” — less than 5% probability that the apparent link, correlation, association is owing to pure chance. On the face of it, this means 1 out of every 20 “statistically significant” correlations is not a correlation at all.
Furthermore, that a correlation is not owing to chance doesn’t entail that the correlation is owing to what the researchers or observers take it to be: there may be some unknown factor involved.
Because this approach only tests how likely a result is to be owing to chance, it greatly overestimates the statistical significance of the hypothesis actually being tested, the more so, the less inherently likely the correlation is on general grounds, grounds of experience, grounds of common sense — see R. A. J. Matthews (1998), “Facts versus factions: The use and abuse of subjectivity in scientific research”, reprinted as pp. 247-282 in J. Morris (ed.), Rethinking Risk and the Precautionary Principle, Butterworth, 2000; and (1999), “Significance levels for the assessment of anomalous phenomena”, Journal of Scientific Exploration, 13: 1-7.
Keep in mind too that if one looks for a large number of possible associations, some will crop up purely by chance: 1 out of every 20 if one uses the simpleminded interpretation of p<0.05. Therefore if one looks for a possible connection between processed meat and 20 different kinds of cancer, one is likely to find an apparent but spurious correlation with one of them.
If one looks for a correlation between colon cancer and 20 different possible constituents of diet, one is likely to find an apparent but spurious correlation with one of them.
For all these reasons based in statistics, any claim of a 20% change in risk is more than questionable. Given all the uncertainties, perhaps claims of a several-fold change might suggest that there’s something to the claim. A claimed 20% is easily shrugged off. When it is stated more precisely as 21%, it becomes absolutely ridiculous.
An often-overlooked aspect of the miracles of life is the degree to which living systems are able to protect themselves against harm and to repair damage. One of the typical extrapolations made in assertions like cancer and hot dogs is that effects are linear, that eating 100 hot dogs has 100 times the effect that eating 1 hot dog has. This is wrong. Our bodies can repair certain damages at certain rates, and harm comes only if the damage comes at a greater rate. In small amounts delivered not too rapidly, a toxic substance may actually improve health, a phenomenon known as hormesis; see for example J. M. Kauffman, “Radiation hormesis: Demonstrated, deconstructed, denied, dismissed, and some implications for public policy”, Journal of Scientific Exploration 17 (2003) 389-407; A plausible explanation is that the toxin (or radiation) stimulates the immune system to work “overtime” and wipe out not only the toxin but other threats as well.
When it comes to cancer, the mechanism that sets it going is not known. Much mainstream belief holds that several steps are needed as well as some predisposition in the form of oncogenes. That makes it unlikely in the extreme that any single factor like processed meat could decrease by some universally average amount the risk of contracting any given cancer.
There is absolutely no reason to give even a moment’s credence to the asserted connection between colon cancer and processed meat.
* * * * * * * *
It’s easy and rather natural to accept at face value organizations that call themselves something like American Institute for Cancer Research or Physicians Committee for Responsible Medicine. However, there are reasons for not doing so automatically: “let the buyer beware” (caveat emptor) remains excellent advice. For instance there is the phenomenon of “astroturfing”, organizations that pretend to be genuine “grass roots” but which are actually fronts for self-interested commercial or political entities. Even when charities and non-profits are independent, honestly sui generis, we are warned periodically to check their bona fides and financial arrangements before contributing to them, to assess for example how much is spent on the organization’s professed purpose as against how much benefits the management and staff. So I looked briefly into PCRM and AICR.
PCRM is a not-for-profit cancer charity, with revenues of $8.8 million in 2009 of which $2.6 million was spent on salaries and wages. The website advertises a book (21-Day Weight Loss Kickstart) and book-tour by PCRM’s president. There is fairly extreme dietary advice, about the cheese-obesity link, for example, or the risks from milk: “In a study of 142 children with diabetes, 100 percent had high levels of an antibody to a cow’s milk protein. It is believed that these antibodies may destroy the insulin-producing cells of the pancreas. . . . The lactose sugar, when it is digested, releases galactose, a simple sugar that is linked to ovarian cancer and cataracts.” PCRM also campaigns against the use of animal subjects in research.
AICR is a not-for-profit cancer charity that spent $5-6 million on research in 2010 and about the same on salaries and wages. Three times that much (>$17 million) was spent on “public health education”, nearly $4 million on management and more than $7 million on fund-raising. Eight officers received >$100,000 each. The company contracted to raised funds by phone solicitation brought in nearly $11 million, but at a cost of $7.5 million, whereas direct mail solicitation raised $11 million at a cost of only $650,000. The cancer-prevention advice offered by AICR seems banal and anything but original (see Schedule O of “FY 2010 Form 990” on the AICR website).
None of this means that what these groups assert is necessarily wrong; but it should not be taken as disinterested and purely evidence-based. With cancer and hot dogs, as I have just shown, they are not to be trusted.
* For a full discussion of this approach, and of how to reach a personal decision as between mainstream dogma and unorthodox claims, see chapter 7 in Science or Pseudoscience: Magnetic Healing, Psychic Phenomena, and Other Heterodoxies)