More professionally published garbage
Posted by Henry Bauer on 2011/05/13
No sooner had I posted about news not fit to print than TIME Healthland brought the headline,
“Study: Gay Men Are Twice as Likely to Have Cancer”
It took me a little while to find the original source, in part because there are two journals titled “Cancer” and in other part because this ground-breaking announcement had not yet been published and was available only in the “Early View” list on the journal’s website.
The actual report reveals that the headline was utterly misleading: the article concerns relationships between sexual orientation and cancer survivorship, not cancer incidence:
“these data only address self-reported survivorship; therefore, they do not adequately reflect cancer incidence” (Boehmer et al., “Cancer survivorship and sexual orientation”, Cancer DOI: 10.1002/cncr.25950).
Furthermore, the authors themselves list reasons why their “study” should not be taken seriously: it is restricted to a California group and not representative of the groups allegedly being studied, namely, those differing in sexual orientation. The second part of the article’s “Discussion” section explains quite well why the first half, which reports alleged differences by sexual orientation, should be ignored.
But it would not even be necessary to read that far to understand why this was “news” not fit to print. The authors merely searched in data from surveys published in 2001, 2003, and 2005 by the California Health Interview, looking for correlations by sexual orientation. Statistics 101 teaches that if you look for enough correlations you are sure to find some, the actual rate depending upon what criterion you choose as the threshold for “statistical significance”. If the correlations being looked for are inherently plausible, and the typical significance level of p<0.05 is used, perhaps only 1 in 20 “correlations” will be entirely spurious (but at least 1 in 20); if the correlations that seem to pop up are implausible, the rate of false-positive correlations will be considerably higher (see for example R A J Matthews, “Significance levels for the assessment of anomalous phenomena”, Journal of Scientific Exploration 13 #1  1-7).
Boehmer et al. “found” a higher incidence of cancer among gay men, albeit not for melanoma, colon cancer, or “multiple cancers”, and a “significantly” lower rate of prostate cancer. How many correlations did they look at for different types of cancer? Perhaps the apparently significant correlations are spurious.
So too with women: “There were no significant differences between lesbians and heterosexual women with respect to their health perception or any of the cancer sites (results not shown). Bisexual women, however, differed significantly from heterosexual women”. How plausible is it that bisexual women but not lesbians differ significantly from heterosexual women in any way at all? I wager that no replication of this “finding” will ever be published. But that does not prevent the authors from claiming that the results identify “bisexual women as a new risk group for cervical cancer”! And they propose that this group be targeted for “screening intervention to reduce the prevalence of cervical cancer in this population”.
For that matter, what conceivable reason is there to look for correlations between sexual orientation and cancer, be it incidence or survivorship? The vaunted (albeit mythical) scientific method begins with a plausible hypothesis of some sort and then seeks to test it against evidence; what was the plausible hypothesis to be tested here?
The authors recommend “interventions that target lesbian and bisexual cancer survivors to improve their health perceptions”. One might wonder why that has any importance at all. Is “health perception” another new medical condition, an illness, to be diagnosed and treated? (Read Selling Sickness by Moynihan & Cassels.),
The Boehmer et al. article illustrates nicely what I had pointed out in the previous post, that the purpose of publication is to pad vitae and to impress potential research funders; note how the authors emphasize that their findings are novel: “Our novel findings”; “this study’s novelty”; “novel findings”; “new risk group”; and “Future research” will need to identify why health perceptions differ by sexual orientation, calling for more data to be gathered “to identify possible factors, such as discrimination, social support, coping, or the patient-physician relationship”.
Clearly this sets the stage for enough grants to last several working life-times (if one cares to dignify this activity as work). Moreover, future exercises along these lines are guaranteed to bear fruit. After all, it is too superficial to analyze the data only in terms of overall sexual orientation. Obviously the category of gay men ought to be sub-divided into those who always take the passive role, those who always take the active role, and those who switch hit; similarly with lesbians, there needs to be a sub-division into the three analogous groups; and the category of bisexual women, too, should be sub-divided into those who are bisexual lifelong and always, those who begin as heterosexual and later recognize their true inclinations, those who begin and change in the opposite way, and those who make such a change back and forth more than once. In other words, it is easy to identify perhaps as many as a couple of dozen variables worth “studying”; and it can be guaranteed that some startling correlations, really novel “findings”, will emerge if correlations are sought among another dozen or two variables; it’s money in the bank.
The immediate news release about the Boehmer article, published only on-line and awaiting print publication, illustrates the publicity-hunting ethos of contemporary researchers and the determined efforts of the journalists to get there first with everything, thereby flooding the media with untested claims that turn out to be chiefly rubbish, since real science — reliable science — isn’t news.
Policy makers and research funders would do well to ignore authors who praise the novelty of their own findings, especially when those findings are merely the result of indiscriminate data-mining; and they should ban all “statistical” research that relies on frequentist rather than Bayesian approaches, see R A J Matthews, “Facts versus Factions: The use and abuse of subjectivity in scientific research”, in Rethinking Risk and
the Precautionary Principle (Ed: Morris, J.) (Oxford : Butterworth) 247-282 (2000).