Rats, Tumors and Critical Assessment of Science

My email box exploded with new messages.  A flurry of notes contained a link to a new peer-reviewed paper, a work showing that rats fed “GMO” corn developed massive tumors and died early, compared to controls.  Immediately I smelled a Seralini paper.

A click on the link did not disappoint-- it's Seralini again.  I was electronically whisked to a PDF of the whole text and began to read.  Within minutes I was blown away by the lack of rigor, poor experimental design, attention to controls and loose statistics.  Most of all, I was blown away by the conclusions drawn by a study with tiny numbers of subjects in a rat line known to grow endochrine tumors.

The anti-GMO interests were quick to anoint this new work as a rigorous pillar of exceptional science, a hard-science detailing of the danger of transgenic food.  They want this to influence public policy.

I was really impressed by how the scientific media and the science blogosphere pounced.  The best names in the business, Terwavas, Leyser, Goldberg and many others were interviewed and provided detailed analysis of the work, pointing out its many flaws.  Those reviews can be foundthroughout the internet, and they are awesome. Like this one! I don’t need to reiterate them here.

What I will do, which is highly uncharacteristic and but consistent with the post hoc analysis done all the time, is provide a level of analysis that was not explored.  There are features of this paper that hint at a motive, an intent.  I do not believe this was a hypothesis tested.  I believe that this was an experiment designed to frighten.  I believe that this is blatant mis-use of science to forward an agenda.

Those are strong words and I never thought I’d cast such allegations at someone else’s peer-reviewed research.  That’s usually pretty low.  However, there are facets of this work that are clearly indicate the intent of the authors is to provide shock, not a good test of a hypothesis.  In fact, the word “hypothesis” does not appear once.  

This is why the report is in Food and Chemical Toxicology and not in Nature, where it would be if it was a properly conducted study.

Here are some red flags the others have not mentioned.  I’m reading between the lines here. I will describe what a good scientific report should not do and then give you some strong inferences from what the paper does not show, as well as how data are presented.

1. The first line of the paper claims an “international debate”, yet he cites himself and nobody else.  Easy to claim a debate when nobody else is participating in it.

2.  Figure 3.  This one really makes me see red.  Look at tumors.  Look at massively deformed rats.  Shocking, isn’t it?   The authors tell us in Table 2 that control rats also develop tumors.  Why not show them?  Why are the controls not shown in that figure?  It is because if they are identical to the experimental treatment rats then the fear factor is gone.   This is inexcusable and the authors, reviewers and editors should be ashamed.

Sometimes the way data are presented can expose the relative objectivity and hidden intent of a study. Left-rat that ate GMO corn.  Center- rat eating GMO corn and roundup. Right- rat fed roundup. Their associated tumors shown on the right. Wait!  What about the control rats, the ones that also got tumors?  How convenient to leave them out!   

3.  The labeling on the figure is “GMO” or “GMO+R” (R stands for Roundup).  GMO is not a product. It is not a genetic line of corn.  It is a technique.  There are many kinds of GMOs, plant lines bearing different transgenes.  Even if these results linked rat tumors to the food (which they don’t in my assessment) they would  link it to one kind of transgenic crop, not any transgenic crop.  This again shows the authors’ intent to overstep the data in a manner that will inflame the reader and further vilify a technology. To be fair, they do state it properly in the conclusion, but few are reading past the sensational photos.

4.  They show comparable effects of Roundup treatment and the transgene.  This should be a tip-off as well.  What is the likelihood of both inducing identical problems?

5.  Low numbers of subjects are a sign of poor design.  When tumor incidence is 30%, vs 50% or 70% that means three rats vs. five or seven.  The incidence of endocrine tumors in Sprague-Dawley rats is 70-80%.  Imagine you roll a die and numbers 1-4 mean develop tumors, 5 and 6 mean tumor free.  Now roll it ten times and log the result.  You’ll find that there will be times when you consistently roll 5 or 6, maybe 5 times out of ten.  Other times you’ll roll 5 or 6 only 2 times out of ten.  That’s natural random variation, and if you roll it 100 times, 1000 times, then the real probabilities will even out. 

6.  Low numbers + a line known to get tumors = some frequency of data that will prove the authors’ beliefs.

7.  A prediction-- the larger study will never be done and these results will not replicated by other labs.

8.  The Discussion.  Lots of guesses on how to link the food or Roundup to the symptoms. Quite a bit of speculation and hand waving, with no likely mechanisms discussed.

I could go on all day. For fun reading review the press conference. It was a bigger joke.  

The bottom line is that if we look at the report and what it says, and compare it to what the data really say, there is limited concordance.  To the trained eye the data say that these rats get endocrine tumors at high incidence and that what is being observed is the natural variation of the tumors in small numbers of rats, where the authors'  “significance” is found in statistically meaningless samples.

Alas, it is now part of the true-believers' war chest of crap information that now will be used to steer the unsophisticated and influence public policy.