This is a tongue-in-cheek examination of a real study that illustrates how bad science can be done while looking credible, getting published, and giving erroneous results. The facts are all true, but do not try this at home.
How to Design a Really Bad Study
of the Feingold Diet
In our first lesson, we will review the Gross et al study, done in 1987, and published in the Journal of American Academy of Child and Adolescent Psychiatry, 26, 53-55.
The effect of diets rich in and free from additives on the behavior of children with hyperkinetic and learning disorders.
- click to see abstract & full text
First, it must be stressed that studying 39 children in a summer camp is a brilliant idea - you have them under your complete control and you actually have the opportunity of doing a great study. So pay attention, now, to make sure you don't. Here is how the Gross et al team did it:
This study was intended to test the use of the Feingold diet for hyperactive children. However, only 18 of the children in the study were hyperactive, and 19 of them were not. Since all results would be averaged - combined - this leaves little chance of seeing a diet reaction should one (gasp) occur.
- Of the 18 hyperactive children, 17 were on medications (with coloring). They stayed on their meds during the study; thus, since their behavior was theoretically under good medical control, it should be next to impossible to know if the diet actually affected them.
That left only ONE valid child to study. He made it through the first week, but left "for homesickness" the second week. Could his distress have been caused by the food dyes added the second week? Don't even think about it.
- Another two children were sent home the second week (the "additive-rich" week) for misbehavior. Now, misbehavior is one of the indicators of increased hyperactivity, so make sure to send home anybody who might actually be reacting to your additive challenge.
- Gross used an unpalatable (his own words) experimental "Feingold" diet. He allowed no condiments, not even those permitted by the Feingold Program, and to explain the lack of desserts and snacks, he told the children, "the good stuff didn't come yet." Being unpalatable is certainly not a diet requirement, but making sure the kids hate the diet is always useful when you want to do a Really Bad Study.
- He ignored additives in toothpaste, medicine, cleaning supplies, etc., so the children were certainly exposed to at least some artificial colorings, flavorings, and petroleum-based fragrances throughout the "additive-free" week.
- The actual experimental "Feingold" diet lasted only ONE WEEK. It usually takes a week on the Feingold diet for any improvement to begin to appear. This is a brilliant idea - something like being on a weight-loss diet from breakfast to lunch.
The justification for a one-week diet was that when children eat items forbidden by the diet, parents report a reaction within minutes to hours. The researchers carefully ignored the detail that one first has to have had an actual response to the diet, which takes several days to several weeks, depending on age, circumstances, errors, etc.
- To track the results of their study, they videotaped the children for 4 minutes while they were eating. Remember, the children were filmed during the meal, whereas any effect from what they ate would have taken place after the meal.
The people analyzing the 4-minute films were supposed to be "blind" to whether the children were on the experimental diet during any particular segment. Gross himself was one of these "blind" film analysts, and his results were averaged with the others. Since this was a mealtime, food was presumably visible in the film. How could the PRIMARY INVESTIGATOR possibly consider himself "blind to the diet order?" Well, if you do that in your own Really Bad Study, make sure to slip it in without comment, as he did.
- The person filming the children had control over what was filmed. This is important to a Really Bad Study - always make sure somebody NOT BLIND controls what the "blind" analyzers see.
Considering the design of this study, any actual response to the one-week experimental diet would be astonishing.
Finally, always make sure to have an excuse in case somebody actually does respond to your experimental diet in spite of all your precautions.
- One hyperactive, medicated, boy did become "more boisterous during the second (additive-rich) week, and was sent home on the 12th day." The authors did not conclude that the additives bothered him, but that his dosage of Cylert had become suddenly "inadequate."
- The camp director and all teachers at the camp "felt that during the second week the children were noisier and more active." This dangerous assertion was nullified by getting them to agree that "normal" kids are also noisy at summer camp.
Abstract: Thirty-nine children in a summer camp were given the Feingold Diet, which eliminates artificial additives and salicylate-containing foods, for 1 week, followed by administration for 1 week of food containing those ingredients. The behavior of all children was monitored by videotape for 4-minute intervals at mealtime. All children were classified by public school psychologists as having moderate to severe learning disorder; 18 were also hyperkinetic, and 17 were taking medication for the latter condition. Three raters blind to the respective diets the children were on rated the children's behavior for motor restlessness, disorganized behavior, and misbehavior. No significant differences were found in behaviors during weeks 1 and 2. The authors conclude that the Feingold Diet has no beneficial effect on most children with learning disorders, or on hyperkinetic children taking medication.
Full text || Get Password || BACK