Reading and Interpreting Research Reports

Reading and Interpreting Research Reports

I have been reading the research on Diet for almost forty years. Over that time, I have found that researchers interpret research data according to their preference for the outcome of the research. The “findings and conclusions” are tilted toward the researcher’s biases. Very much like the divide between liberals and conservatives in politics, dieting researchers belong to either the “Low Fat” or the “Low Carbohydrate” group and interpret their data accordingly.

For the past thirty years, the FDA committee that writes the “Dietary Guidelines for Americans” has produced their recommendations every five years. In looking at the Guidelines, the committee belongs to the low-fat camp. I have read their reports which rely heavily on epidemiological reports with some clinical trials.

For those who know little about epidemiological studies of diet, the researchers collect data through phone calls to selected people and ask them what they have eaten over the past week. People are asked their gender, race, ethnic group, what their current health concerns are, what foods and beverages were consumed over the past week, how much and how many times. (Data collection can last for decades producing interim reports of findings.)

From the thousands of answers to these questions, the researchers do a statistical analysis of the data based upon pre-set questions. These are then categorized according to the breakdown of known nutrients: fat, carbohydrate, protein, sugars, vitamins and minerals. This data is then re-categorized by the other factors of age, gender, and health issues.

Researchers look to see which variables seem to influence which variables. When reporting their research (which has an enormous amount of data), the researchers can have selective inattention. They may report the data that supports their diet belief system and ignore all the data that does not agree with their POV. Since no one else sees the data other than the research team, there is no one to cry “foul.”

The results are published and shared with the media. What the media does not understand about epidemiological studies is that the findings only show a statistical association between two or more bits of information. Epidemiological studies can never say “this variable causes this other variable to change.” So, basing the Dietary Guidelines on epidemiological data primarily is like building a house on shifting sand.

Diet researchers also report on their clinical trials/experimental studies. These studies are the gold standard in research because they are so tightly controlled. The researchers must have some reasons (previous research findings) for including a certain variable and then deciding how it is to be studied and how long it will be studied. In diet research, a thorough knowledge of the physiology of processing food is critical. If the research is trying to compare a low-fat diet to a low-carbohydrate diet, the researcher must know that there are differences in how the genders process foods and age also affects food processing. Therefor a good study will select only one gender and age group to study. (If a study includes both genders and a range of age groups, then all the data must be compartmentalized by these variables. The results are only valid for specific gender and age groups. They need knowledge on how long it takes the human body to adapt to a change in diet, how different foods trigger insulin production, store foods, breakdown foods, are eliminated without being absorbed. This knowledge is critical to creating a research project that is both reliable and valid. If the project is too short, the findings are invalid. If they prepare foods in certain ways, they lose nutrients, which will influence the findings. The gold standard for dieting research is one year. The usual results looked for are: amount of weight loss, changes in body measurements, blood values such as cholesterol levels and blood glucose levels. Even more specific tests can reveal the action of food on subjects.

Since 2007, there have been publications challenging the research upon which they have based the US Dietary Guidelines. (To date, these studies have not been considered by the committee producing these Guidelines. It does not list them in the references the committee says it used to produce the Guidelines.)

Two publications are worth noting. Both published in 2007. The first is the monumental literature review of dieting research and politics by Gary Taubes (Good Calories, Bad Calories). The second is the publication of the A to Z study by Gardener et al. Gardener was a member of the low-fat group. He and his team at Stanford, designed an experiment that compared four different diets: Atkins, Dietary Guidelines, The Zone and ultra-low-fat diet by Dean Ornish. They wanted to know which diet his overweight over-forty women would stick to the longest and which diet showed the greatest weight loss, the best blood values and stayed on it longest. To Gardener’s surprise, The Atkins group lost the most weight, stuck to it the longest and had the best blood values. In contrast, the low-fat group gave up the quickest, lost the least amount of weight and had the worst blood values. The committee publishing the US Dietary Guidelines has consistently ignored these two publications.

A recent publication, belonging to the low carbohydrate group, by Volek and Phinney (2011) is adding to the research data on low-carbohydrate diets. They have conducted many clinical trials/experimental designs (with not much success at getting published in prestigious journals that espouse the low-fat belief system). What they have found, which flies in the face of previous research, is that exercise does not contribute to weight loss in many people (although it is important for fitness reasons), and that people with insulin resistance do well on a low-carbohydrate diet, while people with insulin sensitivity do well on low-fat diets (finding by Gardener et al.). This tells us that the dichotomy of low-fat versus low-carb isn’t the answer. It depends on the physiology of each individual person. One size does not fit all. To my mind, this is a breakthrough in dieting research. This publication is also ignored by the committee producing the Dietary Guidelines for Americans.

Why does the committee insist upon restating the low-fat mantra despite the research showing that it is palpably false? Two answers are probable. Either the committee cannot see that they have a low-fat bias which skews their report, or there are vested interests that have convinced the committee to stand fast. Until the Medical Research Council funds a large scale study comparing the diets–similar to the Gardener et al. study–there will be no changes in the Dietary Guidelines for Americans.