|
March 2011
Your Survey Savvy Checklist: Things to
Learn about a Survey
Sponsorship of the Survey
Purpose
Sample Description and Size
Dates of Data Collection
Who Conducted the Study
Exact Question Wording
A previous Readex Review article highlighted one way to increase your survey savvy. Giving a little extra thought about the statistics you stumble upon can help you determine how much confidence to place in them. Heres a recent example of how this process can work.
A variety of news outlets recently reported research results from the Northern Manhattan Study released by the University of Miami Miller School of Medicine and made the shocking declaration that diet soda drinkers may at increased risk for vascular events, or strokes, over people who dont drink soda at all.
In this case, and whenever research flies in the face of conventional wisdom, statistic savvy begins by getting an idea of the research studys specifics. Depending on the news outlet, these details may or may not have been provided. When details arent available in the article, a simple search can usually lead you to the original release that the media used to create their reports. In the case of our diet soda findings, some articles featured criticism regarding the research that identified flaws with the sample and the way the data was collected. So, who is a non-researcher to believe?
Heres a situation where a little survey savvy will go a long way in being able to confidently determine whether you should drop your diet soda habit or stand-by for further information. As an example of how you can quickly evaluate research findings, well use the checklist about how to be a good research consumer offered in previous articles.
The release from the Miller School's website is excerpted below on the left.
Look to the right of the highlighted areas to get an idea of one way to synthesize the information in order to base judgment.
Miller School Researchers Link Diet Soda and Salt to Cardiovascular Risk
Two new studies by University of Miami Miller School of Medicine researchers link diet soda and salt to increased risk of vascular events, including stroke. The research was presented today during the American Stroke Associations International Stroke Conference 2011 in Los Angeles.
In the first study, the scientists found that if you drink diet soda C instead of the sugar variety C you could still have a much higher risk of vascular events compared to those who do not drink soda. In findings involving 2,564 people in the large, multi-ethnic Northern Manhattan Study, the researchers said people who drank diet soda every day had a 61 percent higher risk of vascular events than those who reported no soda drinking.
"If our results are confirmed with future studies, then it would suggest that diet soda may not be the optimal substitute for sugar-sweetened beverages for protection against vascular outcomes," said Hannah Gardener, Sc.D., lead author and epidemiologist in the Department of Neurology at the Miller School.
...Information removed about the high salt intake study...
At the start of both studies, researchers assessed diet by a food frequency questionnaire.
The Northern Manhattan Study is a collaboration of investigators at the Miller School and Columbia University in New York launched in 1993 to examine stroke incidence and risk factors in a multi-ethnic urban population. A total of 3,298 participants over 40 years old (average age 69) were enrolled through 2001 and continue to be followed. Sixty-three percent are women, 21 percent are white, 24 percent black and 53 percent Hispanic.
In the soda study, researchers asked subjects at the outset to report how much and what kind of soda they drank. Based on the data, they grouped participants into seven consumption categories: no soda (meaning less than one soda of any kind per month); moderate regular soda only (between one per month and six per week); daily regular soda (at least one per day); moderate diet soda only; daily diet soda only; and two groups of people who drink both types: moderate diet and any regular, and daily diet with any regular.
During an average follow-up of 9.3 years, 559 vascular events occurred (including ischemic and hemorrhagic stroke, which is caused by rupture of a weakened blood vessel). Researchers accounted for participants age, sex, race or ethnicity, smoking status, exercise, alcohol consumption and daily caloric intake. And even after researchers also accounted for patients metabolic syndrome, peripheral vascular disease and heart disease history, the increased risk persisted at a rate 48 percent higher.
...Information removed about the high salt intake study...
At the start of both studies, researchers assessed diet by a food frequency questionnaire. The fact that participants reported their dietary behavior is a key limitation of both studies, Gardener said.
In the soda study, investigators also lacked data on types of diet and regular drinks consumed, preventing analysis of whether variations among brands or changes over time in coloring and sweeteners might have played a role.
Miller School co-authors for the soda and sodium studies are Tatjana Rundek, M.D., Ph.D., associate professor of neurology and vice chair for clinical translational research; Clinton Wright, M.D., M.S., associate professor of neurology and scientific director of the Evelyn F. McKnight Brain Institute; and Ralph L. Sacco, M.D., M.S., professor and chair of neurology and president of the American Heart Association 2010-2011.
Funding for both studies was provided by a Javits award from the National Institute of Neurological Disorders and Stroke and the Evelyn F. McKnight Brain Institute.
Uncovering Details from
the Checklist Above
Here's the finding that had been reported by a variety of news outlets.
This quote, from the lead author and epidemiologist in the Department of Neurology and the university that conducted the study, includes the clause, "If our results are confirmed with future studies..." indicating the results shouldn't be taken as unconditional.
This is a hint toward when data collection occurred, but does not provide details.
The Miller School and Columbia University conducted the study. The fact that a university conducted the study lends credibility to
its objectivity. Its purpose: to examine stroke incidence and risk factors in a multi-ethnic urban population.
The sample description and size is described. Participants were enrolled over
eight years from a localized population and weren't derived from a sample. The group, and therefore the results,
do not represent the general U.S. population.
As far as the dates of data collection, specifics aren't available. These details indicate that participants self-reported their soda intake at a single point in time via questionnaire.
Following an extended amount of time, an average of almost a decade, researchers looked at the occurrences of vascular events.
Although researchers accounted for a variety of factors, diet soda consumption at one moment in time is the basis of the findings and doesn't take into account the decrease or increase in consumption that may have occurred in the years that followed data collection.
While the specific wording of questions isn't available, this release points out that behavior was self‑reported. However unintentional, people don't always accurately report their own behaviors.
Finally, as mentioned in the findings, the report is general in nature despite the variety of soft drinks available to consumers.
The sponsor is identified and does not indicate a conflict of interest.
|