March 12, 2015

Presenter:

Andrew W. Brown, PhDNutrition & Obesity Research Center - Office of Energetics

University of Alabama at BirminghamModerator:

James M. Rippe, MD – Leading cardiologist, Founder and Director, Rippe Lifestyle Institute

Approved for 1 CPE (Level 2) by the Commission on Dietetic Registration, credentialing agency for the Academy of Nutrition and Dietetics.

NUTRI-BITES®

Webinar Series

In the Eye of the Beholder:Critical Evaluation of Nutrition

Research

Celebrate National Nutrition Month!

Thank you Registered Dietitian

Nutritionists and Dietetic Technicians,

Registered for all you do!

ConAgra Foods Science Institute With a mission of:

Promoting dietary and related choices affecting wellness

by linking evidence-based understanding

with practice

Webinar logistics CEUs – a link to obtain your Continuing Education Credit

certificate will be emailed and available on this webinar’s page at www.ConAgraFoodsScienceInstitute.com within 2 days.

A recording of today’s webinar, slides, and summary PowerPoint will be available to download as a PDF within 2 days at: www.ConAgraFoodsScienceInstitute.com

The presenter will answer questions at the end of this webinar. Please submit questions by using the ‘Chat’ dialogue box on your computer screen.

Today’s Faculty

Andrew W. Brown, PhDNutrition & Obesity Research CenterOffice of EnergeticsUniversity of Alabama at Birmingham

Moderator:James M. Rippe, MD – Leading cardiologist, Founder and Director, Rippe Lifestyle Institute

Learning Objectives

Review potential influences of biases in nutrition research Discuss the importance of critically evaluating new research

(i.e. whether it confirms or refutes standard clinical practice or commonly held beliefs)

Describe steps to minimize misinterpretation of research Identify strategies health professionals can use to objectively

translate scientific knowledge to clinical practice

NUTRI-BITES®

Webinar Series

Critical Evaluation of Nutrition Research

In the Eye of the Beholder: Critical Evaluation of Nutrition Research

Andrew W Brown, Ph.D.2015, 03-12

awbrown@uab.edu

I am involved with research funded by the National Institutes of Health and the non-profit Coca-Cola Foundation. UAB has received funding or gifts from numerous sources.

Acknowledgments and Disclosures

David B AllisonEd ArcherMichelle M Bohan Brown

Patrice CapersKathryn A KaiserDwight Lewis

Full citations and slides are available upon request.

The contents of this presentation represent my views and do not necessarily reflect the views of UAB or my colleagues.

Colleagues who may have helped with slides

• How do we know about nutrition?• Critically evaluating research to minimize

misinterpretation• What exactly was studied?• How exactly was it studied?• How does that compare to how it was communicated?

• Translating science to clinic or policy

Outline

How do we ‘know’ things in Nutrition Science?

Wesleyan QuadrilateralReason

Scripture

Tradition Experience

Nutrition Quadrilateral

Research

How do we ‘know’ things in Nutrition Science?

Reason

Tradition ExperienceThe quadrilateral requires of a [nutrition scientist] no more than what he or she might reasonably be held accountable for: which is to say, a familiarity with [scientific literature] that is both critical and faithful; plus, an acquaintance with the wisdom of [nutrition science history]; plus, a taste for logical analysis as something more than a debater’s weapon…

adapted from Outler. Wesleyan Theological Journal. 1985;20:1,p17

Word Definition 1 Definition 2Nutritive Containing micronutrients Containing calories

Significant Important P<0.05Reduce Decrease Gaining electronsToxicity Acutely hazardous Capacity to cause harm

Uncertainty Anything is possible Constraining the knownDiet Restrictions on eating

behaviorsEating behaviors

Bias Mathematical deviation from true results

Human distortion

Are we using the same language?

How Researchers Define Snacks and Meals• By time: 8-10AM, 12-2PM, and 6- 8PM = meals; Other times = snacks

• By food composition/type: Based on ‘taxonomy’ of food, or calories in eating occasion

(Gregori et al, 2011; Gregori, & Maffeis, 2007)

Abstract concepts

How Researchers Define Snacks and Meals• By time: 8-10AM, 12-2PM, and 6- 8PM = meals; Other times = snacks

• By food composition/type: Based on ‘taxonomy’ of food, or calories in eating occasion

(Gregori et al, 2011; Gregori, & Maffeis, 2007)

How Individuals Define Snacks and MealsMeal Related-Perceptions Snack Related-Perceptions

Eating with family vs. Eating alone

Cloth napkin vs. Paper napkin

Sitting while eating vs. Standing while eating

Expensive vs. Inexpensive

Prepared food vs. Packaged food

‘Healthy’ food vs. ‘Unhealthy’ food(Adapted from Wansink et al, 2010. Appetite. 54(1), 214-16)

Abstract concepts

Euphemisms for ‘Bad Foods’ Over Time

“The common fruits, because of their low nutritive value, are not, as a rule, estimated at their real worth as food.” – USDA Farmer’s Bulletin, 1917

Custom searches of

Coun

ts/Y

ear

Food

Nebulous Descriptors of Food

“We encourage parents to stop serving cheese, meat, and other junk foods to children…” – Physicians Committee for Responsible Medicine, 2012

How we each see nutrition

A CornB Alfalfa sproutsC Hot dogsD SpinachE PeachesF BananasG Milk chocolate

Assume you are alone on a desert island for one year and you can have water and one other food. Pick the food that you think would be best for your health (never mind what food you would like). Check the food you would pick.

How we each see nutrition

0

10

20

30

40

50

Corn Alfalfasprouts

Hot dogs Spinach Peaches Bananas Milkchocolate

% o

f Res

pond

ents

StudentsPhysical plant workersNational sampleFacultyStudents (+oranges)

• How do we know about nutrition?• Critically evaluating research to minimize

misinterpretation• What exactly was studied?• How exactly was it studied?• How does that compare to how it was communicated?

• Translating science to clinic or policy

Outline

What exactly are we comparing?

orThe Tale of Two Cheese Sandwiches

orThe Tale of Two Cheese Sandwiches

What exactly are we comparing?

“Whole” Food Processed Food

Bread Multi-grain bread with whole sunflower seeds and whole-

grain kernelsWhite bread

Cheese Cheddar cheese Processed cheese product

Fat 17.5 g 14.5 g

Protein 20 g 15 g

Carbohydrates 40 g 49.5 g

Sandwich 2 slices of bread2 slices of cheese

3 slices of bread2.28 slices of cheese

Differential weight (lbs) from age 6-12 (6 years)kcal/meal 485 550 600

Model 3500 kcal Hall et al. 3500 kcal Hall et al. 3500 kcal Hall et al.Fast food/wk

1-3 23.4 0.5 12.0 0.2 3.0 0.128+ 327.0 6.5 165.0 3.3 40.2 0.8

Are assumptions adequate?

Original paper PMID: 24304430Brown et al. Child Obes. 2014 Dec;10(6):544-5

Modeling potential effects of reduced calories in kids' meals with toy giveaways

VSOrder of magnitude

misestimation of weight effects of children’s meal policy proposals

Differential weight (lbs) from age 6-12 (6 years)kcal/meal 485 550 600

Model 3500 kcal Hall et al. 3500 kcal Hall et al. 3500 kcal Hall et al.Fast food/wk

1-3 23.4 0.5 12.0 0.2 3.0 0.128+ 327.0 6.5 165.0 3.3 40.2 0.8

“The purpose of this collection is to make information on obesity prevention policies and the underlying evidence base easily searchable and accessible.”

Are assumptions adequate?

Original paper PMID: 24304430Brown et al. Child Obes. 2014 Dec;10(6):544-5

Modeling potential effects of reduced calories in kids' meals with toy giveaways

VSOrder of magnitude

misestimation of weight effects of children’s meal policy proposals

Percent under (BLUE) or over (RED) estimation of dietary components with semiquantitative FFQ

Bold/outlined cells significant at p<0.05

DIETUsual American NCEP Step II Very Low Fat

Energy -20 -18 -23Fat -36 -17 10SFA -49 64 7

MUFA -42 -31 23PUFA -28 -50 29

Cholesterol -28 108 -8Carbohydrate -11 -18 -28

Protein 1 -7 -21Fiber -18 -32 -28

Fat (% EI) -19 2 42SFA (% EI) -35 113 46

MUFA (% EI) -27 -13 59PUFA (% EI) -12 -39 72

Carbohydrate (% EI) 11 -1 -6Protein (% EI) -27 13 0

Are the methods good enough?

Sweet

Starchy

Healthy

Ratio of EI to TEEMean (95% CI)

Difference Between EI and TEEMean (95% CI)

TEE = Total Energy Expenditure, measured by doubly labeled waterEI = Energy Intake estimated by FFQDietary patterns determined by cluster analysis of FFQ

Are the methods good enough?

0

0.2

0.4

0.6

0.8

1

EI/T

EE

-8

-6

-4

-2

0

EI-T

EE (M

J/d)

0

1000

2000

3000

4000

Normalweight Overweight

kcal

/d

Self-reported and Observer-estimated Energy Intake

1 day food record Research dietary history “It appears, therefore, that, unless special precautions are applied to the study of the fourth of the adult population which is overweight, any data collected on the caloric intake of populations by the record method is likely to be an underestimate.”

Are the methods good enough?

JADA, January 1953

“We … offer the contrary view that [self-report measures of EI] are so poor as measures of actual EI … that they no longer have a justifiable place in scientific research aimed at understanding actual EI...” – N.V. Dhurandhar et al., Int J Obes (Lond). 2014 Nov 13

0

1000

2000

3000

4000

Normalweight Overweight

kcal

/d

Self-reported and Observer-estimated Energy Intake

1 day food record Research dietary history “It appears, therefore, that, unless special precautions are applied to the study of the fourth of the adult population which is overweight, any data collected on the caloric intake of populations by the record method is likely to be an underestimate.”

Are the methods good enough?

JADA, January 1953

Dietary Guidelines for Americans2015 Dietary Guidelines Advisory Committee Report

“Most of the DGAC data analyses used the National Health and Nutrition Examination (NHANES) data and its dietary component, What We Eat in America (WWEIA)” (Appendix E-4)

Many recommendations to expand questionnaires.

What do we do with these problematic data?

I Properly randomized controlled trial.II–1 Well-designed controlled trials without randomization.II–2 Well-designed cohort or case-control analytic studies.

II–3 Time series with or without the intervention.III Opinions.

Hierarchy of research designs

Media High Impact Journals

Bias in Media Exposure

I17%

II-17%

II-268%

II-33%

III5%

I40%

II-10%

II-251%

II-31%

III8%

Media Cover Inferior Study Designs

What’s Being Published?

0

10,000

20,000

30,000

40,000

201520051995198519751965

Nutrition-Related Studies: 886,658In humans: 433,131

RCTs: 31,848

Studies indexed in PubMed

Spin perpetuates throughout the reporting

Spin: specific reporting strategies, intentional or unintentional, emphasizing the beneficial effect of the experimental treatment

Bias Presented to the Public

Abstracts were categorized based on results and conclusions about breakfast and obesity

Breakfast was more likely to be mentioned in conclusions if results were pro-breakfast (p=0.0492)

Biasing Interpretations of Own Results

Brown A W et al. Am J Clin Nutr 2013;98:1298-1308

Selective Reporting

22%

78%

Not Pro-Breakfast Results

65%

35%

Pro-Breakfast Results

Mentioned in Conclusions

Not mentioned in conclusions

Stubborn Preconceptions

“Our findings do not lead to conclude that SSB replacement with water does not support reduction of metabolic syndrome risk factors but rather that intervention provided was ineffective in reducing other sugary beverage intake.”

Meeting abstract presented at Experimental Biology:

Stubborn Preconceptions

“Our findings do not lead to conclude that SSB replacement with water does not support reduction of metabolic syndrome risk factors but rather that intervention provided was ineffective in reducing other sugary beverage intake.”

“… the [water] group increased water intake and decreased SSB intake significantly over time…”

-252 -115

Intervention Control

kcal

/d

9 month change in 'Beverages with Sugar'

Substituting water for sugar-sweetened beverages reduces circulating triglycerides and the prevalence of metabolic syndrome in obese but not in overweight Mexican women in a randomized controlled trial.

PMID: 25332472

Meeting abstract presented at Experimental Biology:

Final published paper in the Journal of Nutrition

Title:

CONCLUSIONS: “These schoolchildren are exposed to an obesogenic environment, and it is not surprising that in this situation, many of these children are already overweight and will likely become obese as adults.”

RESULTS: “Based on our observations, it appears that those who have higher BMIs are less likely to consume fast food as often.”

Conclusions not matching results

PMID:22721691

• How do we know about nutrition?• Critically evaluating research to minimize

misinterpretation• What exactly was studied?• How exactly was it studied?• How does that compare to how it was communicated?

• Translating science to clinic or policy

Outline

Simple terms for complex concepts

Which of the following has the greatest number of empty calories?

A 5 ounces of table wineB 2 x 1 ounce patties of pork sausageC 3 ounces of roasted chicken thigh with skin (cooked weight)D 1 cup of frozen yogurtE 3 ounces of regular, 80% lean ground beef (cooked weight)F 1 medium, 2 ounce croissantG 1 cup of fruit flavored, low-fat yogurtH 1 small, 2 ounce blueberry muffinI 3 x 1 ounce slices of beef bologna

Food Total Calories

Empty Calories

Percent Empty

A 5 ounces table wine 121 121 100B 2 x 1 oz. patties of pork sausage 204 96 47C 3 oz. roasted chicken thigh w/ skin 209 47 22D 1 c. frozen yogurt 224 119 53E 3 oz. regular, 80% lean ground beef 229 64 28F 1 medium, 2 oz. croissant 231 111 48G 1 c. fruit flavored, low-fat yogurt 250 152 61

H 1 small, 2 oz. blueberry muffin 259 69 27I 3 x 1 oz. slices of beef bologna 261 150 57

Simple terms for complex concepts

http://www.choosemyplate.gov/weight-management-calories/calories/empty-calories.html

Simple terms for complex concepts

http://www.choosemyplate.gov/weight-management-calories/calories/empty-calories.html

Age and gender Total Empty PercentFemales 31-50 yrs 1800 160 9%

Males 31-50 yrs 2200 265 12%

Food Total Calories

Empty Calories

Percent Empty

A 5 ounces table wine 121 121 100B 2 x 1 oz. patties of pork sausage 204 96 47C 3 oz. roasted chicken thigh w/ skin 209 47 22D 1 c. frozen yogurt 224 119 53E 3 oz. regular, 80% lean ground beef 229 64 28F 1 medium, 2 oz. croissant 231 111 48G 1 c. fruit flavored, low-fat yogurt 250 152 61

H 1 small, 2 oz. blueberry muffin 259 69 27I 3 x 1 oz. slices of beef bologna 261 150 57

Adapted from: Brown A W et al. Am J Clin Nutr 2013;98:1298-1308

Being cognizant of our own humanity

Food X is Bad!

Food X is Bad!

Food Xis

BAD!

Food X is OKAY

0

20

40

60

80

100

0 50 100

Cert

aint

y (%

)

Studies

Strength of BeliefStrength of Evidence

Discarded information

Often impossible to tell if something ‘worked’ for an individual (e.g., responders vs non-responders)• Improvements could have been spontaneous• Improvements may have been better with another

option• Impairments may have been mitigated

Anecdote vs Data

Often impossible to tell if something ‘worked’ for an individual (e.g., responders vs non-responders)• Improvements could have been spontaneous• Improvements may have been better with another

option• Impairments may have been mitigated

Scientific investigation tells us whether, on average, a group does better under one condition than another• RCTs in particular tell us whether a change in

exposure causes a change in outcome

The plural of anecdote is not ‘data’

Anecdote vs Data

Nutrition Quadrilateral

Research

Clinic, Policy, or Science?

Reason

Tradition Experience

www.obesityandenergetics.org

www.healthnewsreview.org/

Resources

www.senseaboutscience.org

www.nhs.uk/News/

In the Eye of the Beholder: Critical Evaluation of Nutrition Research

Andrew W Brown, Ph.D.2015, 03-12

awbrown@uab.edu

Questions?

Based on this webinar the participant should be able to: State potential influences of biases in nutrition research Discuss the importance of critically evaluating new research

(i.e. whether it confirms or refutes standard clinical practice or commonly held beliefs)

Describe steps to minimize misinterpretation of research Identify strategies health professionals can use to objectively

translate scientific knowledge to clinical practice

NUTRI-BITES®

Webinar Series

Critical Evaluation of Nutrition Research

ConAgra Foods Science Institute Nutri-Bites® Webinar details

A link to obtain your Continuing Education Credit certificate will be emailed within 2 days

Today’s webinar, including certificate link, will be available to download within 2 days at: www.ConAgraFoodsScienceInstitute.com

For CPE information: acontinelli@rippelifestyle.com

Recent CEU webinars archived at the ConAgra Foods Science Institute website: Functional Foods: Phytochemicals – Hidden Nutrition Gems Sodium: Too much, too little or just right? A Decade of Nutrigenomics: What Does it Mean for Dietetic Practice? Ethics for All: Applying Ethics Principles across the Dietetics Profession Sports Nutrition: The Power to Influence Exercise Performance Culinary Competency to Enhance Dietetic Practice

Nutrition Support for the Bariatric Surgery Patient: When and Why Nutrition Support is Needed

Kellene A. Isom, MS, RD, LDNBariatric Program Manager and Senior Bariatric Dietitian

ANDKris M. Mogensen, MS, RD, LDN, CNSC

Team Leader DietitianBrigham and Women's Hospital

Boston, MA

Date: May 6, 20152-3 pm EDT/1-2 pm CDT

www.ConAgraFoodsScienceInstitute.com

NextConAgra Foods Science Institute Nutri-Bites® Webinar

How are we doing?

Stay on the line for a brief survey about today’s ConAgra Foods Science Institute Nutri-Bites® webinar:

Thank you!

Critical Evaluation of Nutrition Research