Beth St. Jean

Mega Subramaniam

Natalie Taylor

Rebecca Follman

Christie Kodama

Dana Casciotti

“BECAUSE I JUST KNOW THINGS AND I’M SURE!”: HOW SOCIOECONOMICALLY

DISADVANTAGED TWEENS’ PRE-EXISTING BELIEFS CAN INFLUENCE THE SUCCESS

OF THEIR ONLINE HEALTH-RELATED INFORMATION SEARCHES

Presentation for the ALISE 2015 Annual Conference

STUDY CONTEXT

INTRODUCTION

Pre-existing knowledge

Search behaviors

Information retrieved

Accuracy of

decisionsHealth

behaviors

Health outcomes

LITERATURE REVIEW: HEALTH-RELATED INTERNET USE AMONG CHILDREN AND TEENS

LITERATURE REVIEW: CHALLENGES FACED BY YOUTH IN ONLINE SEARCH AND CREDIBILITY ASSESSMENT

Overly confident;Lack domain knowledge;Lack search experience;Search and skim quickly;Assume information is correct;Choose most easily/quickly accessible sources;Rely on surface characteristics; andRely on their familiarity with the vocabulary,

media, and source.

LITERATURE REVIEW: POSITIVE HYPOTHESIS TESTING

• Anchoring effect (Lau & Coiera, 2007 and 2009)

• Confirmation bias (Keselman et al., 2008)• Positive hypothesis testing (Kayhan, 2013)• Positive test strategy (Klayman & Ha, 1987)

RESEARCH QUESTIONS

How accurate are tweens’ perceptions regarding their knowledge of particular health conditions?Why do tweens feel sure (or unsure) about

their answers to health-related questions following their online searches for answers to these questions?

METHODS: RECRUITMENT

METHODS: DATA COLLECTION

# Topic Questions

1 Diabetes a. Can eating too much sugar cause diabetes?b. Can type 2 diabetes be hereditary – that is, does it sometimes run in families?

2 Ulcers a. Are ulcers caused by stress?b. Are ulcers caused by a virus?

3 Cancer a. Can cancer be caused by using a cell phone?b. Can eating a lot of oranges cause cancer?

SAMPLE PRE-TASK QUESTIONNAIRE

Activity 2: UlcersPre-questionnaire:1. How would you rate your knowledge about ulcers?

1 2 3 4 5Poor Not very Neutral Pretty Very

good good good

2. How interested are you in the topic of ulcers?1 2 3 4 5

Not at all Somewhat Neutral Somewhat VeryInterested interested interested interested

3. What is an ulcer? _______________________________________________________________________________________________________________________

4. What causes ulcers? _____________________________________________________________________________________________________________________

SAMPLE POST-TASK QUESTIONNAIRE

Post-questionnaire:

Are ulcers caused by stress? YES NO

What words did you use to search for this information? _________________________________________________________________________________________

List the URL(s) where you got this information: _______________________________________________________________________________________________

How sure do you feel about your answer to this question? 1 2 3 4 5

Not at all Not very sure Neutral Pretty sure Very sure

Please explain why you feel sure (or unsure) about your answer to this question: ______________________________________________________________________________________________________________________________________________________________________________________________________

METHODS: DATA ANALYSIS

“I think diabetes is taking too much sugar, then I

think it turns into fat, then it makes a big lump on

your stomach.”

Gender Male: 2 (18%)Female: 9 (82%)

Age(M = 12.9, SD = .83)

11: 1 (9%)12: 1 (9%)13: 7 (64%)14: 2 (18%)

Race/Ethnicity

Black or African-American: 7 (64%)Hispanic or Latino: 2 (18%)Asian: 1 (9%)Other: 1 (9%)

STUDY PARTICIPANTS

PARTICIPANT SELF-RATINGS

Ratings of Health Information Literacy Skills

ActivityNo

Experience Poor Fair GoodVery Good Excellent

Find 0 (0%) 0 (0%) 1 (9%) 7 (64%) 1 (9%) 2 (18%)

Understand 0 (0%) 0 (0%) 3 (27%) 5 (46%) 2 (18%) 1 (9%)

Assess trustworthiness

0 (0%) 1 (9%) 2 (18%) 2 (18%) 5 (46%) 1 (9%)

Assess personal relevance

1 (9%) 1 (9%) 2 (18%) 2 (18%) 2 (18%) 3 (27%)

Apply 0 (0%) 0 (0%) 4 (36%) 1 (9%) 4 (36%) 2 (18%)

Sum 1 (2%) 2 (4%) 12 (22%) 17 (31%) 14 (25%) 9 (16%)

= 72% Total

FINDINGS: RQ #1 RATE YOUR KNOWLEDGE ABOUT THE TOPIC

1; 9% 1; 9%

7; 64%

2; 18%

6; 55%

1; 9%

3; 27%

3; 27% 1; 9%4; 36%

2; 18%1; 9% 1; 9%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Cancer Diabetes Ulcers

5: Very good

4: Pretty good

3: Neutral

2: Not very good

1: Poor

Michael: Rated his knowledge of diabetes as “not very good”; but provided an insightful definition.

Madison: Rated her knowledge of ulcers as “pretty good,” but offered an incorrect definition.

Elena (who rated her knowledge of cancer as “not very good”) offered a correct definition

FINDINGS: RQ #1 ACCURACY OF “KNOWLEDGE PERCEPTIONS”

“A disease that has to do with your body’s insulin.”

“I think ulcer is when you don’t eat too much and then the worms in your stomach start wa[s]ting your stomach and then it leave[s] a bruise and it really hurts.”

“Cancer is a disease that creates tumors in your body which can lead to death.”

FINDINGS: RQ #2HOW SURE DO YOU FEEL ABOUT YOUR ANSWER?

1; 9%2; 18%

1; 9% 1; 9%

2; 18%

2; 18%

2; 18%

2; 18%

2; 18%

2; 18%

5; 46% 7; 64%

6; 55%

6; 55%

4; 36%

4; 36%

4; 36%3; 27%

2; 18%3; 27%

2; 18%3; 27%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Diabetes (1a) Diabetes (1b) Cancer (1a) Cancer (1b) Ulcers (1a) Ulcers (1b)

5: Very sure

4: Pretty sure

3: Neutral

2: Not very sure

1: Not at all sure

FINDINGS: RQ #2WHY SURE/UNSURE OF THEIR ANSWERS

• “I feel sure of my answer [because] my friend’s mom has diabetes and she told him it’s not from eating too much sugar” (Chloe)

• “I feel sure about my answer because (1) my grandmother has diabetes, (2) my mom is a nurse, and (3) I searched up diabetes a few years ago.” (Tiana)

• “I feel sure because I have heard of people saying it and doctors and also because of I searched it on the Internet.” (Madison)

FINDINGS: RQ #2SURE OF THEIR ANSWERS - KNOWLEDGE

• “I’m sure in my answer because I got it off the Internet.” (Gabrielle)

• “I’m sure of my answer because the website [kidshealth.org] has lots of information on health topics.” (Tiana)

• “It [medicinenet.com] gives good information.” (Diamond)

• “Because the URL [cancer.gov] ends in .gov so it comes from the government.” (Elena)

• “The website [diabetes.org] gave me good reasons to why it happens and how it happens.” (Diamond)

FINDINGS: RQ #2SURE OF THEIR ANSWERS - TRUST

• “I have a feeling in my gut.” (Diamond)

• “It’s a belief.” (Kimi)

• “I just have a feeling.” (Diamond)

• “Because I just know things and [I] am sure.” (Alyssa)

FINDINGS: RQ #2SURE OF THEIR ANSWERS - HUNCH

•“I’m not sure because there’s not enough information.” (Chloe)

•“I feel neutral because I never knew that using too much of cell phone could cause cancer.” (Madison)

•“I don’t feel sure because I don’t have background information about it.” (Chloe)

FINDINGS: RQ #2UNSURE OF THEIR ANSWERS

DISCUSSION

Our participants:• Lacked domain knowledge and

prior search experience;• Were confident in their search

ability and quickly conducted searches;

• Skimmed web pages and guessed if they couldn’t quickly find an answer;

• Automatically trusted the Internet;

• Made credibility judgments that were primarily intuitive (hunch-based) or heuristic (based on generalized rules of thumb)

Prior beliefs affected search; many used search to confirm what they thought they already knew (or didn’t search at all based on their confidence in their pre-existing knowledge).

DISCUSSION

IMPLICATIONS

1.Incorrectly believes he/she knows answer May either not engage in search or may focus search on locating corroborating information

2. Correctly believes he/she already knows answer, but is not confident Conducts search and finds incorrect information, revising previously correct belief

3. No relevant knowledge Conducts search and accepts first answer, without attempting to assess accuracy or credibility

Lack of domain knowledge, lack of search experience, and

high confidence in ability to use the Internet to find info.

Tendency to quickly and haphazardly search, select search

results, skim Web pages, and assume that information is correct

Finding and believing incorrect information and/or

strengthening of incorrect preexisting beliefs

IMPLICATIONS

Don’t assume you’re rightEven when you feel very sure about an answer, keep in mind that the answer may not be correct.

Don’t assume you’re wrongYou may actually have correct information, even if the information you encounter on the Internet does not match.

Take an open approachMaintain a critical stance by not including your pre-existing beliefs as keywords.

Verify, verify, verifyInformation found on the open Web needs to be validated, no matter how the information appears on the site.

RECOMMENDATIONS

CONCLUSION AND FUTURE WORK

PROJECT SUPPORT

We appreciate the support of the National Library of Medicine, the Information Policy & Access Center at the University of Maryland and the students and school librarians who participated in HackHealth.

QUESTIONS?

Contact Us:HackHealthHackhealth.umd.eduhackhealth@umd.edu@kidshackhealth

Beth St. Jeanbstjean@umd.edu

Natalie Greene Taylorngreenetaylor@gmail.com@ngreenetaylor

REFERENCES

A g o s to , D . E . ( 2 0 02a) , “ B o u n d ed r a t i o n a l i t y a n d s a t i s f i c in g in yo u n g p e o p le ’ s We b - b a s e d d e c i s io n m a k in g ” , Jou r n a l o f t h e A m e r i c an S oc i e t y f o r I n f o r mat ion S c i e n c e a n d Te c h n o l og y, Vo l . 5 3 N o . 1 , p p . 16 –27.

A g o s to , D . E . ( 2 0 02b) , “A m o d e l o f yo u n g p e o p le ’ s d e c i s i o n - m ak ing in u s in g t h e We b ” , L i b r a r y & I n f o rmat i on S c i e n c e Re s e arch , Vo l . 24 N o . 4 , p p . 31 1 –341 .

A m e r i c a n A s s o c ia t i o n o f S c h o o l L ib r a r ia n s ( A A S L ) ( 2 0 09) , S t a n dards f o r t h e 21 s t -c en t ur y l e a r ne r i n a c t i on , A m e r i c a n A s s o c ia t io n o f S c h o o l L ib r a r ia ns , C h ic a g o , I L .

E a s t in , M . S . ( 2 0 08) , “ Towa r d a c o g n i t i ve d eve lo p m e n t a l a p p ro a c h to yo u t h p e r c e p t ion s o f c r e d ib i l i t y ” , i n M et z g e r, M . J . a n d F la n a g in , A . J . ( E d s . ) , D i g i t a l M e d i a , Yo u t h , a n d C r e d ib i l i t y ,T h e M I T P r e s s , C a m b r id g e , M A , p p . 101 -1 22 .

E a s t in , M . S . , Ya n g , M . , a n d N a t h a n s o n , A . ( 2 0 06) , “ C h i ld r en o f t h e n et : A n e m p i r i c a l ex p lo r a t io n in to t h e eva lu a t io n o f I n te r n et c o n te n t ” , Jou r n a l o f B r oadc as t i ng & E l e c t r on i c M e d i a , Vo l . 5 0 , p p . 21 1 -230 .

Fe r g ie , G . , H u n t , K . , a n d H i l to n , S . ( 2 01 3) , “ W h a t yo u n g p e o p le wa n t f ro m h e a l t h - r e la te d o n l in e r e s o u r c e s : A fo c u s g ro u p s t u d y ” , Jou r n a l o f You t h S t u d i es , Vo l . 16 N o . 5 , p p . 57 9 -596 .

F id e l , R . , Dav ie s , R . K . , Do u g la s s , M . H . , H o ld e r, J . K . , H o p k in s , C . J . , Ku s h n e r, E . J . , M i ya g i s h im a , B . K . , a n d To n ey, C . D . ( 1 9 99) , “A v i s i t to t h e in fo r m a t ion m a l l : We b s e a r c h in g b e h av io r o f h ig h s c h o o l s t u d e n t s ” , Jou r n a l o f t h e A m e r i c an S oc i e t y f o r I n f o rmat ion S c i e n c e , Vo l . 5 0 N o . 1 , p p . 24 - 37.

F la n a g in , A . J . , a n d M et z g e r, M . J . ( 2 0 08) , “ D ig i t a l m e d ia a n d yo u t h : U n p a r a l l e le d o p p o r t u n i ty a n d u n p r e c e d e n te d r e s p o n s ib i l i t y ” , i n M et z g e r, M . J . a n d F la n a g in , A . J . ( E d s . ) , D i g i t a l m e d i a , you t h , a n d c r e d i b i l i t y, T h e M I T P r e s s , C a m b r i d g e , M A , p p . 5 - 27.

Fo s s , E . , D r u in , A . , Y ip , J . , Fo r d , W. , G o lu b , E . , a n d H u t c h in s o n , H . ( 2 01 3) , “A d o le s ce nt s e a r c h ro le s ” , Jou r n a l o f t h e A m e r i c an S oc i e t y f o r I n f o rm at i on S c i e n c e a n d Te c h n o logy, Vo l . 6 4 N o . 1 , p p . 17 3 -1 89 .

G a s s e r , U . , C o r t e s i , S . , M a l i k , M . , a n d L e e , A . ( 2 0 1 2 ) , Y o u t h a n d d i g i t a l m e d i a : F r o m c r e d i b i l i t y t o i n f o r m a t i o n q u a l i t y . T h e B e r k m a n C e n t e r f o r I n t e r n e t & S o c i e t y R e s e a r c h P u b l i c a t i o n S e r i e s , N o . 2 0 1 2 - 1 . A v a i l a b l e a t : h t t p : / / dm l 2 0 1 1 . d m l h u b . n e t / s i t e s / dm l c e n t r a l / f i l e s / r e s o u rc e _ f i l e s / s s r n -i d 2 0 0 5 2 7 2 . pd f ( a c c e s s e d 7 J u l y 2 0 1 4 ) .

G u t n i c k , A . L . , R o b b , M . , T a k e u c h i , L . , a n d K o t l e r , J . ( 2 0 1 0 ) , A l w a y s c o n n e c t e d : T h e n e w d i g i t a l h a b i t s o f y o u n g c h i l d r e n . A v a i l a b l e a t : h t t p : / / w w w . j o a n g a n z c o o n e y c e n t e r . o r g / w p -c o n t e n t / u p l o a d s / 2 0 1 1 / 0 3 / j g c c _ a l w a y s c o n n e c t e d . pd f ( a c c e s s e d 7 J u l y 2 0 1 4 ) .

H a n s e n , D . L . , D e r r y , H . A . , R e s n i c k , P . J . , a n d R i c h a r d s o n , C . R . ( 2 0 0 3 ) , “ A d o l e s c e n t s s e a r c h i n g f o r h e a l t h i n f o r m a t i o n o n t h e I n t e r n e t : A n o b s e r v a t i o n a l s t u d y ” , J o u r n a l o f M e d i c a l I n t e r n e t R e s e a r c h , V o l . 5 N o . 4 , e 2 5 . d o i : 1 0 . 2 1 9 6 / j m i r . 5 . 4 . e 2 5

H i r s h , S . ( 1 9 9 9 ) , “ C h i l d re n ’ s r e l e v a n c e c r i t e r i a a n d i n f o r m a t i o n s e e k i n g o n e l e c t r o n i c r e s o u r c e s ” , J o u r n a l o f t h e A m e r i c a n S o c i e t y f o r I n f o r m a t i o n S c i e n c e , V o l . 5 0 , p p . 1 2 6 5 - 1 2 8 3 .

H i r s h , S . ( 2 0 0 4 ) , “ D o m a i n k n o w l e d g e a n d c h i l d r e n ’ s s e a r c h b e h a v i o r ” , i n C h e l t o n , M . K . a n d C o o l , C . ( E d s . ) , Y o u t h i n f o r m a t i o n - s e e k i n g b e h a v i o r : T h e o r i e s , m o d e l s , a n d i s s u e s , S c a r e c r o w P r e s s , L a n h a m , M D , p p . 2 4 1 – 2 7 2 .

H s i e h - Y e e , I . ( 1 9 9 3 ) , “ E f f e c t s o f s e a r c h e x p e r i e n c e a n d s u b j e c t k n o w l e d g e o n t h e s e a r c h t a c t i c s o f n o v i c e a n d e x p e r i e n c e d s e a r c h e r s ” , C a s e o f D I A L O G S e a r c h e s o f E R I C , V o l . 4 4 , p p . 1 6 1 – 1 7 4 . d o i : 1 0 . 1 0 0 2 / ( S I C I ) 1 0 9 7 - 4 5 7 1 ( 1 9 9 3 0 4 ) 4 4 : 3 < 1 6 1 : : A I D - A S I 5 > 3 . 0 . C O ; 2 - 8

J e n k i n s , H . ( 2 0 0 9 ) , “ C o n f r o n t i n g t h e c h a l l e n g e s o f p a r t i c i p a t o r y c u l t u r e : M e d i a e d u c a t i o n f o r t h e 2 1 s t c e n t u r y ” , T h e M a c A r th u r F o u n d a t i o n , T h e M I T P r e s s , C a m b r i d g e , M A .

K a y h a n , V . O . ( 2 0 1 3 ) , “ S e e k i n g h e a l t h i n f o r m a t i o n o n t h e W e b : P o s i t i v e h y p o t h e s i s t e s t i n g ” ,I n t e r n a t i o n a l J o u r n a l o f M e d i c a l I n f o r m a t i c s , V o l . 8 2 N o . 4 , p p . 2 6 8 – 2 7 5 .

K e s e l m a n , A . , B r o w n e , A . C . , a n d K a u f m a n , D . R . ( 2 0 0 8 ) , “ C o n s u m e r h e a l t h i n f o r m a t i o n s e e k i n g a s h y p o t h e s i s t e s t i n g ” , J o u r n a l o f t h e A m e r i c a n M e d i c a l I n f o r m a t i c s A s s o c i a t i o n , V o l . 1 5 N o . 4 , p p . 4 8 4 -4 9 5 .

K l a y m a n , J , a n d H a , Y . ( 1 9 8 7 ) , “ C o n f i r m a t i o n , d i s c o n f i r m a t i o n , a n d i n f o r m a t i o n i n h y p o t h e s i s t e s t i n g ” , P s y c h o l o g i c a l R e v i e w , V o l . 9 4 N o . 2 , p p . 2 1 1 - 2 2 8 .

REFERENCES

L a u , A . Y . S . , a n d C o i e r a , E . W . ( 2 0 0 7 ) , “ H o w d o c l i n i c i a n s s e a r c h f o r a n d a c c e s s b i o m e d i c a l l i t e r a t u r e t o a n s w e r c l i n i c a l q u e s t i o n s ? ” , S t u d i e s i n H e a l t h T e c h n o l o g y a n d I n f o r m a t i c s , V o l . 1 2 9 P t . 1 , p p . 1 5 2 - 1 5 6 .

L a u , A . Y . S . , a n d C o i e r a , E . W . ( 2 0 0 9 ) , “ C a n c o g n i t i v e b i a s e s d u r i n g c o n s u m e r h e a l t h i n f o r m a t i o n s e a r c h e s b e r e d u c e d t o i m p r o v e d e c i s i o n m a k i n g ? ” , J o u r n a l o f t h e A m e r i c a n M e d i c a l I n f o r m a t i c s A s s o c i a t i o n , V o l . 1 6 , p p . 5 4 - 6 5 .

L e n h a r t , A . , P u r c e l l , K . , S m i t h , A . , a n d Z i c k u h r , K . ( 2 0 1 0 ) , “ S o c i a l m e d i a & m o b i l e I n t e r n e t u s e a m o n g t e e n s a n d y o u n g a d u l t s . P e w I n t e r n e t & A m e r i c a n L i f e P r o j e c t ” A v a i l a b l e a t : h t t p : / / w w w . p e w i n t e r n e t . o r g / ~ / m e d i a / F i l e s / R e p o r ts / 2 0 1 0 / P I P _ S o c i a l _ Me d i a _ a n d _ Y o u n g _ A d u l t s _ R e po r t _ F i n a l _ w i t h _ t o p l i n e s . p d f ( a c c e s s e d 7 J u l y 2 0 1 4 ) .

M a d d e n , M . , L e n h a r t , A . , D u g g a n , M . , C o r t e s i , S . , a n d G a s s e r , U . ( 2 0 1 3 ) , T e e n s a n d T e c h n o l o g y 2 0 1 3 . A v a i l a b l e a t : h t t p : / / w w w . pe w i n t e r n e t . o r g / 2 0 1 3 / 0 3 / 1 3 / t e e n s - a n d - t e c h n o l o g y - 2 0 1 3 / ( a c c e s s e d 7 J u l y 2 0 1 4 ) .

R i d e o u t , V . ( 2 0 1 1 ) , Z e r o t o 8 : C h i l d r e n ’ s M e d i a U s e i n A m e r i c a . A v a i l a b l e a t : h t t p : / / w w w . c o m m o n s e n s e m e d i a . o r g / r e s e a r c h / z e r o - t o - e i g h t - c h i l d r e n s - m e d i a - u s e - i n - a m e r i c a( a c c e s s e d 7 J u l y 2 0 1 4 ) .

R i e h , S . Y . a n d H i l l i g o s s , B . ( 2 0 0 8 ) , “ C o l l e g e s t u de n t s ' c r e d i b i l i t y j u d g m e n t s i n t h e i n f o r m a t i o n s e e k i n g p r o c e s s ” , i n M e t z g e r , M . J . a n d F l a n a g i n , A . J . ( E d s . ) , D i g i t a l M e d i a , Y o u t h , a n d C r e d i b i l i t y , C a m b r i d g e , M A , T h e M I T P r e s s , p p . 4 9 - 7 2 .

S h e n t o n , A . K . a n d D i x o n , P . ( 2 0 0 4 ) . I s s u e s a r i s i n g f r o m y o u n g s t e r s ' i n f o r m a t i o n - s e e k i n g b e h a v i o u r . L i b r a r y & I n f o r m a t i o n S c i e n c e R e s e a r c h , 2 6 ( 2 ) , 1 7 7 - 2 0 0 .

S u b r a m a n i a m , M . T a y l o r , N . G . , S t . J e a n , B . , F o l l m a n , R . , K o d a m a , C . a n d C a s c i o t t i , D . ( i n p r e s s ) . A s s i m p l e a s t h a t ? : T w e e n c r e d i b i l i t y a s s e s s m e n t i n a c o m p l e x o n l i n e w o r l d . J o u r n a l o f D o c u m e n t a t i o n .

S u n d a r , S . S . ( 2 0 0 8 ) . T h e M A I N M o d e l : A h e u r i s t i c a p p r o a c h t o u n d e r s ta n d i n g t e c h n o l o g y e f f e c t s o n c r e d i b i l i t y . I n M . J . M e t z g e r & A . J . F l a n a g i n ( E d s . ) , D i g i t a l M e d i a , Y o u t h , a n d C r e d i b i l i t y ( p p . 7 3 - 1 0 0 ) . C a m b r i d g e , M A : T h e M I T P r e s s .

T h o m p s o n , K . M . , J a e g e r , P . T . , T a y l o r , N . G . , S u b r a m a n i a m , M . , a n d B e r t o t , J . C . ( 2 0 1 4 ) . D i g i t a l l i t e r a c y a n d d i g i t a l i n c l u s i o n : I n f o r m a t i o n p o l i c y a n d t h e p u b l i c l i b r a r y , S c a r e c r o w , L a n h a m , M D .

REFERENCES