Listening to the Learners: A Case Study in Health Information Website Design Jennifer Turns Acting Assistant Professor Technical Communication Tracey Wagner

Listening to the Learners:A Case Study in Health Information

Website Design

Jennifer TurnsActing Assistant Professor

Technical Communication

Tracey WagnerEducational Technology Analyst

Program for Educational Transformation Through Technology

Acknowledgements: This work has been supported by the Program for Educational Transformation Through Technology (PETTT). Many people have contributed to this work including Scott Macklin, Aaron Louie, Kristen Shuyler, Brett Shelton, Kristina Liu, Alice Tanada, Jake Burghardt, Julianne Fondiller, Regina Yap, Ralph Warren, and Dr. Frederick Matsen.

Today’s Goal

– Background

– Case Study

– Results

– Implications

– Conclusions

Demonstrate how a characterization of users as learners can inform the design of an informational website.

Background - Case Study - Results – Implications - Conclusion

Informational Website Design

• Examples: – legal information

– cooking information

– home repairs and renovations

• Issues– Type of content

– Organization of content

– Style of content

– Access to content


Audience Analysis

• “Know thy user”

• Common dimensions include– Role – Dominant persona of users (job, affiliation)

– Goals – Reason for the interaction

– Knowledge – The extent and nature of prior relevant knowledge

– Circumstances of Use – Setting, resources, strategy, timing

– Culture – Group level beliefs, language, preferences

– Ergonomics – Relevant perceptual & motor abilities, skills


When users are learners

• Learner centered design (Soloway et al, 1994)

• Key Challenges– Diversity: learners are diverse– Engagement: learners need support for

engaging in learning– Growth: learning changes the learner


Design Relevance?

• What is the nature of diversity, engagement, and growth in a specific domain and with a specific user populations?

• How can we characterize users along these dimensions?

• How do we design informational web sites to address these dimensions?


Arthritis Source

• Origin: Developed in 1995 by Dr. Rick Matsen

• Goal: Help learners-at-large satisfy their knowledge needs with authorized information.

• Challenges: Expand scope and ensure effectiveness of Arthritis Source


Audience Analysis Methods

• Challenge– Getting to actual users

• Our methods include– Online Survey– Phone Interviews– Logfile Analysis


Diversity

• Users with many roles • Many goals / questions– Condition information (54%)– Ways to minimize pain (7%)– Seek clarification or a second

opinion on diagnoses (5%)– Preparing for surgery (5%)– Desire to better control

arthritis condition (2.5%)– …

Relation10%

Medical Professional

5%

Researcher2%

Student1%

Other20%

Person with Arthritis

62%n=462/472


Diversity

• Educational spread • Misconceptions– Bone spurs cause arthritic

pain

– Joint injections are helpful

– Low bone density is associated with Osteoarthritis

– There is little you can do

– (and 3 others)

Secondary35%

Associates23%

Bachelors22%

Masters15%

Doctorate5%

n=399/472


DiversityInternational Use- 128 countries

in 2½ months


Engagement

– Motivated learners: 48% from search engine– Personally engaged: 65% have arthritis

– Heavily one-time visitors: 80% first visit– Moderate time on page: avg. visit ~ 7 mins.

– Common misconceptions and questions suggest a need for longer engagement periods and deeper interaction with information


Growth

– Windows into growth• Many statements of information needs

• Users at different points in patient careers

– Opportunities for growth• Efforts to articulate questions

– Ongoing interest in exploring Growth


Design Challenges• Diversity

– Support users with variety of questions/goals– Support users in different contexts– Support users with different knowledge

• Engagement– Help users find relevant information quickly– Find ways to engage users in knowledge that might be useful to

them (e.g., correcting misconceptions)

• Growth– Support user concerns and knowledge over time


Design Direction – Content

• Content via Templates– Common organization

– Linked to questions

– Ensures coverage

• Links to– Diversity

– Engagement

– Growth


Design Direction – Search

• Question-based Search– Start with question– Choose from “similar”

questions– Enter content at point where

question is answered.

• Links to– Diversity– Engagement– Growth


What we are working on

• Building on these strategies– Expanding content

– Improving search

• Evaluation Studies – Is it working? – User evaluation of content

– “Longitudinal” Study - Impact of extended site interaction on knowledge and health outcomes


Moving in the Right Direction:Evidence of Growth?

• Sequences of searches showing growth?• 8:26 am "my husbands leg bone below knee is

splinering and knee has moved over it,is there a surgery to save his leg"

• At 9:22 am "aproximately 6inches below knee cap the long bone is spintering is there a surgery that can cover this and help support the leg bone“

• 10:01am "can a femur tibia splintering of the bone be surgicially be repaired ?“


Concluding Remarks• Learner-Centered Design - Diversity, engagement, growth

• Learner-Centered Design and Audience Analysis– One strategy for audience analysis– Complements other audience analysis approaches (e.g., roles, tasks)

• Learner-Centered Design and Our Case Study– Variety of data can be organized via this framework– The dimensions are powerful organizer– Diversity is easy to capture– Engagement & growth are challenging to capture for web populations. – Using this framework informed our design


Method – Logfile Analysis

• Analyze record of users interactions with site

• Available data include– Entry/exit pages, pages viewed, time on page, country of

origin, etc.

Table 1: Sample log file data 1/1/96 0:09:30 dial18.chemek.cc.or.us. :bonejoint:kkakkkkk2_1.html 1/1/96 0:09:32 dial18.chemek.cc.or.us. :bonejoint:gif:Clip.GIF 1/1/96 0:10:03 dial18.chemek.cc.or.us. :bonejoint:mov:ScopeACLTear.mov 1/1/96 0:10:47 dial18.chemek.cc.or.us. :bonejoint:mov:ACLgraft.mov 1/1/96 0:10:56 dial18.chemek.cc.or.us. :bonejoint:Arthritis.idx.html 1/1/96 0:13:01 pm5-00.magicnet.net. :bonejoint:nzzzzzzz1_2.html 1/1/96 0:15:00 pm5-00.magicnet.net. :bonejoint:xzzzzyzz1_1.html


Method - Phone Interview

About Visit

• Could you tell me about your visit or visits to the Arthritis Source?

• Could you tell me what you were trying to do when you visited the Arthritis Source?

• Did you benefit from your visit or visits to the Arthritis Source?

• What kind of information do you think other arthritis patients should know?

About Knowledge of Condition

• Could you tell me what you think arthritis is in general?

• Could you tell me how RA/OA affects the body?

• Do you know what contributes to getting RA/OA?

• Do you know how RA/OA is diagnosed? If no, Do you remember what your doctor told you about your diagnosis?

• What is most difficult to understand about RA/OA?

Background - Approach – Results - Implications

Method – Online Survey• Questions: Adaptive, ~25 questions

• Participants – Duration: 9/1/2000 – 7/2/2001 (10 months)– 472 respondents / 710 starts

• Analyses– Descriptive Statistics – Content Analysis – Qualitative Coding – Statistical Analysis


What we are working on

• Supporting goals (finding/using content)

• Support evolving content

• Evaluating effects– Learning– Health outcomes– Use patterns– Various changes over time—growth


Method – Online Survey• Questions: Adaptive, ~25 questions

– Demographics– Role– Goal – Successful during today’s visit

• Participants – Duration: 9/1/2000 – 7/2/2001 (10 months)– 472 respondents / 710 starts


Method - Phone Interview• Questions:

– About your visit (5): Could you tell me what you were trying to do when you visited the Arthritis Source?

– Knowledge of condition (5): Could you tell me how RA/OA affects the body?

• Participants – 20 users (10 OA, 10 RA)

• Analyses– Conceptions/misconceptions– Overarching Goals – Specific Information Needs


Method – Logfile Analysis

• Analyze record of users interactions with site• Available data include

– Entry/exit pages, pages viewed, time on page, country of origin, etc.


Sample Logfile Data: 2 requestscpe-65-172-233-177.sanbrunocable.com [29/Apr/2002:00:01:07 -0700] "GET /shoulder_elbow/shoulder/roughness/management/exercise/02/exercise06 HTTP/1.1" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; Q312461)"

cpe-65-172-233-177.sanbrunocable.com [29/Apr/2002:00:01:11 -0700] "GET /shoulder_elbow/shoulder/roughness/management/exercise/02/exercise05 HTTP/1.1" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; Q312461)"

Documents

Listening to the Learners: A Case Study in Health Information Website Design Jennifer Turns Acting Assistant Professor Technical Communication Tracey Wagner