Distributed Cognitive Aid Distributed Cognitive Aid with Scheduling and with Scheduling and

Interactive Task GuidanceInteractive Task Guidance

Edmund LoPresti, AT Sciences LLC Edmund LoPresti, AT Sciences LLC

Ned Kirsch, University of Michigan Ned Kirsch, University of Michigan

Debra Schreckenghost, Metrica TRACLabs Debra Schreckenghost, Metrica TRACLabs Richard Simpson, University of PittsburghRichard Simpson, University of Pittsburgh

QuickTime™ and aTIFF (Uncompressed) decompressor

are needed to see this picture.

GoalsGoals

ClinicianClinician CaregiverCaregiver

ClientClient

- Determines techniques to counter impairment

- Evaluates client condition and effectiveness of

aids

Maintain autonomy of lifestyle in face of traumatic cognitive impairment

- Ensure client performs important tasks

- Assist when client has problems

ICueICue

Activity Assistant

Remote Information Server

ICue ArchitectureICue Architecture

Cognition Manager

HTN Planner for Schedules

Reactive Planner for Instructions

Instruction Database

Plan Database

Design Assistant

PlanEditor(Java)

InstructionEditor(Java)

Clinician/Caregiver’s Office/Home

Client’s Home

Design Assistant

Clinician Builds Task InstructionsClinician Builds Task Instructions

1. Define steps of task

2. For each step, identify an instruction to aid client

3. Order the steps into a sequence

4. Define how client errors should be handled (alternative instructions, call in help)

ClinicianClinician Instruction Database

XML

Caregiver Builds Client PlansCaregiver Builds Client Plans

CaregiverCaregiver

Design Assistant

Plan Database

1. Identify activities for next day(s)

2. Add priorities and temporal constraints

3. ICue suggests a client plan

4. Adjust planning criteria to build a better plan, if desired

Identify Activities for ScheduleIdentify Activities for Schedule

Available Tasks

Task Attributes

Add Priorities & Time ConstraintsAdd Priorities & Time Constraints

Task List

Timing

• Required/preferred

Priority

ICue Builds the Client PlanICue Builds the Client Plan

Review/Adjust PlanReview/Adjust Plan

Task Stored as XMLTask Stored as XML

Stored for later retrieval when a task Stored for later retrieval when a task becomes activebecomes active

Transitioning from proprietary XML to Transitioning from proprietary XML to AIMS-XML standardAIMS-XML standard• Developed by AbleLink and RERC-ACTDeveloped by AbleLink and RERC-ACT• Allow sharing between devicesAllow sharing between devices

ICue Tracks Schedule and TasksICue Tracks Schedule and Tasks

Design Assistant

XML

Instruction Database

Plan Database

Cognition Manager

Activity Planner

Instruction Sequencer

Activity Assistant

ICue aids Client with Daily TasksICue aids Client with Daily Tasks

• Remind client when time to perform task Step through task instructions

• Provide alternative instructions when client has difficulty

• Track whether client completes activities

ClientClient

Cognition Manager

HTN Planner for Schedules

Reactive Planner for Instructions

Preliminary Data: Field TrialPreliminary Data: Field Trial

ICue prototype was used by a participant having cognitive impairment following TBI.

The participant was observed performing two individualized tasks with cues from ICue.

The participant was able to respond appropriately to the cues provided by ICue, and was able to successfully complete each task without intervention from a clinician, whereas the participant was unable to perform either task independently without the intervention.

Preliminary Data:Preliminary Data:Usability SurveyUsability Survey

6 clinicians defined tasks for hypothetical clients ICue seemed beneficial (mean score of 4.2 out of 5 for 2

questions) Ease of use was mildly positive (mean scores of 3.7, 3.2,

3.7, and 3.7 for 4 questions); participant responses and investigator observations indicated usability issues to be addressed.

Some concern that ICue might take too long to use in a clinical setting (mean 2.5 out of 5).

Using ICue was not frustrating (mean score of 1.7 on a scale of 1-5 for frustration) and was worth the effort to use (mean 4.3 out of 5).

Future Work: Future Work: Feedback to Client/CaregiverFeedback to Client/Caregiver

Real time (for situations ICue cannot handle automatically)

Logged data for later review; potentially increase or decrease level of cues based on client’s performance.

Future Plans:Future Plans:TestingTesting

Formal clinical testingFormal clinical testing Usability trials with non-professional Usability trials with non-professional

caregiverscaregivers

Future Plans:Future Plans:Internet Risk FactorsInternet Risk Factors

Loss of connectivityLoss of connectivity PrivacyPrivacy

Future Plans:Future Plans:Context Appropriate CueingContext Appropriate Cueing

• Address behavioral issues• Provide simple prompts at regular intervals

(e.g. pay attention, relax, speak more, speak less)

• Provide support to vary these cues (wording and/or frequency) based on what is or is not appropriate in different contexts

• A user’s schedule will be enhanced with information about the social context of their tasks• home, work, school; • alone, with friends, with co-workers;• etc

Discussion / Question 1Discussion / Question 1What are the biggest What are the biggest

challenges/obstacles in challenges/obstacles in automated task guidance?automated task guidance?

A.A.

B.B.

C.C.

D.D.

E.E.

Discussion / Question 1Discussion / Question 1What are the biggest What are the biggest

challenges/obstacles in challenges/obstacles in automated task guidance?automated task guidance?

A.A.

B.B.

C.C.

D.D.

E.E.

Discussion / Question 2Discussion / Question 2What technologies or What technologies or

techniques should be integrated techniques should be integrated to support caregivers?to support caregivers?

A.A.

B.B.

C.C.

D.D.

E.E.

Discussion / Question 2Discussion / Question 2What technologies or What technologies or

techniques should be integrated techniques should be integrated to support caregivers?to support caregivers?

A.A.

B.B.

C.C.

D.D.

E.E.

Discussion / Question 3Discussion / Question 3What broad categories of What broad categories of

context might be important in context might be important in behavioral cueing?behavioral cueing?

A.A.

B.B.

C.C.

D.D.

E.E.

Discussion / Question 3Discussion / Question 3What broad categories of What broad categories of

context might be important in context might be important in behavioral cueing?behavioral cueing?

A.A.

B.B.

C.C.

D.D.

E.E.

AcknowledgementsAcknowledgements

Work thus far:Work thus far:

NIH SBIR #5 R43 HD44277-02NIH SBIR #5 R43 HD44277-02

Environmentaly Appropriate Cueing Environmentaly Appropriate Cueing RERC-ACT, NIDRR Grant #H133E040019RERC-ACT, NIDRR Grant #H133E040019