Sergi Bermúdez i Badia

Inv. Assist. Prof., Universidade da Madeira

Marie Curie Research Fellow, Madeira – ITI

sergi.bermudez@uma.pt

2050

*United Nations Population Fund 2012

• The world population is ageing dramatically

• In Europe ~20% of the population is over 60; it will be 30+% by 2050.2

• One of the main causes of permanent disability in older adults that consumes 2-4% of total healthcare costs worldwide.

1

• The number of strokes will increase during the next 40 years.

2

• It will be difficult to manage the sharp rise of social and healthcare costs.

1Donnan et al., Lancet 2008;

2Howard & Goff, Ann. N.Y. Acad. Sci 2013

Peripheral manipulation of the skeletal-motor system is expensive, boring, treats the sequels but not the source of the problem…

Dobkin, Nat Clin Pract Neurol 2008

1. Treatment frequency and intensity correlate with recovery

(Kwakkel et al., 2004; Sonoda, Saitoh, Nagai, Kawakita, & Kanada,

2004).

Low cost technology for extended deployment

2. Movement practice and repetition play a fundamental role in

recovery (Karni et al., 1995).

Building requirements into game mechanics

3. Specificity of rehabilitation training with respect to the deficits and

required functional outcomes has an impact on recovery (Krakauer

2006).

Accessible interface technology and task personalization

6

VE can provide:

- Fully controlled environments

- Minimally supervised intensive training

- Task-specific movement reiteration

- Individualized training

- Feedback for reward and motivation

3

1

6

2 5

4

1- Virtual Reality technology: - 3D virtual environments - Head Mounted Displays (Oculus Rift, Vuzix)

2- Patient monitoring: - Eye tracking (EyeTribe, Tobii) - Autonomic System responses (Bitalino) - Face tracking (Face API) 3- Accessible Rehabilitation: - Movement kinematics (kinect, wii) - BCI-EEG (g.MobiLab, OpenVibe, BCI2000, EMOTIV) 4- EMG driven Neuro-Robotic orthosis (mPower1000) 5- Augmented Reality (AR) and custom tracking through AnTS 6- We provide it as open and freely available software for research at http://neurorehabilitation.m-iti.org

9

Training paradigm:

- Goal oriented and repetitive actions

- Bimanual training (non-paretic arm

support)

- Parameterized (flying speed,

turning speed, acceptance radius,

distance between objects)

Motivation:

- Embedded in a game

- Extensive visual and sound

feedback

- Automatic computation of training

parameters (Avoid failure and

frustration)

Quantify performance

11

- Optimal performance at intermediate stress levels - We use the Yerkes & Dodson law to optimize time to the average performance of the user

* Yerkes, R.M. and J.D. Dodson, 1908

* Yerkes, R.M. and J.D. Dodson, 1908

12

- Flow is “a state of peak enjoyment, energetic focus, and creative concentration by people engaged in adult play” - Balance user skills with challenge using our psychometric model

* Csikszentmihalyi, M. 1975

* Csikszentmihalyi, M. 1975

A study with 10 healthy participants has

shown that the NTT captures precise

quantitative kinematic information during a NTT training session, including:

• Range of Movement (ROM)

• Movement smoothness • Arm coordination • Arm contribution to task

Bermúdez i Badia, Stroke Research & Treat, 2012

s t a d s*t s*a s*d t*a t*d a*d s2 t2 a2 d2 Movement Smoothness Range of Motion

Arm Displacement Arm coordination

Kinematic measure = c0 + c1*speed + c2*turning + c3*acceptance + c4*distance

+ c5*speed*turning + c6*speed*acceptance + c7*speed*distance

+ c8*turning*acceptance + c9*turning*distance

+ c10*distance*acceptance + c11*speed2 + c12*turning2 + c13*acceptance2

+ c14*distance2

- Not all parameters contribute to all movement kinematic measures

- We have a quantitative way of adapting parameters depending on a higher level

desired kinematic training

• Cognitive domain is not always considered. • ADL’s are mostly cognitive and motor (dual-task).

RGS (Cameirão et al., 2011)

NTT (Bermúdez i Badia & Cameirão, 2012)

What do we know about cognitive demands of these tasks?

16

17

• Cognitive rehabilitation is mostly performed with “paper and pencil tools”.

• Based on “tradition”, not

necessarily scientific evidence.

• Traditional tasks lack ecological validity.

• There is no framework for how to design and select tasks for cognitive rehabilitation.

18

* Taxonomy Revised (Anderson et al., 2001) * Toulouse – Piéron task (Toulouse et

al., 2004)

19

20

Creating ecologically valid

Activities of Daily Living (ADLs)

• Attention

simulated supermarket, post

office, bank, pharmacy

• Executive functions

elementary instructions,

problem solving tasks

• Visuospatial orientation

navigation through simulated

city

Vourvopoulos et al. International Conference on Advances in Computer Entertainment Technology (ACE’14).

22

Inclusion criteria: • Absence of hemi-spatial neglect;

• Sufficient cognitive capacity (MMSE) ≥ 15 (Folstein et al, 1975 English version of Warrior et al., 1994);

• Ability to be seated;

• Education ≥ 4th class or read and write;

• Motivation to participate in the study.

Participants: • 18 (9 experimental + 9 control), with ages between 34 – 90 years old

• 10 female and 8 male

• Time after stroke: 3 months to 14 years

• Localization: 9 left hemisphere + 9 right hemisphere

• 15 participants had no experience with computers

•with prof. Luísa Soares (UMa) Dra. Manuela Barros (SESARAM)

Dr. Rafael Freitas (SESARAM) Dra. Teresa Gois (SESARAM)

23

12 sessions of 20 min of cognitive training (~ 1 month) on top of conventional therapy with pre and post assessment.

• Experimental group: VR based simulation of ADL

• Control group: Cognitive stimulation using traditional tools (puzzles, paper and pencil tasks, etc)

24

• Addenbrooke's cognitive examination - ACE-R - assesses: attention, memory, verbal fluency, language and visual-spatial orientation.

• Trail making test A and B - evaluates attention, sequencing of stimuli, visual search, information processing, eye-hand coordination, etc.

• Layout of pictures - ability to organize and sequence a story.

• Stroke Impact Scale - SIS – subjective evaluation of strength, hand function, memory, emotion, mobility, ADL’s, communication and social participation.

25

26

* Bold indicates p < 0.05 matched-pairs Wilcoxon test

Experimental Group Control Group

Pre Post Pre Post

ACE-R - Total Median 72 81 (+9) 66 69 IQR 15 19 27 20

MMSE Median 23 29 (+6) 23 26 IQR 6 4 6 6

(ACE-R) Attention & Orientation

Median 15 18 (+3) 14 16 IQR 3 2 5 5

(ACE-R) Memory Median 15 18 (+3) 18 18 IQR 5 7 9 9

(ACE-R) Verbal Fluency Median 5 6 6 5 (-1) IQR 4 4 4 3

(ACE-R) Language Median 22 24 (+2) 19 21 IQR 2 5 6 8

(ACE-R) Visuo-spatial Median 12 14 12 14 IQR 7 2 6 9

Layout of pictures Median 2 4 (+2) 2 2 IQR 2 5 3 3

* Bold indicates p < 0.05 matched-pairs Wilcoxon test

* Bold indicates p < 0.05 matched-pairs Wilcoxon test

27

* Bold indicates p < 0.05 matched-pairs Wilcoxon test

* Bold indicates p < 0.05 matched-pairs Wilcoxon test

* Bold indicates p < 0.05 matched-pairs Wilcoxon test

Experimental Group Control Group

Pre Post Pre Post

Memory Median 62,5 71,88 (+9,38) 56,25 62,5 (+6.25)

IQR 37,5 33,48 37,52 32,82

Emotional Estability

Median 75 83,33 (+8.33) 58,33 66,67

IQR 29,15 12,44 27,78 27,78

Comunication Median 75 85,71 67,86 67,86 IQR 30,35 32,15 37,51 39,29

ADLs Median 50 56,25 43,75 45,337 IQR 42,71 37,5 38,54 33,35

Social Participation

Median 63,89 66,67 (+2.78) 36,11 50 (+13.18)

IQR 43,17 29,8 22,21 16,66

Total Recovery Median 50 70 (+20) 40 60 IQR 15 25 15 30

• A close dialog between health practitioners – neuroscientists – technologists is necessary

• Our systems are not the end product, are the hypotheses

• Hypotheses need validation impact assessment

• Game / training mechanics need to include:

• Game parameters need to be automatically personalized

• Challenge vs. skill needs to be quantified and well understood

• Stress levels need to be controlled to ensure maximal performance and consequent maximal learning

• Consider both motor and cognitive aspects

• Relation with Activities of Daily Living

29

Sergi Bermúdez i Badia (Assist. Prof)

Mónica S. Cameirão (Assist. Prof)

Ana Lúcia dos Santos Faria (PhD student)

Athanasios Vourvopoulos (PhD student)

John Edison Muñoz (PhD student)

Teresa Paulino (Research assistant)

Afonso Gonçalves (PhD student)

Andreia Andrade, MSc student Psychology (join thesis with Luísa Soares) Júlio Alves, MSc student Informatics (join thesis with VisLab, IST-Lisbon) Davide Neves, MSc student Informatics Miguel Sousa, MSc student Telecommunications and Networks (join thesis with Luís Gomes) André Ferreira, MSc student Informatics Rúben Jardim, MSc student Informatics

University of Pittsburgh Department of Occupational Therapy

Quality of Life Technologies Center http://www.cmu.edu/qolt Carnegie Mellon University

Daniel P. Siewiorek Asim Smailagic

Scott Bleakley

Myomo Inc http://www.myomo.com

Steve Kelly Ela Lewis

SESARAM - www.sesaram.pt Serviço de Saúde da RAM

Dra. Manuela Barros Dr. Rafael Freitas Dra. Teresa Gois

Dr. Jean-Claude Fernandes Dr. Gil Bebiano

Dr. Rafael Macedo

30

For more information contact:

sergi.bermudez@uma.pt

or visit

http://neurorehabilitation.m-iti.org