LLM project deployment plan

LLM Pilot Programme

Nancy BakerChief Operating Officer,

Global Security [email protected]

Agenda

• Pilot Programme Framework About the LLM Pilot Programme LLM Pilot Execution Ensuring Pilot Quality

• Pilot Programme by Country Austria (CEIT/Municipality of Schwechat) Cyprus (University of Cyprus) France (E-SENIORS) Greece (AUTH & ATHENA RC) Spain (Fundacion INTRAS)

Athens – 20 May 2010Athens – 20 May 2010

Pilot Programme Framework

• About the LLM Pilot Programme Pilot Aims and Objectives Pilot Methodology Key Metrics Key Elements to Guide Pilot Conduct


About the LLM Pilot Programme

• Pilot Aims and Objectives Validate the technical viability of the integrated LLM system Quantify the effectiveness of the LLM solution through the

collection of physical and cognitive measures Validate each element of

the product as a precursorto delivering the LLM solution to the market


• Installation• Training• Support• Monitoring Procedures

• User Experience• Cognitive Outcomes • Physical Outcomes• Legal and Ethical

Compliance

• Integration• Operation• Documentation Accuracy

and Thoroughness• Usability

• End-User Response• Pricing• Competitiveness• Differentiation

Competitive Market Position

Technology Solution Efficacy

Effective Supporting Processes

Achieve Desired

Outcomes for User

Not merely a test of technological accuracy and effectiveness, but a

holistic test of each aspect of the solution


• Pilot Methodology 1-year pilot programme in five countries Well-defined pilot execution plan for each

country Complies with all national legal and ethical

guidelines Utilizes several types of environments serving

the elderly to identify individuals who may benefit from the programme:

• Home setting• Senior day centres• Longer-term health facilities



• Key Elements to Guide Pilot Conduct Clearly defined pilot protocols Clearly defined roles and responsibilities for all

Consortium members Clearly defined pilot execution plan for each country Clearly defined and consistent inclusion and exclusion

criteria Thorough informed consent procedures Consistently applied screening tests across all countries User privacy/data protection and anonymisation



• Key Pilot Metrics by Objective Technical and operational effectiveness of integration of

ICT solutions• Survey of pilot staff• Support call logs• System log files

End-User acceptance of solution• User survey• System log files (to assess compliance with programme)

Acceptability to the market across EU countries• User survey (usability, accessibility, pricing)• Internal audit for legal and ethical compliance



• Key Pilot Metrics by Objective Effectiveness of cognitive and physical outcomes

• Analysis of Clinical Results, including specific tests for MCI, Mild Dementia, Alzheimer’s subjects.

• Tests to include a 90-minute battery comprised of a combination of the following:

CVLT-II IADL MOCA Senior Fitness Test Digit Span MMSE Reaction Time Test Social Activity Test WHOQOL



• LLM Pilot Execution Pilot Programme Environments Pilot Screening Criteria Pilot Documents Pilot Training Pre-Pilot Activities Pilot Activities Post Pilot Activities


LLM Pilot Execution

• Pilot Programme Environments Home settings, assisted living/groups, social

centres, and clinical environments Data collected:

• will enable comparison of specific aspects of the LLM system, its efficacy, and impact

• will be specific to the pilot site, and not necessarily comparable with data from all other sites/countries

• analysis will distinguish between the results of the various environments versus results based upon data collected from consistently applied testing


LLM Pilot Execution

• Pilot Screening Criteria

Pilot will be run as a clinical research programme including screening and randomization

Well-defined inclusion criteria

Well-defined exclusion criteria


LLM Pilot Execution

• Pilot Documents To ensure consistency of communication To conform as nearly as possible to Good Clinical

Practice (GCP) Documents required by all pilot sites include:

• LLM Information Sheet• Informed Consent Form


LLM Pilot Execution

• Work to-date has been focused on pre-pilot activities

• Now beginning to ramp up for Pilot Execution activities

Place and date of eventPlace and date of event

Months 15-29Months 1-14 Months 29-30

LLM Pilot Execution

• Pre-Pilot Activities (months 1 – 14) Obtain pilot regulatory approvals Verify Quality of Service Assurance and auditing

procedures Develop staffing and training plan Verify installation/technical documentation Verify support procedures Install the LLM system Recruit participants Perform pre-pilot screening tests


LLM Pilot Execution

• Pilot Training Objective: to ensure that all relevant procedures

required to ensure the quality and consistency of the pilot programme is maintained across all pilot locations

To be accomplished in a cascading fashion• Will be conducted by the technical team for 3 audience

types• Will be localised to meet specific pilot site requirements


LLM Pilot Execution

• Pilot Activities (months 15 – 29) Pilot conduct/procedures

• Programme Protocol• Monitoring Procedures – ensure compliance,

document, • Data reporting

Scientific/clinical support Technical support and adaptations Internal audits Manage risks


LLM Pilot Execution

• Post Pilot Activities (month 30) Post-pilot follow-up with participants

• user survey• outcome measures

Technical data and clinical data analysis Assemble and disseminate final report



• Ensuring Pilot Quality Quality of Service Assurance Pilot Measurement Tools


Ensuring Pilot Quality

• Quality of Service Assurance• Pilot testing directly with end-users• Testing of all aspects of the service and

product• Quality of the trial process and adherence

to the defined protocol• Thorough training for all pilot staff• Monitoring of pilot activities by a third party

within the LLM consortium (GSI)


Ensuring Pilot Quality

• Internal Audit• Conducted at least once in each pilot

country• Defined template/checklists to ensure

Consistency in approach Adherence to defined protocol Adherence to user privacy/data protection procedures


Pilot Programme by Country

Greece – Christos FrantzidisAustria – Walter Hlauschek

Cyprus – Dr. Marios NeofytouFrance – Philippe Uziel

Spain – Fundacion Intras


Presentation Agenda

Pilot Site Profile Unique Elements of Pilot Study Pilot Conduct/Protocol Legislative Requirements Ethical Guidelines

Athens, 20 May 2010Athens, 20 May 2010


Greece

Christos FrantzidisLab of Medical Informatics

Aristotle University of Thessaloniki

Pilot Site Profile

• Pilots in Greece will be held in Thessaloniki and Athens. Pilots will run under the scientific co-ordination of the Medical School, and certain Neuropsychiatry/Gerontology experts.

• The intervention will be held at the premises of the Greek Association of Alzheimer’s Disease and Relative Disorders – centre of St. Eleni, where all installations will be carried out. It is a non-profit organization founded in 1995, promoting support to people suffering of Alzheimer’s disease. The association establish collaborations with foreign institutions promoting scientific clinical and research activities.


Physical description

• The LLM solution will be installed in two distinct rooms at the buildings of the Greek Association of Alzheimer Disease and Relative Disorders – St. Eleni.

• The CTC intervention will be performed in a room equipped with 15 personal computers and an equal number of headphones.

• The second room will be equipped with both the ILC and the PTC equipment.

• A limitation posed by the physical layout of this room is the fact that the PTC will be performed in groups, much like the CTC part of the LLM intervention. Therefore, the ILC functions will be limited to those of the scenario where fall detection is feasible but the participant’s identity cannot be extracted.


Access to pilot target Access to pilot target population population

• Participants will be: Mild Cognitive Impairment (MCI) Mild Dementia (Alzheimer) Cognitively Healthy Elderly

• Control Group(s) – Active Control Group (group with cognitive stimulation and

performing daily life activities, but not LLM solution) Passive Control Group (no cognitive stimulation; treatment as

usual; waiting list for future iteration of pilot)

• The staff at the Greek Association of Alzheimer Disease and Relative Disorders – St. Eleni psychologists and technicians will help during both recruitment and intervention.


Pilot dates

• Acquisition of equipment (already acquired and tested a sample of Wii devices)

• Acquisition of total equipment – February 2010.• LLM system completion – Late March – early April

2010• Equipment installation and system configuration

April 2010• Initial Users tests May-June 2010• Initial active use in a planned pilot phase -

September 2010


# of iterations

• 3 pilot iterations will be carried out in the buildings St. Eleni and St. Ioannis. The iteration’s duration was formed considering both the intervention requirements and the limitations posed by the Greek holidays (summer, Christmas, New Year’s Eve and Easter). Therefore the first one starts on September (month 16) till November (month 18). The second iteration starts on December (month 19) till March (month 22). The third iteration starts on Mid March (month 22) until June (month 25).

• Before the initiation of each iteration, the screening procedure will take place for all the participants. After each iteration, the cognitive and physical status of the participants would be evaluated using the outcome measures.


Timeline

• The intervention will be held in Saint Eleni and Saint Ioannis buildings in Thessaloniki.

• The LLM intervention would be divided in three groups. • The first group would be consisted of Mild Cognitive Impaired (MCI)

participants, the second by mildly demented Alzheimer Disease and the third group would be consisted of cognitively healthy participants.

• The same division would be followed for the cognitive intervention and for the active control participants performing stimulation tasks

• Therefore, there are 9 groups, of 10 participants each. • The participants will be expected to complete 40-48 cognitive and 40-

48 physical sessions. • The intervention is consisted of 4 sessions per week for a period of 12

weeks. The session’s duration is 1 hour (clear time).


Cognitive Training Procedure/Duration

• The cognitive training procedure is expected to utilize 10 PCs equipped with headphones.

• The cognitive intervention would be performed using the Brain Fitness software (provided by the Posit Sciences) containing auditory stimuli.

• During each training session, the participant is expected to work with four of the six exercises for 15 minutes per exercise (clear time).


Physical Training Procedure/Duration

• The physical exercise program is systematic and contains 4 basic types:

Endurance Strength Balance (static and/or dynamic) and Flexibility

• Duration is 1 hour per day, 3 times per week, consisting of: three portions:

Warm up (20-25 min)• Use of ergometer bikes, treadmills• Control of steady speed (15 – 16 km/h)• Measurement of time, speed and elapsed distance

Main part (30-35 min )• In early stages each exercise shall be executed in 2 sets of 8 repetitions with 1

minute interval• Maximum number of sets: 3 Maximum number of repetitions: 15

• Equipment to be used: Wii related devices, PCs, touch screens, gymnastics tires, weights of one kilo each, rhythmic gymnastics ball (16 cm diameter), gymnastics bars (1 meter), fitness classes


Legislative RequirementsLegislative Requirements

• All project activities are approved by the Ministry of Health and Social Solidarity. More specifically, there is the Central Health Council which is responsible for the observance of the protocol and the Directory of Education and Research which is responsible for the co-ordination of each research program.

• Relevant EC legislation. The charter of fundamental rights of the European Union. Directive 2001/20/EC of the European Parliament and of the Council of 4 April 2001

on the approximation of the laws, regulations and administrative provisions of the member states relating to the implementation of good clinical practice in the conduct of clinical trials on medicinal products for human use.

Directive 95/46/EC of the European Parliament and of the Council of 24 October 1995 on the protection of individuals with regard to the processing of personal data and on the free movement of such data. During the project the European and International standards (EN 12182 and EN ISO 10535) will be considered

The principles of Helsinki Declaration



Austria

DI Walter HlauschekManaging Director

CEIT Central European Institute of Technology

Pilot Site ProfilePilot Site Profile

• municipality of SCHWECHAT next to Vienna, 16.500 inhabitants part of the AAL Living Lab Schwechat “Seniorenzentrum” Schwechat (senior’s centre)

sheltered living, day care centre, nursing station service and contact point for local living seniors, senior’s organisation, care

taking organisations

• CEIT RALTEC Central European Institute of Technology, Institute of Rehabilitation and

Assisted Living Technologies; founded in April 2006 by the municipality of Schwechat

extra-faculty research and development institute Ambient Assisted Living (AAL) eHealthcare eHomecare older citizens and persons with special needs

Main actor of the AAL Living Lab Schwechat and member of



• Pilot site description Senior Citizen Centre

pre-pilot-trial all LLM components in one room

Private Homes pilot trials 5 LLM-prototypes in apartments of elderly people

• space is limited no treadmill or ergobike, only wii-equipment

• ILC in the whole apartment• PTC and CTC: wherever the participant wants them to be placed

• Access to pilot target population Participants of CEIT RALTEC’s user pool Fitness trainer of Schwechat’s senior citizen centre

Members of senior gymnastics group


Unique Elements of Pilot StudyUnique Elements of Pilot Study

• Pilot dates Prepilot: Senior Citizen Centre

3 months beginning with month 12 (May 2010) One iteration with 2 users

Pilot: Private Homes 3 pilot iterations duration of 3 months each one month of evaluation and service adaptation in between Participants in the passive control group of one iteration can be users of the LLM-installation in later

iterations.

• Target pilot population Only cognitively healthy people corresponding passive control group For At-Home-installations:

single-person-household (without cat/dog) apartment size must not exceed 3 rooms



• LLM solution deployment at site Schwechat LLM-service installed in apartments of single living seniors. Focus on physical and cognitive training within an assistive smart

home solution to support older people in living autonomously and independent

• Site specific limitations and requirements Limited space in private homes -> no treadmills / ergobikes Whole functionality of ILC Make sure that the participants can perform the exercises alone

supervised training in the beginning 24/7 support for trials at home


Pilot Conduct/ProtocolPilot Conduct/Protocol

• Cognitive Training Procedure/Duration LLM-computer with headphones Brain Fitness Software 1 hour per day, 3 times a week

Possibility to split into a couple of shorter sessions per day

• Physical Training Procedure/Duration 30 minutes per day, 2 times per week Warm up (10 min)

Equipment to be used: resistance band, yoga mat Main part (15 min )

Fit For All using wii-equipment Recovering (5 min)

Stretching and breathing exercises --> relaxation of the subject and reset of heart rate at normal levels

• Independent Living Component Enhance safety Support communication and alarming abilities



• Federal Act concerning the Protection of Personal Data (Datenschutzgesetz 2000– DSG 2000)

• Based on Data Protection Directive 95/46/EC of the European Parliament and of the Council of 24 October 1995

• Austrian trial fulfills all requirements

• “Medizinproduktegesetz” (MÜG, BGBL I Nr. 77/2008) for any kind of non-pharmaceutical medical products

• LLM-trial in Austria is treated as validation of usability and end-user acceptance of the LLM-service as a non-medical product, therefore no approval necessary


Ethical GuidelinesEthical Guidelines

• In Austria there are no common approved guidelines

Only for medical trials for pharmaceutical and other medical products such guidelines exist published by the national bioethical commission (“Bioethikkommission”)

Nevertheless RALTEC and municipality of Schwechat consider the publications and opinions of this commission and

the ethic commission of Lower Austria “Niederösterreichische Ethikkommission” is informed about the LLM-pilot

Personal data of test-persons are anonymized Test-persons sign an informed consent similar to the one

already in use within other projects



Cyprus

Dr. Marios NeofytouUniversity of Cyprus

Department of Computer ScienceMedical Informatics Laboratory

www.cs.ucy.ac.cy/medinfo

University of Cyprus

• Medical Informatics Laboratory www.cs.ucy.ac.cy/medinfo

• The eHealth Lab was established in 2003 and is located to the new university campus. It was created in order to continue many years of research in the fields of Medical Informatics and Biomedical Engineering, performed by several members of the department of Computer Science. The research activities of the lab include applied and basic research.


The Cyprus Institute of Neurology & Genetics

• The Institute was established in 1990 and has moved to its current purpose-built premises in 1995 after a generous donation from the United States, through the Cyprus office of the United Nations High Commission for Refugees (UNHCR).

• The Institute is a bicommunal, non-profit making institution. Its staff specializes in neurology, molecular biology and all aspects of human genetics. It has an international reputation and works closely with the Cyprus Government, private doctors and scientists, the University of Cyprus as well as other Institutions and Universities from abroad.


Social Services Center of Ayios Dometios

• The Social Services Center of Ayios Dometios (multipurpose center) is a non profit private organisation responsible for elderly people health care in the area of Agios Dometios. Is a unique organization in Cyprus providing quality health and social services for the elderly people in Nicosia.

• This is achieved by helping individuals to acquire the skills to live responsible, healthy and independent through the provision of a variety of care support services to elderly and their families. Member of the center has a long term expertise in carrying out research and national studies as regards the provision of health and social care in Cyprus.


Golden Age elderly home

• The nursing home “Golden Age” is a private residential institution for providing care and support to elderly people who are unable to look after themselves. There is a capability of accommodating 54 persons.

• Residential are enjoying a pleasant environment where they can communicate and maintain their functionality. All of them are supervised by trained staff. A collaborative physician and a physiotherapist are regularly visiting the elderly to provide services tailored to the needs of each individual. In addition, community psychiatric nurses of Cyprus Ministry of Health are providing support to the elderly and their families.



France

Philippe Uziel

E-Seniors

E-Seniors

• e-Seniors, A non-profit NGO whose aim is to empower seniors and

impaired people with ICTs.

• Pilot sites Maps: France & Ile de France (Parisian region) http://maps.google.fr/maps?hl=fr&tab=wl Paris (OSE, MAPI, etc.) Marne la vallée, et plus généralement la RP. Foyer résidence “les mésanges” Vaires sur Marne Janzé (Ille et Vilaine) Vic sur Cère (Cantal) Saint Girons / Pays Couserans (Ariège)


Unique Elements

• Pilot dates: In principle, we plan to begin in the first week of September.

• # of iterations: As many as possible over all the sites within the planned time frame and depending on hw & sw availability

• Target pilot population: Valid 65+ers and MCI and MDI 65+ers • At Vic sur Cère, a Ets de SSR, we plan to have people

recovering from accidents, and to have our e-Home site.• Specific limitations: We intend to use the Wii™ slab massively,

as opposed to other devices analysed as “potentially risky if used by elderly people”. e-Seniors’ medical advisers have issued warnings about the risks induced by using treadmills and bicycles with elderly people. At this point in time, it is very unlikely that such devices will be used in the French environment.


Unique Elements

• We’re having a « médecin », she’s here with us today:

Coordinating locally the medical test material and issues

Supervising the • medical protocols, • medical aspects of the french pilots, and the • non MD health professionals and paramedics.


Unique ElementsUnique Elements

• Staffing and/or training requirements We are eagerly awaiting the training session to

know more about the systems.


Pilot Conduct/Protocol

• Cognitive Training Procedure/Duration Will actually depend on the voluntary populations

and how far they want to go.

• Physical Training Procedure/Duration Will actually depend on the voluntary populations

and how far they want to go.



• Loi Huriet-Sérusclat Afssaps CPP Informed consent.



• CPP Comité de protection des personnes



Spain

Fundación INTRAS


• Organisational background INTRAS is a non profit organization founded in 1994 whose main

fields of work are research, development and innovation and social-health assistance, addressed to improve the quality of life of people with mental health disorders

• Pilot site description The pilot will take place at INTRAS’ and other collaborators’

facilities: -INTRAS: Two memory clinics and a Psychosocial Rehabilitation

center (CRPS).- Collaborators: Provincial Hospital of Zamora, National Reference Center for Alzheimer and dementia and two residence facilities.

The access to the target population will be through a recruiting process of the beneficiaries of INTRAS and the users of the services of the collaborator organizations.



• Pilot dates: -The pre-pilot phase will take place from June to August 2010. -The pilot phase will last from September 2010 to July 2011.

-Post pilot phase will last form August to September 2011

• 3 iterations• Target pilot population: People without cognitive decline, MCI

and MDI • The LLM solution: It will be composed of two CTC (GRADIOR and

Brainfitness), CTC FitforAll and ILC e-Home • Staffing and/or training requirements:

- 5 Neuro-Psychologists

- 3 Psychiatrists - 2 Geriatrists


Pilot Conduct/ProtocolPilot Conduct/Protocol

• Cognitive Training Procedure/Duration - GRADIOR:

30 minutes/ 3 times a week Exercises to train perception, attention, memory, calculus, reasoning and

orientation

- BRAINFITNESS: 1 hour/ 3 times a week Exercises to train language function and auditory processing

• Physical Training Procedure/Duration - FITforALL

3 hours/ 3 times a week Stretching, balance, muscular endurance, flexibility, etc.



• Compliance of the 15/1999 National protection data Law• Royal Decree 223/2004 that regulates the conduct of clinical

trials• 41/2002 Law that regulates the rights and obligations of

patients, users and professionals

• According to the Spanish Law (223/2004), a Clinical Research Ethics Committee must approve all research projects involving human participants (CEICS). The procedure takes 60 days.




Thank You