ACM SRC Grand Final Paper

The ClockMe System: Computer-Assisted Screening Tool for Dementia

School of Interactive Computing | College of Computing | Georgia Institute of Technology Hyungsin Kim

Advisor: Ellen Yi-Luen Do {hyungsin, ellendo}@gatech.edu

Abstract

Due to the ever rapidly growing senior population, there is a strong need to develop alternatives to enhance the current cognitive screening practices. The Clock Drawing Test, a paper-and-pencil test, has been used as one of the most popular cognitive screening tools for dementia. In this paper, I present my approach to develop a computerized Clock Drawing Test, the ClockMe System. Based on my observational study of the current practice of dementia screening at a clinic, I have designed and developed the ClockMe System. This System includes two different applications - (1) ClockReader for patients who take the Clock Drawing Test and (2) ClockAnalyzer for clinicians who use the CDT results to make diagnoses or to monitor patients. The preliminary study shows promise for computerized screening in clinical environments. Based on the initial study results, I have enhanced the current ClockMe System and have deployed it to conduct a three-month field study with volunteers at the Emory Alzheimer’s Research Center. The contribution of this research is the creation of a computerized screening tool that can help clinicians identify cognitive impairment through a more accessible and quick-and-easy screening process.

1. Introduction In 2003, the Alzheimer’s Foundation of America proclaimed the third Tuesday of November as National Memory Screening Day [1]. The goal of this initiation is to promote the early detection of Alzheimer’s disease and related disorders (ADRD), and to encourage timely intervention and treatment. The senior population has been dramatically increasing. A prominent public health challenge caused by aging is cognitive dysfunction, or poor mental functioning associated with confusion, forgetfulness, and difficulty concentrating. Alzheimer’s disease is one of these representative disorders. Unlike other diseases, which are physically visible, early detection of cognitive dysfunction is rarely easy. In fact, fewer than 50% of Alzheimer’s cases are diagnosed, and only approximately 25% are treated, even after several years of progressive cognitive decline [1]. Unfortunately, there are no known treatments for curing Alzheimer’s disease [7]. Therefore, in order to properly treat cognitive dysfunction, it is critical to identify the early process of cognitive impairment.

2. Problem and Motivation The current practice of administering most cognitive impairment screening tests, such as the Saint Louis University Mental Status (SLUMS), Mini-Mental State Examination (MISE), and Montreal Cognitive Assessment (MoCA), is based on traditional analog media, such as paper-and-pencil tests in hospital environments by clinicians [7].

Most tests consist of verbal interactions complemented by drawing or written interactions. For an example of a drawing interaction, each patient is asked to either draw a clock or some complex figures with a pencil on a given sheet of paper. Then, clinicians such as neuropsychologists or neurologists need to spend hours analyzing and scoring the tests. The process is long and tedious. In addition, sometimes different administrators of the test may have different scoring criteria [2].

Patients often believe that talking over their potential health problems with neurologists is considerably expensive and time consuming. Thus, many patients do not consult doctors until their symptoms are all too apparent, at which point they may be too late to treat successfully. Furthermore, most primary care doctors do not conduct any kind of neuropsychological examination [3]. However, primary care doctors may be best positioned to detect cognitive impairment in its early stages since older adults are more likely to have regular visits for physical check-ups [3].

In brief, it is often difficult to detect the early stages of cognitive impairment because (1) it is hard to differentiate cognitive impairment from normal cognitive degeneration due to aging; (2) there is limited opportunity for seniors to meet with specialists, such as neurologists or neuropsychologists, unless they have serious observable symptoms; and (3) the disease usually develops progressively; thus, capturing it at the appropriate moment is challenging, as it normally requires continuous monitoring through everyday activities.

Due to the reasons listed above, an alternative way to support patients and neurologists is needed, one which must be more accessible and effective. The significant amount of research currently being conducted regarding phone- or computer-based dementia screening indicates that dementia screening should no longer be limited to clinicians’ offices [3]. Therefore, first I explore the possible technological opportunities to develop a neuropsychological screening tool for detecting and monitoring cognitive impairment. Based on the observational study, I designed, implemented, and evaluated a computer-assisted screening tool for dementia, called the ClockMe System.

3. Background and Related Work

With the advent of personal computers in the early eighties, studies on developing computerized neuropsychological assessments have been actively conducted [2]. Luciana and Nelson performed an assessment of neuropsychological functioning through the use of a computer-automated battery with 4- to 12- year-old healthy children [11]. Gualtieri and Johnson conducted a study on the reliability and validity of CNS Vital Signs (CNSVS), which is a computerized neurocognitive test battery [8]. Their study shows that CNSVS is appropriate for a screening tool, but it cannot be good enough for replacing formal neuropsychological testing [8]. Research from De Luca et al. also addressed issues in using the computer-based Cambridge Neuropsychological Test Automated Battery (CANTAB) [4]. Despite many trials, until now there have been no accomplished great successes in terms of clinical utilization [4].

The potentials of computerized assessments that many studies support lie in the precision of measurement, shorter assessment time, standardization, automatic scoring, minimization of subjectivity, and the impact that the examiners may have on the participants [4, 6]. However, some researchers in studies have concerns about participants’ unfamiliarity with computers [6]. Iverson et al. conducted research on the relationship between self-reported computer familiarity and performance on computerized neurocognitive testing [10]. Their studies show that frequent computer users performed better than people with less computer experience. In particular, both rapid visual scanning and accurate keyboard use can significantly influence testing results [12]. In order to solve these issues, some studies suggest using a touchscreen rather than using a keyboard and mouse [2].

On the contrary, two recent studies contend that computer familiarity does not affect the outcomes of cognitive testing [6]. Fredrickson et al. conducted a usability study of a brief computerized cognitive screening test in older people. They argue that computerized testing can be completed successfully by older adults with less-experienced computer use if some supervision or practice is provided [6]. Moreover, Wagner and Trentini conducted a comparison study between the manual and computer-based versions of the Wisconsin Card Sorting Test [2]. With the results showing no difference between the two versions of the tasks, there is strong evidence to support that the two different versions of the test are equivalent [2].

Moreover, recent research in Human Computer Interaction indicates that Tablet-PCs can provide an appealing platform for older adults [12]. Unlike WIMP (Windows, Icons, Menus, and Pointers)-based interaction, a pen can provide the elderly with more direct interaction in using the system. Users can put the stylus in the exact place where the cursor is, unlike a relative pointing device such as a mouse, which requires hand-eye coordination (moving the mouse on the desktop while looking at the screen to find the cursor location). Besides providing a familiar form of interaction, as well as retaining the full advantage of hand-eye coordination skills, pen-based interaction can truly provide authentic traditional neuropsychological assessment, but it would be empowered with computational benefits. For example, most neuropsychological testing relies heavily on drawing tasks, which are based on paper-and-pencil interaction.

4. Approach and Uniqueness: The ClockMe System The ClockMe System is a computerized Clock Drawing Test. The traditional Clock Drawing Test (CDT) is a rapid and reliable instrument for the early detection of cognitive dysfunction. Neurologists often notice missing or extra numbers, or misplaced clock hands in drawings from people with cognitive impairment and use scoring criteria to make diagnoses and plan treatment. The drawings in Figure 1 clearly show cognitive degradation with abnormal clock drawings from patients with Alzheimer’s disease and suspected frontotemporal dementia. The patients were asked to draw a clock by putting the numbers on the clock and setting the time to ten past eleven.

No dementia (CDT = 15/15)

Alzheimer’s disease (CDT= 8/15)

Suspected front temporal dementia (CDT = 3/15)

Figure 1. Normal Clock of aging person [left] Abnormal Clocks due to dementia [middle and right] [5]

The overarching goal of the ClockMe System is to develop an automated analysis of the Clock Drawing Test. First, the system records and recognizes a patient’s freehand drawing data. Then based on the scoring criteria, the system automatically analyzes the drawing and reports the score. The system can reduce the extensive labor needed for manual scoring. Furthermore, capturing and monitoring data through computerization provides additional dimensions (such as pressure, airtime, and the drawing sequence) in understanding cognitive impairment for medical practitioners.

The ClockMe System includes two different applications - (1) ClockReader for patients who take the Clock Drawing Test and (2) ClockAnalyzer for medical practitioners who use the CDT results to make diagnoses or to monitor patients. The ClockMe System with an online sketch recognition algorithm is developed in C# programming language. Each application in the ClockMe System has a different User Interface (UI). It can also run independently as a separate application. However, the ClockAnalyzer Application will share all of the data collected from the ClockReader Application.

The design goal of the ClockReader User Interface is to provide patients with an environment that is similar to paper-and-pencil based testing. As can be seen in Figure 2 (left), the UI consists of two main parts: the Drawing area and the Action area. In the Drawing area, similar to the traditional CDT, a user would see a pre-drawn circle in the middle of the tablet computer screen. The bottom part of the UI, what I call the Action area, includes an automatically generated–ID, two radio buttons, and three submit buttons. The radio buttons are designed to provide users with two different input options such as “Pen” and “Eraser.” The default setting would be “Pen.” If users want to erase a part of their drawing, they can select the “Eraser” radio button; then, the stylus would work like an eraser.

The ClockAnalyzer is designed to support the medical practitioner’s decision-making by performing several different analyses of the automated data collection. For example, the ClockAnalyzer will help doctors analyze the data by automatically scoring the criteria, as well as by visualizing graphs from the existing data. The goal of the ClockAnalyzer UI design is to provide a data analysis and an interpretation effectively through multiple visualization methods to medical practitioners.

Figure 2. A patient is drawing a clock on the ClockReader System (right) and the default User Interface of the ClockAnalyzer Application (left)

As can be seen in Figure 2 (right) above, the user interface of the ClockAnalyzer Application consists of three main parts: the Drawing Output area, the Analysis area, and the Monitoring panel. The drawing output area is in the top left window. The application provides two different drawing outputs: static drawing and active animated drawing. When a user opens a patient’s file, the user would see the static clock drawing from the CDT. The static drawing becomes an animation when a user clicks the “Arrow” button on the bottom of the window. The Analysis area includes three automatic data-capturing outputs: (1) Automatic Process Capture – The system would capture and analyze a patient’s “planning strategy.” (2) Automatic Time Capture – The system would capture and analyze the “airtime,” which is the time when the patient is not-drawing – the time of pausing. (3) Automatic Pressure Capture – The system would measure pressure information during the drawing process. The automatic data-capturing methods are based on one of the leading neuropsychologists, Kaplan’s process-oriented approach [9,13]. Many researchers within her school of thought argue that despite the heavy use of drawing tasks in neuropsychological assessment, they are not fully utilized in supporting neuropsychologists’ decision-making processes [13]. These drawings are predominantly analyzed at the end of the tests as a final product, while the process is ignored. This extra data would provide clinicians with new useful information that was not previously available from paper-and-pencil tests. Lastly, the monitoring panel provides multiple clock drawings of a patient collected over time for easy comparison. With a graph of historical CDT results, medical practitioners will benefit by having a quick comparative overview of the patient’s cognitive condition.

5. Results and Contributions

To deploy a computerized Clock Drawing Test in a real clinical environment, the system should be usable for patients, especially those who are not familiar with using a computer. Here is the question I tried to answer from our preliminary study: “Can a computerized tool replicate the exact same results as a paper-and-pencil test? ” Specifically, I examined how patients, especially older adults, feel about taking a computerized test for their cognitive impairment assessment and how clinicians analyze drawings from computer testing compared to drawings from paper-and-pencil testing. I conducted a usability comparison study with 24 older adult volunteers. The participants were recruited from the HONOR Research Registry in Clinical Research in Neurology (CRIN) at Emory Alzheimer’s Disease Research Center in Atlanta. The usability test consisted of 1) a computer familiarity survey; 2) two drawing tasks: one using the ClockReader and the other using a paper-and-pencil version; and 3) a post-interview. The computer familiarity survey results of computer familiarity show that all participants considered themselves as having low- to medium-level computer literacy. None of them considered themselves as having high computer literacy. It is not surprising, considering the participants’ average ages. They are seniors who did not use computers when they worked, and personal computers were becoming popular after they retired. However, unlike their self-reported computer literacy, all participants successfully completed the Clock Drawing Test using the ClockReader application. In the post- interview, I also asked whether using the stylus to draw a clock on the surface of the computer was difficult. None of the participants expressed difficulties in using a stylus. Also, from the results of the two different drawings that each participant did, it was evident that there were no critical difficulties for older

adults in using a tablet computer. Figure 4 below shows two clock drawings from a 92-year-old male participant. Both drawings are almost identical. The only difference shows the shakiness of the clock hands from a drawing using a ClockReader application. In fact, the participant has a tremor. Since there is less friction on the surface of the computer, a participant’s drawing using the application can apparently show the tremor problems more easily. The paper-and-pencil drawing, on the other hand, does not show any signs of tremor. This shows the potential that using a computerized sketch-analysis may identify the early stages of Parkinson’s disease.

Figure 4. Two clock drawings from a 92-year-old male participant: A paper-and-pencil drawing (left) and a drawing using a stylus on the tablet computer (right)

Furthermore, two different evaluators evaluated the drawings of a 92-year-old male participant. Two evaluators unanimously scored four drawings as 12 points out of 13. Since the time should have been set at 11: 10 instead of 10: 50, one point was deducted. This clearly shows that there is no difference between a paper-and-pencil-drawing and one from the ClockReader application. The preliminary study shows promise for computerized screening in clinical environments. Based on the preliminary study results, I am currently conducting a three-month field deployment study with volunteers at the Emory Alzheimer’s Research Center.

The contribution of this research is the creation of a computerized screening tool that can help clinicians identify cognitive impairment through a more accessible and quick-and-easy screening process. Automatically captured and stored data through such a tool would provide an effective way to identify people with dementia and replace the tedious effort and error of human scoring with a consistent scoring practice and analysis. Furthermore, the tool will provide computer-collected novel behavioral data such as airtime, pausing tendency, millisecond-level completion time, planning strategy, and patterns of exerting pressure, which may be able to offer new insights and understanding of a patient’s cognition. This novel behavioral data can also be applied to the process of making diagnoses or diagnostic models, thereby resulting in more evidence-based decision-making.

Acknowledgement

This material is based upon work supported by the National Science Foundation under Grant No 1137527 (SHB) and Emory ADRC (P50 AG025688) and ACTSI (supported in part by PHS Grant UL1 RR025008, KL2 RR025009 or TL1 RR025010 from the Clinical and Translational Science Award program, National Institutes of Health, National Center for Research Resources). Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation. We thank our collaborating researchers and medical practitioners at the Emory Alzheimer’s Disease Research Center for their valuable feedback on the development of the ClockReader System. We also appreciate the volunteers who participated in our usability testing.

References

[1] J. Ashford. Screening for Memory disorders, dementia and Alzheimer's disease. Aging Health (2008) (4), 399- 432. [2] G. Wagner & C. Trentini. Assessing executive functions in older adults: a comparison between the manual and the computer-based versions of the Wisconsin Card Sorting Test (2009) Psychology & Neuroscience 2(2) 195-198. [3] Contador, I., Fernández-Calvo, B., Ramos, F., Tapias-Mezrino, E., and Bermejo-Pareja, F. Dementia screening in primary care: critical review. Revista De Neurologia 51, 11 (2010).

[4] De Luca, C.R., Wood, S.J., Anderson, V., et al. Normative data from the CANTAB. I: development of executive function over the lifespan. Journal of Clinical and Exper Neuropsychology (2003). [5] Feldman, H.H., et al., Diagnosis and treatment of dementia: 2. Diagnosis. Canadian Medical Association Journal, 2008. 178(7): p. 825-836. [6] Fredrickson, J., Maruff, P., Woodward, M., et al. Evaluation of the usability of a brief computerized cognitive screening test in older people for epidemiological studies. Neuroepidemiology 34, 2 (2010), 65-75. [7] Goldstein, G. and Incagnoli, T.M. Contemporary Approaches to Neuropsychological Assessment. Springer, (1997). [8] Gualtieri, C.T. and Johnson, L.G. Reliability and validity of a computerized neurocognitive test battery, CNS Vital Signs. Archives of Clinical Neuropsychology. 21, 7 (2006), 623-643. [9] Igor Grant and K.M. Adams, eds. Neuropsychological Assessment of Neuropsychiatric and Neuromedical Disorders. 2009, Oxford University Press: USA. [10] Iverson, G.L., Brooks, B.L., Ashton, V.L., Johnson, L.G., and Gualtieri, C.T. Does familiarity with computers affect computerized neuropsychological test performance? Journal of Clinical and Experimental Neuropsychology 31, 5 (2009) [11] Luciana, M. Practitioner Review: Computerized assessment of neuropsychological function in children: clinical and research applications of (CANTAB). Journal of Child Psychology and Psychiatry 44, 5 (2003). [12] Moffatt, K.A. Increasing the accessibility of pen-based technology: an investigation of age-related target acquisition difficulties. Ext. Abstracts CHI 2009, ACM Press (2008), 2625-2628. [13] Poreh, A.M., The quantified process approach: an emerging methodology to neuropsychological assessment. Clin Neuropsychol, 2000. 14(2): p. 212-22.