Thomas Austin

Page 1 of 12

A critical review; ‘The relationship between

neuropsychological performance and daily functioning in

individuals with Alzheimer’s disease: Ecological validity of

neuropsychological tests’ (Farias, Harrell, Neumann, & Houtz,

2003).

Completed as part of a BSc Psychology degree

Coursework Deadline: 22/04/2014

Word Count: 1649

Thomas Austin

Page 2 of 12

A critical review; ‘The relationship between neuropsychological performance

and daily functioning in individuals with Alzheimer’s disease: Ecological validity of

neuropsychological tests’ (Farias, Harrell, Neumann, & Houtz, 2003).

Summary

In the reviewed study, two measures of functionality were used with

neuropsychological performance data, focusing on the cognitive domains of acute

and delayed verbal memory, attention, visuospatial, language, executive function

and apraxia, to examine the relationship of cognitive domains and functionality in

Alzheimer’s disease (AD) individuals. One measure of functionality was performance

based, whereas the other was informant based. A sample of 42 geriatric participants,

all diagnosed using the National Institute of Neurological and Communicative

Disorders and Stroke and the Alzheimer’s Disease and Related Disorders

Association (NINCDS-ADRDA) tool were recruited. Simple comparison t-tests,

correlational and stepwise multiple regression revealed confirmation of the

hypothesis; neuropsychological scores can significantly predict the functionality of

AD individuals. However, Farias et al. concludes that the neuropsychological

domains measured cannot fully explain the functionality scores of AD participants.

Introduction

In the reviewed study the authors concisely introduce the research problem

and rationalise the components of the study. The clear definitions of terms,

measures and theoretical concepts enable a transparent framework of the study.

However, one measure, the mini-mental state examination (MMSE) score was not

defined but frequently referred to; it is a component of the inclusion criteria.

Thomas Austin

Page 3 of 12

Despite this, the research questions and hypothesis are appropriately and

succinctly presented.

Methodology

Participants

The participants in Farias et al.’s study were recruited from a pre-

existing population that attended a memory disorders clinic, this results in improving

the ecological validity of the sample. However, due to a possibility of varied disorders

being addressed by the clinic, the authors incorporated current clinical procedures in

diagnosing possible or probable AD. These qualification criterion for the participants

consisted of an NINCDS-ADRDA diagnostic tool and protocol (McKhann et al.,

1984). The NINCDS-ADRDA dementia diagnosis that was conducted for each

participant has received high reliability and high validity, when the characteristics

were examined by Blacker et al. (1994), thus suggesting a similar level of reliability

and validity for the sample used in the reviewed study.

The reliability and validity of the inclusion criterion is a key issue for any study

examining a diagnosed disorder; the use of clinically and academically recognised

diagnostic tools and protocols are a major component of standardisation and thus

enable widespread generalizability, from the study’s results and the population

represented by the sample. Furthermore, the standardisation of diagnosis criterion

encourages the comparison of results between studies focusing on the same

disorder.

Thomas Austin

Page 4 of 12

Despite the aforementioned benefits of the NINCDS-ADRDA diagnostic tool,

the sample displayed an irregular gender distribution; there were more males than

females at a ratio of 3:11. This may have jeopardised the generalizability of the

results, as well as the reliability, because the statistical analysis relied upon the

measurement of AD symptoms. The role of gender has been investigated and found

to correspond with a difference in cognitive manifestation of AD (Buckwalter, Sobel,

Dunn, Diz, & Henderson, 1993), however there is support for a contradictory

argument from a study that found no significant relationship between gender and

behavioural symptom variation (Ott, Tate, Gordon, & Heindel, 1996). Furthermore,

the NINCDS-ADRDA criteria were used as a robust inclusion criterion and ruled out

non-AD dementia disorders.

The wide range of demographic information recorded by Farias et al. was a

thorough procedure and encompassed age as well as education level and the length

of time between initial cognitive decline and admission to the memory clinic. These

are clinically relevant characteristics of the sample population, especially when

focused upon dementia disorders (Moore, Palmer, Patterson, & Jeste, 2007).

Despite the sample size being stated, there was no sufficiency comment

made. However, the sample size is sufficient in size for the analyses conducted.

Materials and Procedure

The neuropsychological tests used to measure the various cognitive

domains examined acute and delayed verbal memory, attention, visuospatial,

language, and executive function and apraxia levels. The measures used were

subsets from common clinical neuropsychological tests.

Thomas Austin

Page 5 of 12

Components of the California Verbal Learning Test (CVLT) and the Wechsler

Memory Scale (WMS) relating to acute and delayed verbal memory were used, as

well as the WMS test pertaining to attention. The visuospatial test was measured by

using the Rey-Osterrieth Complex Figure Test (ROCF test) and language was

measured by the Boston Naming Test (BNT). The Controlled Oral Word Association

(COWA) test and a Wechsler Adult Intelligence Test-Revised (WAIT-R) subset were

used to measure executive functioning, while apraxia levels were recorded using a

subset from the Western Aphasia Battery (WAB).

These measures shared common clinical usage and were all appropriate for

the study and the geriatric sample. There is easily accessible reliability, validity and

evaluative articles on each measure (see Benton & de Hamsher, 1989; Delis,

Kramer, Kaplan, & Ober, 1987; Kaplan, Goodglass, & Weintraub, 1983; Kertesz,

1982; Shin, Park, Park, Seol, & Kwon, 2006; Wechsler, 1945, 1981).

However, the effectiveness of the measures, as used in the reviewed study

can be questioned; there is little research or even comment on the use of individual

subsets to provide reliable or valid measurements of the neuropsychological domain

of focus. If this issue has gone un-checked then participants; scores may not

represent what they are supposed to, an issue of validity.

Thomas Austin

Page 6 of 12

The second set of measures, the measures of functional status, consisted of

one performance and one informant based test battery. Performance based

measures use the objective recording of success, often conducted in a laboratory

setting (therefore lack in ecological validity). Whereas informant based measures

rely on the subjective assessment of a participant’s performance at a task, this can

cause an issue with reliability as the informant, often a care-giver, is different

between participants and is a common confounding variable. Despite these issues

and lack of control of them, the reviewed study examines the differences between

them and thus utilised each issue.

Additionally, the emotional functionality of participants was also of

interest and therefore a depression scale was operationalised. However, depression,

although a major affective disorder, is not the only affective disorder. Many disorders

can affect the affective domain, ranging from schizophrenia to post-traumatic stress

disorder (American Psychiatric Association, 2013). However, depression is a

common comorbid disorder in the AD population (Wragg & Jeste, 1989).

Participants underwent all the tests in a standardised order and were

consistently administered.

Results

The descriptive statistics and details of the independent t-tests were initially

displayed. This enabled a basic comparison between the AD group and the

normative geriatric population scores for functional measures.

Thomas Austin

Page 7 of 12

However, the normative data used was not referenced or otherwise sourced.

Any material used, should have been included and acknowledgement of use should

be noted for future replication. Furthermore, the normative data’s descriptive

statistics and demographics were not presented for comparison.

Primarily, the data was used for multiple correlational analyses and a

Bonferroni correction was correctly applied. These simple correlation statistics were

appropriately applied, using two continuous variables. The means, standard

deviations, , and the results are reported and displayed as appropriate.

Secondly, the data was then analysed using multiple bivariate regression

tests and showed the assumptions of multicolinearity were met and detailed,

focusing on Variance Inflation Factor, Condition Indices and Variance-Decomposition

Proportion values as diagnostic measures. The data was re-run using a stepwise

multiple regression analysis. A major critique of the analyses regards the stepwise

regression statistic and it’s susceptibility to Type I errors, as detailed by Thompson

(1995). This susceptibility is caused by the stepwise procedure; the selection of

variables based on computer defined criteria is a key liability. The procedure

therefore, increases the probability of the involvement of chance, resulting in

erroneously high values and no correction of the liberal p value. Additionally, this

procedure can remove variables that are major theoretical framework factors.

Due to only two multiple regression analyses being used there was no need

for an alpha level adjustment (for example, the Bonferroni adjustment).

Thomas Austin

Page 8 of 12

Interpretation and discussion

The detailed interpretation of age, education and depression’s relationships

with the neuropsychological and functionality measures was conducted effectively,

and each finding was used to build upon current theory and clinical issues, such as

the link between depression and functionality scores, in which a complex

characteristic of AD is discussed.

As aforementioned in the current critique, the differences between the

performance and informant based measures of functionality are an issue of control

due to the nature of each measure; such as low validity and low reliability, for

performance and informant based measures respectively. However, this was

acknowledged by Farias et al. and therefore provided a possible limitation of the

study. In addition, the issue of a reductionist measure of emotional functioning, using

only depression to represent emotional functioning in participants was also

acknowledged.

Furthermore, the authors of the reviewed study continue to discuss other

limitations. The most prominent of critiques relates to the cognitive domains that

were not explicitly measured. This problem is discussed and linked to the large

portion of variance that is unaccounted for in the stepwise multiple regression. As

commented by Farias, et al., the large error variance (unexplained variance)

undermines the fundamental research question; by delineating the relationship

between neuropsychological ability as the sole predictor of functionality of AD

individuals, this was also discussed by Loewenstein, Rubert, Arguelles, and Duara

(1995). Despite this conclusion, Farias et al. drew a comparison with Loewenstein et

al. (1995)’s conclusions that neuropsychological performance cannot be relied upon

as the only predictor of AD individual’s functionality in everyday life.

Thomas Austin

Page 9 of 12

All interpretations and conclusions are discussed with regard to limitations

and clinical applications. Farias et al. also noted major implications for both, current

clinical procedures and theoretical framework of AD and dementia-related disorders.

In conclusion, the theoretical context and framework of the reviewed study

was effectively introduced and the hypothesis and assumptions were appropriately

rationalised. The methodological section was succinct, yet retained all essential

points necessary for replication. Most limitations were addressed, or discussed later

in the paper. However, one key limitation was not mentioned and could invalidate the

results; stepwise regression’s maximisation of chance and the statistic’s

susceptibility to Type I errors. Overall, the study highlighted some considerable

implications for future research and clinical practice.

Thomas Austin

Page 10 of 12

References;

Association, A. P. (2013). Diagnostic and Statistical Manual of Mental Disorders

(Fith.). Arlington, VA: American Psychiatric Publishing.

Benton, A. L., & de Hamsher, K. S. (1989). Multilingual Aphasia examination. Iowa

City, IA: AJA Associates.

Blacker, D., Albert, M. S., Bassett, S. S., Go, R. C., Harrell, L. E., & Folstein, M. F.

(1994). Reliability and validity of NINCDS-ADRDA criteria for Alzheimer’s

disease. The National Institute of Mental Health Genetics Initiative. Archives of

Neurology, 51(12), 1198–1204.

Buckwalter, J. G., Sobel, E., Dunn, M. E., Diz, M. M., & Henderson, V. W. (1993).

Gender differences on a breif measure of cognitive functioning in Alzheimer’s

Disease. Archives of Neurology, 50(7), 757–760.

Delis, D. C., Kramer, J. H., Kaplan, E., & Ober, B. A. (1987). The California Verbal

Learning Test: Adult version. San Antonio, TX: The Psychological Corporation.

Farias, S. T., Harrell, E., Neumann, C., & Houtz, A. (2003). The relationship between

neuropsychological performance and daily functioning in individuals with

Alzheimer’s disease: ecological validity of neuropsychological tests. Archives of

Clinical Neuropsychology, 18, 655–672.

Kaplan, E., Goodglass, H., & Weintraub, S. (1983). The Boston Naming Test (2nd

ed.). Philadelphia: Lea & Febiger.

Kertesz, A. (1982). Western Aphasia Battery. San Antonio, TX: The Psychological

Corporation.

Thomas Austin

Page 11 of 12

Loewenstein, D. A., Rubert, M. P., Arguelles, M. P., & Duara, R. (1995).

Loewenstein, D. A., Rubert, M. P., Arguelles, T., & Duara, R. (1995).

Neuropsychological test performance and prediction of functional capacities

among Spanish-speaking and English-speaking patients with dementia.

Archives of Clinical Neuropsychology, 10, 75–88.

McKhann, G., Drachman, D., Folstein, M., Katzman, R., Price, D., & Stadlan, E. M.

(1984). Clinical diagnosis of Alzheimer’s disease: Report of the NINCDS–

ADRDA work group under the auspices of the Department of Health and Human

Services Task Force on Alzheimer’s disease. Neurology, 34, 939–944.

Moore, D. J., Palmer, B. W., Patterson, T. L., & Jeste, D. V. (2007). A review of

performance-based measures of functional living skills. Journal of Psychiatric

Research, 41(1-2), 97–118. doi:10.1016/j.jpsychires.2005.10.008

Ott, B. R., Tate, C. A., Gordon, N. M., & Heindel, W. C. (1996). Gender difference in

the behavioral manifestations of Alzheimer’s disease. Journal of the American

Geriatrics Society, 44(5), 583–587.

Shin, M. S., Park, S. Y., Park, S. R., Seol, S. H., & Kwon, J. S. (2006). Clinical and

empirical applications of the Rey-Osterrieth Complex Figure Test. Nature

Protocols, 1(2), 892–899.

Thompson, B. (1995). Stepwise Regression and Stepwise Discriminant Analysis

Need Not Apply here: A Guidelines Editorial. Educational and Psychological

Measurement, 55(4), 525–534.

Thomas Austin

Page 12 of 12

Wechsler, D. (1945). A standardized memory scale for clinical use. Journal of

Psychology, 19, 87–95.

Wechsler, D. (1981). Manual for the Wechsler Adult Intelligence Scale—Revised

(WAIS-R). San Antonio, TX: The Psychological Corporation.

Wragg, R. E., & Jeste, D. V. (1989). Overview of depression and psychosis in

Alzheimer’s disease. The American Journal of Psychiatry, 146(5), 577–587.