What is the most common biological test used to assess patients in a clinical assessment for a mental disorder?

Dialogues Clin Neurosci. 2012 Mar; 14(1): 91–99.

Language: English | Spanish | French

Aplicaciones clínicas de la evaluación neuropsicológica

Applications cliniques de l'évaluation neuropsychologique

Abstract

Neuropsychological assessment is a performance-based method to assess cognitive functioning. This method is used to examine the cognitive consequences of brain damage, brain disease, and severe mental illness. There are several specific uses of neuropsychological assessment, including collection of diagnostic information, differential diagnostic information, assessment of treatment response, and prediction of functional potential and functional recovery. We anticipate that clinical neuropsychological assessment will continue to be used, even in the face of advances in imaging technology, because it is already well known that the presence of significant brain changes can be associated with nearly normal cognitive functioning, while individuals with no lesions detectable on imaging can have substantial cognitive and functional limitations.

Keywords: neuropsychology, disability, schizophrenia, Alzheimer's disease

Abstract

La evaluación neuropsicológica es un método basado en el rendimiento para evaluar el funcionamiento cognitivo. Este método se emplea para examinar las consecuencias cognitivas del daño cerebral, de enfermedades cerebrales y de enfermedades mentales graves. Hay varios usos específicos de la evaluación neuropsicológica, que incluyen elementos para información diagnóstica y para el diagnóstico diferencial, para la evaluación de la respuesta terapéutica y para la predicción del potencial funcional y de la recuperación funcional. Se anticipa que se continuará aplicando la evaluación neuropsicológica clínica, a pesar de los avances en la tecnología de imágenes, ya que es bien sabido que la presencia de importantes cambios cerebrales pueden estar asociados con un funcionamiento cognitivo cercano a lo normal y que individuos sin lesiones detectables en las imágenes pueden tener limitaciones significativas tanto cognitivas como funcionales.

Résumé

L'évaluation neuropsychologique est une méthode basée sur la performance permettant d'évaluer le fonctionnement cognitif. Cette méthode est utilisée pour analyser les conséquences cognitives des lésions cérébrales, de la pathologie cérébrale et des maladies mentales sévères. II existe plusieurs utilisations spécifiques de l'évaluation neuropsychologique, comprenant le recueil d'informations diagnostiques, d'informations diagnostiques différentielles, d'évaluation de la réponse au traitement et de la prévision du potentiel fonctionnel et de la récupération fonctionnelle. Nous prévoyons que l'évaluation clinique neuropsychologique continuera à être utilisée malgré les avancées technologiques de l'imagerie, car il est déjà bien connu que des modifications cérébrales significatives peuvent être associées à un fonctionnement cérébral presque normal tandis que certaines personnes sans lésion détectable à l'imagerie peuvent présenter des limitations fonctionnelles et cognitives importantes.

Introduction

Neuropsychological assessment is the normatively informed application of performance-based assessments of various cognitive skills. Typically, neuropsychological assessment is performed with a battery approach, which involves tests of a variety of cognitive ability areas, with more than one test per ability area. These ability areas include skills such as memory, attention, processing speed, reasoning, judgment, and problem-solving, spatial, and language functions. These assessments are commonly performed in conjunction with assessments designed to examine lifelong academic and cognitive achievement and potential,1 for a variety of reasons described below. The assessment battery can be standardized or targeted to the individual participant in the assessment. Assessment data may be collected either directly by a psychologist or by a trained examiner, who performs and scores assessments and delivers them to the neuropsychologist. While neuropsychological assessments were originally targeted at individuals who had experienced brain injuries in wartime,2 the populations for whom neuropsychological assessments are useful spans the whole range of neuropsychiatric conditions.3

Neuropsychological tests are intrinsically performance-based. They are structured to require individuals to exercise their skills in the presence of an examiner/observer. Self-reports of functioning, as well as observations of behavior while performing testing, are critically important pieces of information, as described below. Self-reports of functioning are often affected by the presence of neuropsychiatric conditions,4 and do not have the same value as performance under standard conditions, which is compared with normative standards. A critical concept in neuropsychological assessment is normative comparison.5 This involves taking the performance of an individual at the time they are tested and comparing that performance to reference groups of the same age, sex, race, and educational attainment. All of these demographic factors impact performance on the tests in a neuropsychological assessment battery, and interpreting the test performance of people, regardless of the illness or injury that they have experienced, is based on comparisons with individuals who are similar to them. These normative comparisons allow for determination whether an individual is performing as would be expected, given their lifetime levels of achievements and their educational attainment, or if their performance is poorer than expected. Performance that is poorer than expectations can be quantified and interpreted accordingly.

Definition of a meaningful cognitive deficit

Neuropsychological assessment provides both general and specific information about current levels of cognitive performance. An average or composite score across multiple ability areas provides an overall index of how well a person functions cognitively at the current time. As noted below, these global scores are the most reliable results of a neuropsychological assessment. These global scores are the indices most commonly used to predict real-world functional milestones and to make judgments about functioning in conditions where multiple ability domains are affected (eg, serious mental illness or traumatic brain injury).6

However, it is also important to be able to make judgments about specific differential deficits across ability areas. For instance, an individual who experiences a focal stroke or brain injury may have limited cognitive deficits, with most abilities unchanged. Thus, when making a judgment about the presence of a single cognitive deficit such as amnesia or a broader condition such as dementia it is critical to be able to identify exactly what a “differential deficit” would be. This judgment process is complicated by the fact that healthy individuals with no evidence of, or risk factors for, neuropsychiatric conditions show some variability across their abilities.7 As a result, it is important to consider several different factors when identifying normal variation between ability areas from neuropsychological deficits.

There are several factors that impact on within-individual variation across cognitive ability areas. These include the reliability of the measures, the normative standards for the measures, and the level of performance of the individual. Tests with less reliability produce more variable scores at both single assessment and retest. The discrepancies between ability areas that can be interpreted as truly different from each other also depend on whether the normative standards for the tests were developed in a single sample (ie, co-normed) or separately.8 For example, meaningful differences between individual subtests on intelligence tests such as the Wechsler Adult intelligence scales9 are smaller than differences between tests that were developed completely separately from each other, because of their co-norming on a single sample. Likewise, normative comprehensive standards for extended neuropsychological assessment batteries have also been developed with the same purposes in mind.10 Finally, extremes in performance, both higher and lower, lead to greater apparent discrepancies between ability areas. This is because that, at the tails of the distribution, smaller absolute score differences lead to larger normative differences.

In terms of interpretation of meaningful differences between abilities in neuropsychiatric conditions, a widely accepted rule of for a clinically meaningful difference between two ability areas is about one -half of a standard deviation.11 This translates into about 7 IQ points and this level of difference has been shown to be detectable by observers. Specific, multiple studies have suggested that untrained observers can detect differences in functioning that occur over time that reach this threshold. As a result, treatment studies for cognitive impairments would not need to induce treatment effects smaller than this, because they might not be detectable.

It should be noted that the changes seen in many neuropsychiatric conditions are much more substantial than this 0.5 SD threshold. As a clear example, data regarding immediate memory changes, particularly rapid forgetting, at the outset of Alzheimer's disease (AD) are considerably more substantial than 0.5 SD. Data examining differences in performance across ability areas at the time of diagnosis has suggested memory performance about 3.0 SD below that of demographically similar healthy controls.12 Further, differential deficits between abilities at the time of diagnosis are also substantial. In that same, very large-scale study, memory performance was about 2.0 SD below that of confrontation naming at the time of diagnosis.13 Although subtle differences can be detected by observers as described above, many of the differences between abilities in neuropsychiatric conditions are not subtle.

Conditions where neuropsychological assessment provides useful information

Situations where an illness or injury has the potential to adversely impact on cognitive functioning is one where neuropsychological assessment is indicated. These situations include illnesses or injuries that directly impact on cognition (Degenerative dementias or traumatic brain injuries) or where the treatment for the illness impacts on cognitive functioning (chemotherapy for breast cancer). Finally, as neuropsychiatric conditions are complex, many of them have the potential to induce changes in mood or motivational states that can have secondary impacts on cognitive functioning. As these secondary impacts can cause cognitive changes that are as just as real as those caused by a brain injury, part of a comprehensive contemporary neuropsychological assessment requires an assessment of other factors that may be contributing to impaired cognitive functioning.

Information obtained from neuropsychological assessment

There are several different uses for neuropsychological assessments. These include assessment for the purpose of diagnosis, differential diagnosis, prediction of functional potential, measuring treatment response, and clinical correlation with imaging findings. Some of these uses are related to each other and some are impossible in certain circumstances, because neuropsychological assessments do not provide information helpful for these tasks. These uses are presented in Table I.

TABLE 1.

Uses of neuropsychological assessment.

Diagnosis

Some conditions are defined by the presence of cognitive impairment. A prototypical example is dementia as defined by the DSM-TV-TR.14 Dementia requires the presence of functional deficits and cognitive impairments. These impairments must be in two domains: memory, and one other cognitive deficit. In contrast to dementia, amnesia, also defined in DSM-TV-TR, requires only the presence of memory deficits for its diagnosis.

For these conditions, therefore, neuropsychological assessment would serve to provide diagnostic information, because the presence of specific or multiple cognitive deficits, including memory, would provide information for a diagnosis. Similarly there are other conditions, such as postconcussion syndrome where the presence of cognitive impairments of various types is required as a part of the diagnosis. Further, mental retardation requires the presence of a certain level of current intellectual functioning that can only be obtained psychometrically.

The way the DSM-TV-TR is structured, however, there is no diagnosis that is confirmed simply as a function of the data obtained in a neuropsychological assessment. In the case of dementia, for instance, there are multiple additional criteria that must be met as well, and many of these pieces of information are obtained from other sources. These include history (eg, prior better levels of functioning), assessment of current adaptive deficits, and identification of a potential cause of the condition. As a result, neuropsychological assessments are only part of the diagnostic process.

Due to the way the DSM-TV-TR is set up, neuropsychological assessment does not provide information relevant to the diagnosis of most conditions where cognitive impairments are present. For example, many serious mental illnesses are marked by the presence of substantial cognitive impairments. Schizophrenia,15 bipolar disorder,16 and major depression17 have substantial cognitive deficits as a common feature of their presentation, even in patients with current minimal levels of symptoms. Since these impairments are not part of the diagnostic criteria, neuropsychological assessment does not provide diagnostically relevant information. As noted below, however, there is considerable information that can be obtained from neuropsychological assessments in these conditions, particularly in functional and prognostic domains.

Differential diagnosis

There are some conditions where neuropsychological assessment can be important for differential diagnosis. As noted above, dementia requires memory deficits in the presence of other cognitive impairments, while amnesia is diagnosed by the presence of only deficits in memory. Detection of multiple cognitive impairments would therefore rule out the presence of amnesia and argue for a diagnosis of dementia in this case.

Differential diagnosis is much more challenging for most conditions, however. For example, studies attempting to differentiate between dementing conditions of different etiologies, such as vascular dementia as compared with AD, have found little evidence of differential diagnostic utility from neuropsychological assessment.18 In fact, a fascinating book by Zakzanis et al19 that broadly approached this topic has suggested that for many conditions there is very little differential diagnostic information contained in a neuropsychological assessment that even allows for differentiation between healthy populations and patients with a variety of neuropsychiatric conditions. Their meta-analysis includes all of the research published on neuropsychological test differences between healthy controls and several neuropsychiatric target populations during the years 1980-1997. As a result, there is a wealth of detail on how much information each of these neuropsychological tests provides for test-based differential diagnosis of the target populations compared with healthy comparison subjects.

It is important in this area to consider the differences between differential diagnosis and statistically significant differences in performance across different conditions. An effect size of .6 SD in the difference of two means, by convention a large effect and easy to detect in samples as small as 20 individuals per group, is associated with 62% overlap between the two samples. In order to be able to tell with 90% certainty that an individual's test score is consistent with a psychiatric or neurological diagnosis and not part of the lower end of the distribution of healthy, an average difference of about 2.5 SD between populations is required.

Many statistically significant differences between samples would fare poorly as candidates for differential diagnosis. For example, people with schizophrenia routinely have more significant cognitive deficits than people with bipolar disorder, regardless of the mood state of the bipolar patients.20 However, since bipolar patients themselves are more impaired in their cognitive performance than healthy people, there is substantial overlap in the distributions of cognitive performance between people with schizophrenia and bipolar disorder and minimal differential diagnostic information available. In contrast to the differences between people with AD and healthy populations on delayed recall memory, there is little discrimination between bipolar and schizophrenia populations. The distributions of patients with severe mental illness and healthy people have substantial overlap. As can be seen in Figure 1, there is considerable overlap in the distributions of scores on neuropsychological assessments of people with schizophrenia and healthy people, even if the means of the distributions are two full standard deviations apart. The r-BANS21 is an abbreviated neuropsychological assessment that examines multiple ability domains in a repeatable format. It is scaled like an IQ test, with a mean of 100 and standard deviation of 15 in healthy populations. As can be seen in Figure 1,22 people with schizophrenia have a mean level of performance that is 2.0 SD below that of healthy people (70 vs 100). However, half of the healthy population is performing within 2 SD of the mean of people with schizophrenia, and 35% of the people with schizophrenia perform within 2.0 SD of the mean of the healthy population. While a score of 115 would be much more rare for someone with schizophrenia than a healthy individual, a score of 85 would be at the 67th percentile for someone with schizophrenia and at the 17th for the healthy population; both of these are clearly within not outlying scores.

Normative data compared with a schizophrenia sample on the RBANS neuropsychological test. RBANS, Repeatable Battery for Assessments of Neuropsychological Status

An additional intriguing result of the Zakzanis et al analyses is that many of the tests that are often described as capturing fundamental characteristics of illnesses such as schizophrenia fare relatively poorly when evaluated with differential diagnostic standards. For instance, the Wisconsin Card Sorting test/23 a multidimensional test of executive functioning, is associated with 40% overlap between the performance of patients and healthy controls. In schizophrenia, in fact, the top five discriminators, all associated with 20% or less overlap, are in the domains of verbal and visuospatial memory. In the domain of chronic multiple sclerosis only 1 test is associated with less than 25% overlap between healthy individuals and MS patients, while many of the tests are associated with about 50% overlap between MS patients and healthy controls. These tests would provide essentially no data useful for differential diagnosis. There are some areas where there a number of excellent differential diagnostic candidates. In the domain of AD there are 15 different tests, all of memory, that are associated with less than 5% overlap between healthy controls and AD samples. Similarly, the difference between schizophrenia patients and AD patients on delayed recall memory was found to be similar to differences between healthy controls and AD patients.

Assessment of functional potential and the course of degenerative conditions

One of the more robust correlations in research in mental health is the association between cognitive performance and achievements in everyday functioning. This relationship has been appreciated for over 30 years and has been replicated across multiple neuropsychiatric conditions. Table II shows multiple examples of exactly this type of relationship. There are also several additional important points about these findings. These findings tend to be most robust for global aspects of cognitive performance, as indexed by performance on composite measures. In fact, in one recent study in severe mental illness the predictive power of a composite score for correlation with functional deficits was 2 to 3 times as great as any individual neuropsychological measure.30 Similarly, functional deficits in AD are more severe and debilitating after the illness has progressed, and there are multiple cognitive processes affected. Although it is quite possible to have functional deficits originating from a single residual cognitive deficit, on average more wide-ranging cognitive deficits, even if moderate in nature, leader to broader functional deficits. There will always be individual cases where a single, apparently delineated, cognitive deficit leads to gross impairment in functioning.

TABLE II.

Neuropsychiatric conditions where cognitive functioning predicts everyday functioning.

The most important clinical implication of what we know about cognition and functioning is this: when individuals affected by a neuropsychiatric condition are found to have current cognitive abilities congruent with pre-illness functioning they are least likely to have functional deficits. This is particularly true in conditions such as HIV neuropathology31 or traumatic brain injury (TBI)32 where changes can occur in the context of unimpaired previous functioning. Multiple studies of TBI have also have shown that recovery of cognitive functioning predicts recovery of everyday functioning much more efficiently than measures of the “severity” of the injury and some studies of TBI have had some success in the identification of the most efficient predictors of recovery of functioning. They tend to be from the domains of executive functioning and processing speed, but some studies also suggest that memory measures may be important (see ref 33, p 12).

It has proven difficult to establish absolute standards for how much impairment in cognitive functioning will definitely lead to functional changes. In addition, the search for specific cognitive to functional relationships has also proven challenging in conditions other than TBI. The group average data do suggest some general guidance, but clinical prediction will require analyses of specific cases. What is clear, however, is that neuropsychological assessment is an excellent tool for the prediction of recovery.

Assessment of changes in cognition in progressive degenerative conditions requires a different approach than required for the initial diagnosis of dementia or the assessment of improvement following TBI. If delayed recall performance is at a level that is close to 0 at the time that dementia is detected, this ability will not be a feature of the illness with the potential to change over time. In fact, research comparing individuals with AD at different levels of illness duration (and progressive course) have suggested that there is a pattern of progression in the worsening of cognitive impairments, with delayed recall nearly completely absent at the time of diagnosis, with other changes occurring in close temporal proximity, including reductions in rate of learning, executive functioning, and processing speed. Later on in the course, changes in longterm memory such as confrontation naming are detected and spatial and perceptual deficits become more severe.12-13 These changes are not necessarily uniform or predictable for individual cases and many individuals will manifest impairments in one ability area that are more severe than expected by their current stage of illness. What is clear from research, however, is that in individuals with AD and considerable cognitive impairments, functional performance tends to worsen quite markedly.

Measurement of recovery of functioning and treatment response

There is major interest in treatment of cognitive deficits in degenerative conditions, attention-deficit disorder, and severe mental illness. These approaches have ranged from in person and computerized cognitive remediation efforts to multiple pharmacological interventions. It makes sense that the same measures of cognitive functioning used to identify functionally relevant deficits across different neuropsychiatric conditions would be used to measure treatment outcomes. This approach has been used in multiple different studies, although there are some issues that require attention in interpreting the results of the studies. These include changes in performance that are due to random variation and practice effects and the fact that certain cognitive measures are more vulnerable to these effects than others, limiting their utility as outcome measures. One of the things that will render neuropsychological assessment consistently important is the new development of rehabilitation therapies. Development and marketing of computerized cognitive remediation interventions has not always been accompanied by the systematic assessment of their efficacy and long-term usefulness. It seems likely the performance on structured neuropsychological measures will continue to be the gold standard for selection of patients for these interventions and evaluation of their efficacy.

One of the strategies that has been developed to understand “real” cognitive improvements vs psychometric artifacts is the “reliable change index (RCI)” method.34 The RCI adjusts for expected practice effects and unreliability of measures in order to develop an index of change on an individual basis that would be definitely non-random. Essentially, a statistic is calculated that takes test scores at two different times and examines the difference between them, establishing a range of scores that could be attributed to practice effects or unreliability of measures. Differences that exceed this range are then considered to be reliable. Thus, measures with greater test-retest reliability and smaller practice effects in healthy controls would be better candidates for detection of small amounts of change that would still be clinically meaningful. Previous results in severe mental illness have suggested that changes in typically administered cognitive assessment batteries would need to be in the vicinity of 1.035-36 standard deviations on the part of an individual patient to be nonrandom, suggesting that quite substantial improvements may be required with current instrumentation.

Reduction, or at least the clear recognition, of practice effects is an important goal, because large practice effects in treatment studies on the part of the patients in the inactive treatment group can make it impossible to detect change in the treatment group.37 Certain measures are particularly vulnerable to such effects, and some of them may actually change in their characteristics upon repeated administration. Episodic memory tests are particularly vulnerable to practice effects, because of the possibility of learning of the content. However, it is critical to have alternate forms of such measures be closely equivalent, because if the alternate forms are different in their difficulty, an apparently improvement effect can be spuriously detected. Problem-solving tests are quite vulnerable to changes with retesting, because if there is only one problem, like in the widely used Wisconsin Card Sorting Test, once it is solved the test is no longer a problem-solving test. As a result, systematic efforts to develop problem-solving tests with similarly problems (like mazes) but with alternative stimuli have been conducted.

One of the major issues in using neuropsychological assessment as a sole outcome measure to measure either spontaneous recovery or treatment response is the lack of definitive information as to how much change is required to be important. In a sense, this is the converse of how much worsening due to illness or injury is significant, because both are equally hard to define without additional reference points. For an adequately powered randomized trial, separation of active treatment from inactive treatment is certainly one standard; one that will be applied by regulatory agencies. Another perspective is the empirically derived standard described above a ½ standard deviation improvement as having clinical meaning. A third strategy, which is optimal in certain circumstances where it can be applied, is that of using concurrent assessment of functional outcomes. As improvement in functioning is the goal of treatment of cognition, whenever possible improvements in functioning occur, accompanying cognitive improvements should be measured.

For instance, in a study of cognitive remediation in schizophrenia published a few years ago, the level of improvement in neuropsychological test performance on the part of patients was less than 0.5 SD compared with the inactive treatment group.38 However, the patients who received cognitive remediation were able to work much more effectively and earned more than 10 times as much money in the ensuing 3-year follow-up period compared with patients randomized to the inactive treatment.39 Thus, the cognitive improvement seen must have been adequate for some patients, in order for them to achieve such substantial functional gains.

The above study is different from many other studies because of its duration and because of the fact that patients who entered were all receiving a psychosocial intervention: supported employment. Such concurrent interventions have been shown to be a prerequisite for functional gains in cognitive remediation studies in severe mental illness.40 In studies where treatments are offered for briefer periods, such as pharmacological efficacy studies, or in cases where patients are not receiving concurrent psychosocial interventions, such outcome measures would not be practical. A suggested approach has been to use performance-based measures of functional capacity,41 which have shown considerable validity in terms of prediction of everyday outcomes and sensitivity to functional decline in very elderly patients with severe mental illness. These measures, because they capture ability and not everyday outcomes, do not require environmental opportunities to perform skills and have been shown to be sensitive to the effects of short-term behavioral interventions.

Clinical correlation

Among the exciting developments in medical technology has been the advent of high-resolution structural and functioning imaging of the brain. These techniques allow for highly precise examination of lesions associated with TBI and stroke, They also can identify potentially dangerous vascular abnormalities which may be repaired before catastrophic ruptures. Also possible is the visualization of previous “silent” ischemic changes, strokes, and other potential lesions. With the advent of ligands that can label amyloid;42 it will also likely be the case that many individuals will be informed that they have substantial potential to experience degenerative changes. A major question that arises after detection of any such a brain change is whether there is any functional importance of these changes. Given the consistent findings that cortical degenerative changes are often found at postmortem in individuals who had no observational evidence of deteriorated cognitive functioning during life,43 there will be considerable need to perform cognitive assessments following such scans. Similarly, serial neuropsychological assessment will likely provide better (and cheaper) information about changes in cognitive functioning than repeated scans.

Conclusions

Neuropsychological assessment has multiple clinical applications, ranging from collecting diagnostic information for dementia to predicting functionality and recovery from TBI. These assessments are not likely to be replaced by technology, because of the issues, reviewed immediately above, regarding the lack of clear prediction of cognition and functioning from cortical changes in late life. Neuropsychological testing does not provide differential diagnostic information for neuropsychiatric disorders, but it provides information that cannot be obtained anywhere else on abilities, motivation, and potential for future outcomes. There are likely to be new advances in assessment technology, but not assessment philosophy, over time, These improvements may include validly deliverable remote assessments and increased ease of administration of assessment tools. At this time, neuropsychological assessment has many uses and adds critical information to psychological, neurological, and neuroimaging assessments.

Acknowledgments

Dr Harvey has received consulting fees from Abbott Labs, Bristol Myers Squibb, En Vivo, Genentech, Johnson and Johnson, Merck and Company, Pharma Neuro Boost, Sunovion Pharma, and Takeda Pharma during the past year.

REFERENCES

1. Tracy Jl., McCrory AC., Josiassen RC., Monaco CA. A comparison of reading and demographic based estimates of premorbid intelligence in schizophrenia. Schizophr Res.1996;22:103–109. [PubMed] [Google Scholar]

2. Goldstein K. The Organism (reprinted from 1939 ed). Cambridge, MA: MIT Press-Zone Books; 1995 [Google Scholar]

3. Adams KM., Grant I. Neuropsychological Assessment of Neuropsychiatric and Neuromedical Disorders. 3rd ed. New York, NY: Oxford University Press, 2009 [Google Scholar]

4. Bowie CR., Twamley EW., Anderson H., Halpern B., Patterson T., Harvey PD. Self-assessment of functional status in schizophrenia. J Psychiatr Res.2007;41:1012–1018. [PMC free article] [PubMed] [Google Scholar]

5. Anastasi A., Urbina S. Psychological testing (7th ed). Upper Saddle River, NJ: Prentice-Hall;1997 [Google Scholar]

6. Neuchterlein KH., Green MF., Kern RS., et al. 2008, The MATRICS consensus cognitive battery: Part 1 . Test selection, reliability, and validity. Am J Psychiatry.2008;165:203–213. [PubMed] [Google Scholar]

7. Zakzanis KK., Jeffay E. Neurocognitive variability in high-functioning individuals: Implications for the practice of clinical neuropsychology. Psychol Rep.2011;108:290–300. [PubMed] [Google Scholar]

8. Kern RS., Nuechterlein KH., Green MF., et al. The MATRICS consensus cognitive battery: Part 2. Co-norming and standardization. Am J Psychiatry.2008;165:214–220. [PubMed] [Google Scholar]

9. Wechsler D. The Wechsler Adult Intelligence Scales. 4th ed. (WAIS-IV). San Antonio, Pearson Assessment:2009 [Google Scholar]

10. Heaton RK., Grant IS., Matthews CG. Comprehensive Norms for an Expanded Halstead-Reitan Battery: Demographic Corrections, Research Findings, and Clinical Applications. Odessa, FL: Psychological Assessment Resources; 1991 [Google Scholar]

11. Norman GR., Sloan JA., Wyrmech KW. Interpretation of changes in health related quality of life: the remarkable universality of 1/2 standard deviation. Med Care.2003;41:282–292. [PubMed] [Google Scholar]

12. Welsh KA., Butters N., Hughes J., Mohs RC., Heyman A. Detection of abnormal memory decline in mild cases of Alzheimer's disease using CERAD neuropsychological measures. Arch Neurol.1991;48:278–281. [PubMed] [Google Scholar]

13. Welsh KA., Butters N., Hughes J., Mohs RC., Heyman A. Detection and staging of dementia in Alzheimer s disease: use of the neuropsychological measures developed for the consortium to establish a registry for Alzheimer's disease (CERAD). Arch Neurol.1992;49:448–452. [PubMed] [Google Scholar]

14. American Psychiatric Association. Diagnostic and Statistical Manual of Mental Disorders. 4th ed, Text Revision. Washington, DC: American Psychiatric Association; 1997 [Google Scholar]

15. Heinrichs RW., Zakzanis KK. Neurocognitive deficit in schizophrenia: a quantitative review of the evidence. Neuropsychology.1998;12:426–45. [PubMed] [Google Scholar]

16. Simonsen C., Sundet K., Vaskinn A., Birkenaes AB., Engh JA., Hansen CF., et al. Neurocognitive profiles in bipolar I and bipolar II disorder: differences in pattern and magnitude of dysfunction. Bipolar Disord.2008;10:245–255. [PubMed] [Google Scholar]

17. Reichenberg A., Harvey PD., Bowie CR., et al. Neuropsychological function and dysfunction in schizophrenia and psychotic affective disorders. Schizophrenia Bull.2009;35:1022–1029. [PMC free article] [PubMed] [Google Scholar]

18. Thai LJ., Grundman M., Klauber MR. Dementia: characteristics of a referral population and factors associated with progression. Neurology.1988; 38:1083–1090. [PubMed] [Google Scholar]

19. Zakzanis K., Leach L., Kaplan E. Neuropsychological Differential Diagnosis. Lisse, the Netherlands: Swets and Zeitlinger; 1999 [Google Scholar]

20. Harvey PD., Wingo AP., Burdick KE., Baldessarini RJ. Cognition and disability in bipolar disorder: Lessons from schizophrenia research. Bipolar Disord.2010;12:364–375. [PubMed] [Google Scholar]

21. Randolph CR. The repeatable battery for neuropsychological screening. San Antonio, TX: Pearson Assessment; 1998 [Google Scholar]

22. Wilk CM., Gold JM., Humber K., Dickerson F., Fenton WS., Buchanan RW. Brief cognitive assessment in schizophrenia: normative data for the repeatable battery for the assessment of neuropsychological status. Schizophr Res.2004;70:175–186. [PubMed] [Google Scholar]

23. Heaton RK., Chellune CJ., Talley JL., Kay GG., Curtiss G. Wisconsin card sorting test manual-revised and expanded. Odessa, FL: Psychological Assessment Resources; 1993 [Google Scholar]

24. Heaton RK., Pendleton MG. Use of neuropsychological tests to predict patients everyday functioning. J Consult Clin Psychol.1981;49:807–821. [PubMed] [Google Scholar]

25. Parente FJ., Anderson JK. Use of the Wechsler Memory Scale for predicting success in cognitive rehabilitation. Cogn Rehab.1984;2:12–15. [Google Scholar]

26. Morris JC., McKeel DW., Storandt M. Very mild Alzheimer's disease: Informant-based clinical, psychometric, and pathologic distinction from normal aging. Neurology.1991;41:467–478. [PubMed] [Google Scholar]

27. Green MF., Kern RS., Braff DL., Mintz J. Neurocognitive deficits and functional outcome in schizophrenia: are we measuring the “right stuff?” Schizophr Bull.2000;26:119–136. [PubMed] [Google Scholar]

28. Bowie CR., Depp C., McGrath JA., et al. Prediction of real world functional disability in chronic mental disorders: a comparison of schizophrenia and bipolar disorder. Am J Psychiatry.2010;167:1116–1124. [PMC free article] [PubMed] [Google Scholar]

29. Janvin CC., Larsen JP., Salmon DP., Galasko D., Hugdahl K., Aarsland D. Cognitive profiles of individual patients with Parkinson's disease and dementia: comparison with dementia with Lewy bodies and Alzheimer's disease. Mov Disord. 2006;21:337–342. [PubMed] [Google Scholar]

30. McClure MM., Bowie CR., Patterson TL., et al. Correlations of functional capacity and neuropsychological performance in older patients with schizophrenia: evidence for specificity of relationships? Schizophr Res.2007;89:330–338. [PubMed] [Google Scholar]

31. Heaton RK., Marcotte TD., Mindt MR., et al. (HNRC Group): The impact of HIV-associated neuropsychological impairment on everyday functioning. J int Neuropsychological Soc.2004;10:317–331. [PubMed] [Google Scholar]

32. Lucas JA. Traumatic brain injury and postconcussive syndrome. In: Snyder PJ, Nussbaum PD, eds. Clinical Neuropsychology. Washington DC: American Psychological Association; 1998 [Google Scholar]

33. Lezak MD., Howieson DB., Loring DW. Neuropsychological Assessment. 4th ed. New York, NY: Oxford University Press; 2004 [Google Scholar]

34. Heaton RK., Temkin N., Dikmen S., et al. Detecting change: A comparison of three neuropsychological methods, using normal and clinical samples. Arch Clin Neuropsychol.2001;16:75–91. [PubMed] [Google Scholar]

35. Harvey PD., Palmer BW., Heaton RK., Mohamed S., Kennedy J., Brickman A. Stability of cognitive performance in older patients with schizophrenia: an 8-week test-retest study. Am J Psychiatry.2005;162:110–117. [PubMed] [Google Scholar]

36. Leifker FR., Patterson TL., Bowie CR., Mausbach BT., Harvey PD. Psychometric properties of performance-based measurements of functional capacity. Schizophr Res.2010;119:246–252. [PMC free article] [PubMed] [Google Scholar]

37. Goldberg TE., Keefe RS., Goldman R., Robinson DG., Harvey PD. Circumstances under which practice does not make perfect: a review of the practice effect literature in schizophrenia and its relevance to clinical treatment studies. Neuropsychopharmacology.2010;35:1053–1062. [PMC free article] [PubMed] [Google Scholar]

38. McGurk SR., Mueser KT., Pascaris A. Cognitive training and supported employment for persons with severe mental illness: one-year results from a randomized controlled trial. Schizophr Bull.2005;31:898–909. [PubMed] [Google Scholar]

39. McGurk SR., Mueser KT., Feldman K., Wolfe R., Pascaris A. Cognitive training for supported employment: 2-3 year outcomes of a randomized controlled trial. Am J Psychiatry.2007;164:437–441. [PubMed] [Google Scholar]

40. Wykes T., Huddy V., Cellard C., McGurk SR., Czobor P. A meta-analysis of cognitive remediation for schizophrenia: methodology and effect sizes. Am J Psychiatry.2011;168:472–485. [PubMed] [Google Scholar]

41. Harvey PD., Velligan Dl., Bellack AS. Performance-based measures of functional skills: Usefulness in clinical treatment studies. Schizophr Bull.2007; 33:1138–1148. [PMC free article] [PubMed] [Google Scholar]

42. Quigley H., Colloby SJ., O'Brien JT. PET imaging of brain amyloid in dementia: a review. Int J Geriatr Psychiatry.2011;26:991–999. [PubMed] [Google Scholar]

43. lacono D., Markesbery WR., Gross M., et al. The nun study: clinically silent AD, neuronal hypertrophy, and linguistic skills in early life. Neurology.2009;73:665–673. [PMC free article] [PubMed] [Google Scholar]

Articles from Dialogues in Clinical Neuroscience are provided here courtesy of Taylor & Francis