The term subthreshold syndrome in a categorical approach to diagnosis implies that the

Under a Creative Commons license

Open access

Highlights

•

When using a categorical approach (i.e., diagnoses), the majority of patients recovered from affective disorders at nine-year follow-up.

When using a dimensional approach (i.e., symptom severity), the majority of patients had a more chronic course.

The majority of patients who did not meet criteria for an affective disorder still experienced subthreshold symptomatology.

Abstract

Background

In longitudinal research, switching between diagnoses should be considered when examining patients with depression and anxiety. We investigated course trajectories of affective disorders over a nine-year period, comparing a categorical approach using diagnoses to a dimensional approach using symptom severity.

Method

Patients with a current depressive and/or anxiety disorder at baseline (N = 1701) were selected from the Netherlands Study of Depression and Anxiety (NESDA). Using psychiatric diagnoses, we described ‘consistently recovered,’ ‘intermittently recovered,' ‘intermittently recurrent’, and ‘consistently chronic’ at two-, four-, six-, and nine-year follow-up. Additionally, latent class growth analysis (LCGA) using depressive, anxiety, fear, and worry symptom severity scores was used to identify distinct classes.

Results

Considering the categorical approach, 8.5% were chronic, 32.9% were intermittently recurrent, 37.6% were intermittently recovered, and 21.0% remained consistently recovered from any affective disorder at nine-year follow-up. In the dimensional approach, 66.6% were chronic, 25.9% showed partial recovery, and 7.6% had recovered.

Limitations

30.6% of patients were lost to follow-up. Diagnoses were rated by the interviewer and questionnaires were completed by the participant.

Conclusions

Using diagnoses alone as discrete categories to describe clinical course fails to fully capture the persistence of affective symptoms that were observed when using a dimensional approach. The enduring, fluctuating presence of subthreshold affective symptoms likely predisposes patients to frequent relapse. The commonness of subthreshold symptoms and their adverse impact on long-term prognoses deserve continuous clinical attention in mental health care as well further research.

Keywords

Depression

Anxiety

Nine-year course

Prognosis

Chronicity

Diagnostic switching

© 2021 The Authors. Published by Elsevier B.V.

World Psychiatry. 2012 Feb; 11(1): 16–21.

Abstract

The method of diagnosing patients used since the early 1980s in psychiatry, which involves evaluating each of several hundred symptoms for their presence or absence and then applying idiosyncratic rules for combining them for each of several hundred disorders, has led to great advances in research over the last 30 years. However, its problems have become increasingly apparent, particularly for clinical practice. An alternative approach, designed to maximize clinical utility, is prototype matching. Instead of counting symptoms of a disorder and determining whether they cross an arbitrary cutoff, the task of the diagnostician is to gauge the extent to which a patient’s clinical presentation matches a paragraph-length description of the disorder using a simple 5-point scale, from 1 (“little or no match”) to 5 (“very good match”). The result is both a dimensional diagnosis that captures the extent to which the patient “has” the disorder and a categorical diagnosis, with ratings of 4 and 5 corresponding to presence of the disorder and a rating of 3 indicating “subthreshold” or “clinically significant features”. The disorders and criteria woven into the prototypes can be identified empirically, so that the prototypes are both scientifically grounded and clinically useful. Prototype diagnosis has a number of advantages: it better captures the way humans naturally classify novel and complex stimuli; is clinically helpful, reliable, and easy to use in everyday practice; facilitates both dimensional and categorical diagnosis and dramatically reduces the number of categories required for classification; allows for clinically richer, empirically derived, and culturally relevant classification; reduces the gap between research criteria and clinical knowledge, by allowing clinicians in training to learn a small set of standardized prototypes and to develop richer mental representations of the disorders over time through clinical experience; and can help resolve the thorny issue of the relation between psychiatric diagnosis and functional impairment.

Keywords: Prototype, diagnosis, classification, ICD-11, DSM-5, categorical diagnosis, dimensional diagnosis, comorbidity

Diagnosis includes two components: the way disorders are classified, and the way patients are diagnosed using that classification system. The DSM-III represented a pivotal moment in the evolution of both. First, it shifted from a classification system that had little grounding in empirical research to one that had at least modest grounding and, more importantly, created the conditions for an explosion of research on psychiatric disorders. Second, it shifted from a way of diagnosing patients with little reliability between any two clinicians or researchers to an approach that had high reliability for research purposes (using structured interviews) but continued to have considerable problems in clinical settings (see 1).

In the intervening decades, thousands of studies have focused on classification – e.g., whether adding, subtracting, or revising this or that diagnostic criterion might make some kind of difference in reliability or validity – yet little research has focused on how to make the diagnostic process more clinically useful, valid, and reliable. The assumption of the framers of subsequent editions of the DSM has been that clinicians need to change their ways and start diagnosing patients the way researchers do.

The problems with that assumption are multifold. DSM-IV-TR 2 is an 886-page manual. The idea that clinicians in everyday practice could, would, or should ask questions about each of hundreds of largely irrelevant criteria for hundreds of largely irrelevant disorders when a relatively high-functioning patient presents with, for example, anxiety symptoms and marital problems, is questionable at best. Further, many of the questions required to make a research diagnosis are unrelated to the tasks of clinical diagnosis and treatment. Whether a patient with bulimic symptoms has binged and purged twice a week every week for an arbitrarily specified period of time is far less useful to know clinically than that the patient is binging and purging frequently (e.g., daily, weekly, or multiple times a day) and that binge episodes seem to be preceded by feelings of rejection or abandonment.

The arbitrary nature of criteria for severity, duration, and number of symptoms met is not just a problem for clinical work but for research as well. In meta-analyzing the results of empirically supported therapies for some of the most prevalent disorders (e.g., mood and anxiety disorders), colleagues and I found that the average study excluded the majority of patients even considered for clinical trials because they did not meet rigid inclusion criteria or they had “comorbidities” that are in fact the norm, not the exception, in both research and clinical work 3. Further, clinical trials require categorical diagnoses as a prerequisite for entry into the study, yet virtually none uses them as a primary outcome measure, because a patient can lose just one or two symptoms of the disorder over the course of several weeks and thus appear to have “remitted” when he or she may in fact remain highly symptomatic. Instead, researchers use dimensional measures of constructs such as depression or anxiety as outcome criteria because they recognize that patients vary on the extent to which they are symptomatic, not just on whether they are symptomatic.

I could offer a long list of such concerns about the count-and-cutoff approach to diagnosis used in psychiatric diagnosis since 1980, such as the difficulty both clinicians and researchers have in remembering the criteria and complex diagnostic algorithms for?even the most common disorders, and the fact that the modal patient receives a low-information “not otherwise specified”(“NOS”) diagnosis in nearly every domain of the diagnostic manual, but will not enumerate such a list here (see 4,5). Suffice it to say that it is perhaps no surprise that a method of diagnosing patients designed for research purposes that was never tested empirically in any way against any alternative other than the failed DSM-I/DSM-II approach would itself run into problems over time, particularly as conceptions of psychopathology have changed (e.g., understanding most disorders as spectrum disorders or as present in varying degrees). The framers of ICD-10 attempted to coordinate with their DSM counterparts, but where they wisely parted company was in creating a distinct manual for clinical diagnosis that built in considerably more flexibility and a much more user-friendly format. The problem with diagnostic flexibility, of course, is that different clinicians can exercise that flexibility differently, leading to problems in reliability of diagnosis in clinical practice.

We have developed an alternative, prototype-matching approach to diagnosis, in which diagnosticians compare a patient’s overall clinical presentation to a set of diagnostic prototypes – for clinical use, paragraph-length descriptions of empirically identified disorders – and rate the “goodness of fit” or extent of match of the patient’s clinical presentation to the prototype. Rather than inquiring about each of several hundred symptoms, assessing whether the patient “has” each symptom, and then adding or otherwise combining symptoms (e.g., 3 from column A, 5 from column B) to determine whether the patient crosses a diagnostic threshold for “caseness”, the clinician uses all available data – including clinical observation, patients’ answers to questions, chart data, data from informants or past treatments, and the narratives the patient offers about his or her problems and relationships – to determine the extent to which the patient matches diagnostic descriptions that weave together diagnostic criteria into a memorable gestalt designed to facilitate pattern recognition.

In our prototype-matching procedure for clinical diagnosis 4,5,6,7, the diagnostician rates the patient on a 5-point scale for degree of match to the prototype (Figure ​1). The scale ranges from 1 (little or no match) to 5 (very good match – patient exemplifies this disorder; prototypical case), with ratings of 4 and 5 corresponding to categorical diagnosis and a rating of 3 indicating subthreshold or clinically significant features of the disorder (much as physicians measure blood pressure treated as a continuous variable but by convention refer to values in certain ranges as “borderline” or “high”). Thus, a single rating yields both a dimensional and a categorical diagnosis without relying on symptom counting. The default value for each diagnosis is 1 (little or no match), so that clinicians only expend time rating prototypes of disorders relevant to the patient, allowing rapid diagnosis. The easy translation of dimensional into categorical diagnosis (e.g., a 3 translating to clinically significant features) is of particular use for communication among professionals, who are unlikely to find it useful to describe a patient as “3 on major depression, 2 on panic” (one of the major limitations of potential dimensional approaches to psychiatric diagnosis). An emerging body of research suggests that clinicians can make prototype judgments of this sort for a wide range of syndromes, from mood and anxiety disorders to personality disorders, with high degrees of reliability 7,8,9,10. The complexity of this diagnostic method can be expanded as much as desirable, for example, by adding secondary ratings of severity or duration or empirically identified aspects of the disorder (e.g., severity of depressive phenomenology, vegetative symptoms, or melancholic symptoms for major depressive disorder; age of onset and severity of attentional deficits, hyperactivity, and impulsivity for attention-deficit/hyperactivity disorder).

Before briefly highlighting some of the advantages of this approach, three points are worth noting. First, both the polythetic diagnostic criteria built into DSM-IV (which require that a patient meet a certain number but not all of the criteria for a disorder) and particularly the clinical manual of the ICD-10 are essentially efforts to operationalize prototype matching. In the case of DSM-III-R and DSM-IV, that goal was explicit, based on early research on prototypes in the emerging field of cognitive science 11. The clinician version of the ICD-10 is already close to a prototype-matching procedure, in which clinicians are presented with what are usually paragraph-length descriptions of a disorder, often with an additional set of considerations, and are instructed to diagnose the patient based on their knowledge of the patient at the time (e.g., after a single session or months of treatment) with whatever degree of certainty they feel comfortable. What the current manual lacks is a way of operationalizing clinical judgment to maximize reliability, so that clinicians can feel confident that when a patient comes to them with a particular diagnosis – or when they diagnose a patient themselves – that diagnosis is as accurate and as clinically useful as practical.

Second, as Reed et al 12 have just shown in a WPA-World Health Organization (WHO) survey of nearly 5,000 psychiatrists across over 40 countries, most practicing psychiatrists, including users of both the ICD and the DSM, prefer a diagnostic method that has many of the features associated with prototype diagnosis, for reasons to be described shortly. Psychiatrists by large percentages preferred approaches that offer flexible rather than strict criteria-based diagnosis; keep the number of psychiatric diagnoses down to a manageable number (somewhere under 30-100); are clinician-friendly and clinically useful (e.g., in allowing clinicians to communicate their diagnostic judgments and to make useful treatment decisions based on them); and allow clinicians to represent dimensional aspects of the patient’s presentation in ways that accurately capture clinical reality (e.g., diagnosing pathology that does not meet criteria for a categorical diagnosis but is nonetheless clinically significant).

Third, as the International Advisory Group for the Revision of ICD-10 Mental and Behavioural Disorders for the WHO has recently noted 13, a diagnostic manual has many uses, including clinical, research, teaching and training, statistical, and public health. No approach is likely to be equally helpful or optimal for all of these uses, but a method of diagnosing patients should be reasonably useful for all of them, and in particular should have clinical utility, including utility in guiding treatment and public health. As will be seen shortly, prototype diagnosis has advantages in each of these domains.

ADVANTAGES OF PROTOTYPE DIAGNOSIS

Prototype diagnosis has a number of advantages. First, it better fits the ways humans naturally think and classify. People (in this case, diagnosing clinicians) tend to categorize complex, novel stimuli (in this case, patient presentations) through a probabilistic assessment of degree of match to a mental model they have formed (a prototype) or prominent exemplars of potentially relevant categories 14,15,16. Research in cognitive science suggests that, in everyday judgment and decision-making, people tend to satisfice (a cross between satisfy and suffice), that is, to make a “good-enough” assessment for their purposes, and to make more precise determinations based on explicit decision rules if the need arises 17,18,19. For example, rather than getting out the ICD-10 or DSM-IV to decide whether a patient with moderate panic symptoms once or twice a week meets formal criteria, most clinicians would diagnose the patient as suffering from moderate, clinically significant panic symptoms, whether or not the patient met formal diagnostic criteria. In light of the dearth of research on the treatment implications of clinical versus subthreshold symptoms and of data suggesting that subthreshold variants often produce similar levels of functional impairment 20,21, satisficing is not an irrational diagnostic strategy in clinical practice.

Compare this approach to the current diagnostic procedures, which were derived from the Research Diagnostic Criteria of the 1970s 22, and require clinicians to remember hundreds of lists of symptoms and, equally problematic, hundreds of distinct decision rules that differ for each disorder. Even putting aside the question of the validity of those lists and cutoffs, humans have trouble remembering long lists, let alone forming a coherent representation based on them. Expecting clinicians to remember how to combine the items from those lists to make a diagnosis is impractical, particularly when the precise number of symptoms from one subcategory or another may be relevant to making a “correct” diagnosis for research purposes but not for clinical practice. The DSM-IV diagnosis of post-traumatic stress disorder (PTSD), for example, requires at least one symptom of re-experiencing, three of avoidance/numbing, and two of hyperarousal. What matters to the clinician, in contrast, is the “gist” – that the patient experienced a traumatic event and is having some combination of these symptoms to varying degrees – in ways that should influence clinical decision making and treatment.

This leads to the second advantage of prototype diagnosis: clinical utility. In multiple studies by multiple research teams, clinicians have rated prototype diagnosis as substantially more clinically useful than the more familiar DSM-IV system and alternative dimensional systems for a range of disorders on a range of measures, from utility in communicating with other clinicians to ease of use 5,7,8,23,24. Perhaps not surprisingly, a growing body of research finds that prototype diagnosis, unlike diagnosis using strict operational criteria, is highly reliable in clinical practice, with correlations typically ranging from .50 to .70 between two clinicians 10,25.

One of the reasons clinicians prefer?prototype diagnosis is its third advantage, namely that it allows them to represent what they observe with their patients and to communicate it to other mental health providers both dimensionally and categorically, and to do so with relative ease. Whereas dimensional diagnosis is probably most precise in most cases, and categorical diagnosis is most familiar and feels most “natural”, prototype diagnosis captures the advantages of both. Consider the DSM-IV category of eating disorders, which includes two diagnoses with two subtypes each – anorexia nervosa with restricting and binge-purging subtypes and bulimia nervosa with purging and non-purging subtypes – as well as an NOS diagnosis, for a total of five categories (and others reportedly on the way). This might be a relatively small number of disorders for an eating disorders specialist, but for the general practitioner – let alone the primary care practitioner – five disorders with multiple criteria and differing cutoffs for each is difficult to remember, particularly when a given eating disorder is just one of the hundreds of disorders in the manual. Even for specialists and researchers, the diagnostic challenge is substantial, as research suggests that this approach relegates roughly half of patients with clinically significant eating pathology to a nondescript NOS category; that over 60 percent of patients diagnosed with some variant of anorexia “switch” to a bulimia nervosa diagnosis at some point and vice versa; and that those with concurrent symptoms of both disorders are arbitrarily classified as a subtype of anorexia 3.

Perhaps not surprisingly, in a study of North American psychiatrists and psychologists treating at least one eating-disordered patient, clinicians overwhelmingly preferred prototype diagnosis to the more familiar DSM-IV system (see 8). In part this likely reflects ease of use, because prototype diagnosis vastly decreases the number of disorders that have to be included in the diagnostic manual. To cover the range of eating pathology, we presented clinicians with only two prototypes: anorexia nervosa (a syndrome characterized by self-starvation) and bulimia nervosa (a syndrome characterized by binging and purging), with both prototypes taken directly from DSM-IV criteria except without the arbitrary severity and duration criteria (clinicians also made secondary ratings such as severity of binging, severity of purging, and severity of weight loss or gain). Rather than counting symptoms and deciding whether the patient met arbitrary requirements and cutoffs (e.g., bingeing and purging at least twice a week for a minimum of 3 months), the clinician’s task was simply to rate the extent to which the patient’s condition matched each prototype. A score of 4 or 5 on the bulimia prototype meant that the patient’s symptom picture strongly enough matched the diagnostic prototype to warrant a categorical diagnosis. A score of 3 on the anorexia prototype meant that the patient’s symptom picture resembled the prototype but not enough to warrant a categorical diagnosis. A patient with both ratings would thus receive a categorical diagnosis of “bulimia nervosa with anorexic features”.

A fourth advantage of prototype diagnosis is that it allows greater flexibility and validity not only in the diagnostic process but also in the definitions of disorders and the criteria that can be included in the prototypes. In the study described above, we simply combined the diagnostic criteria of each of the two major eating disorder syndromes thematically to create the prototypes. In other research, however, we have derived the disorders and criteria that create the prototypes empirically from large samples, using clinicians’ detailed ratings of actual patients in their practice, and relied on statistical procedures such as factor analysis to identify both the disorders to be included and the criteria for those disorders. This allows not only for the development of empirically valid disorders without the need for committee wrangling over which disorders or criteria to include – including culturally relevant or culturally specific disorders that might emerge from a factor analysis in one culture but not in another – but also for clinically richer diagnostic descriptions. For example, Westen and Shedler 26 derived a set of personality prototypes in this way that in many respects resembled the DSM-IV and ICD-10 personality disorders but included a number of further subtle psychological diagnostic criteria that are important clinically and emerged empirically. These have been absent from official diagnostic criteria because they could not be readily self-reported by patients in structured interviews (e.g., for obsessive-compulsive personality disorder, “Is invested in seeing and portraying himself or herself as emotionally strong, untroubled, and emotionally in control, despite clear evidence of underlying insecurity, anxiety, or distress”).

Developing prototypes this way substantially reduces artifactual “comorbidity”, by identifying groupings of patients or criteria that are distinct from others. Even using prototypes derived from current overlapping diagnostic categories and criteria, prototype diagnosis inherently reduces artifactual comorbidity, because clinicians are making conral judgments, not judgments about isolated symptoms. Consider PTSD, which is often found to be comorbid with mood disorders. Some of that comorbidity is undoubtedly accurate. In other cases, however, that comorbidity is an artifact of current diagnostic methods. For example, dysphoria associated with anhedonia and general hopelessness is clearly part of a depressive picture and hence would contribute to a diagnosis of major depression or dysthymia using a prototype system. In contrast, dysphoria associated with persistent thoughts of a traumatic event or survivor guilt would likely be represented as part of the patient’s PTSD.

A fifth advantage of prototype diagnosis is its utility in integrating teaching, training, and subsequent clinical experience. The goal of prototype diagnosis is to help clinicians develop mental representations of different kinds of disorder and, equally important, to standardize those representations across diagnosticians. Instead of trying to memorize symptom lists, the goal is to form mental representations of coherent syndromes, in which signs and symptoms are functionally related 4,8,23. This approach parallels the way the brain functions, working with rather than against naturally occurring cognitive processes 16,27,28,29.

The goal of teaching and training thus becomes to help trainees master a relatively small number of disorders grouped into a relatively small number of categories (e.g., psychotic disorders, mood disorders, substance use disorders, eating disorders, personality disorders, developmental disorders), with typically two to ten prototypes within each category (for a total of 30-100 disorders, depending on the number of non-overlapping disorders that emerges empirically). Formal training in diagnosis would entail learning the prototypes, perhaps with two or three exemplars for each disorder included in the diagnostic manual, and supervision on diagnosis would focus not only on diagnostic interviewing skills but also on learning to recognize the conrations, to rate them, and to make differential diagnoses. Rather than years of experience leading clinicians increasingly to ignore the symptom lists in the diagnostic manual as they learn the complexities of the disorders in real clinical practice, years of experiences would help clinicians “flesh out” and enrich their earlier mental prototypes, essentially “hanging” new?experiences onto prototypes they first learned about while in training.

Finally, prototype diagnosis separates two closely related but non-identical ?questions addressed by the WHO’s Advisory Board 13, namely the extent to which a patient has a given disorder and the extent of functional impairment. Prototype ratings are ratings of degree of match to a disorder. Although all disorders create some degree of dysfunction, they vary tremendously in how much and what kind of dysfunction they produce, which also vary by individual, depending on his or her social, psychological, and other resources. Thus, some patients can perform remarkably well occupationally despite being dysthymic, whereas for others dysthymia is debilitating. Capturing the degree of disability is not the same as capturing the degree to which a patient matches a diagnostic prototype, although we have found that prototype ratings can be extremely useful in measuring the extent to which patients match a prototype of psychological health (see 7).

CONCLUSIONS

By describing the advantages of pro?totype diagnosis, I do not mean to suggest that it is without limitations (although its detractors will no doubt lay those out with great clarity in the pages that follow). Perhaps the most important disadvantage of prototype diagnosis is that it could foster confirmatory biases and other heuristics that can lead clinicians, like all humans, to see what they expect to see, or to stick with hypotheses about a patient despite disconfirming information. Whether it does so more than the approaches in DSM-IV or ICD-10 is an empirical question, although it is certainly possible that encouraging clinicians to match patients to prototypes could make them more likely to gloss over disconfirming data or to adhere doggedly to early diagnostic hypotheses.

The best antidotes to these kinds of diagnostic biases are threefold, all of which we should be teaching young clinicians and practicing throughout our years of clinical experience, regardless of systems of diagnosis. The first is to understand the cognitive and emotional biases to which our minds are naturally prone, most of which are unconscious, and to exercise rigorous and continuous self-examination to try to minimize those biases in working with our patients. The second is to teach our trainees routinely to ask themselves (and to ask ourselves) to generate alternative hypotheses to those we are currently entertaining to understand what we are seeing clinically. The third is always to supplement descriptive diagnosis with functional diagnosis, where the question is not “What does this patient have”, but “Under what conditions does this patient think, feel, and behave in this particular way (e.g., why is this patient depressed to this degree now, and under what circumstances does he function differently)?”. By asking this last question, we not only challenge any diagnostic complacency we may have, but we also focus on what matters most in clinical diagnosis, namely on understanding how this particular patient’s mind and brain work and under what circumstances they may work differently.

Acknowledgement

This research was supported by National Institute of Mental Health grants R01-MH62377 and R01-MH78100.

References

1. Maj M. Psychiatric diagnosis: pros and cons of prototypes vs. operational criteria. World Psychiatry. 2011;10:81–82. [PMC free article] [PubMed] [Google Scholar]

2. American Psychiatric Association. Diagnostic and statistical manual of mental disorders, 4th ed., text revision. Washington: American Psychiatric Association; 2000. [Google Scholar]

3. Westen D, Novotny CM, Thompson-Brenner H. The empirical status of empirically supported psychotherapies: assumptions, findings, and reporting in controlled clinical trials. Psychol Bull. 2004;130:631–663. [PubMed] [Google Scholar]

4. Westen D, Bradley R. Prototype diagnosis of personality. In: Strack S, editor. Handbook of personology and psychopathology. New York: Wiley; 2005. pp. 238–256. [Google Scholar]

5. Westen D, Heim A, Morrison K. Simplifying diagnosis using a prototype-matching approach: implications for the next edition of the DSM. In: Beutler LE, Malik ML, editors. Rethinking the DSM: a psychological perspective. Washington: American Psychological Association; 2002. pp. 221–250. [Google Scholar]

6. Westen D, Shedler J. A prototype matching approach to diagnosing personality disorders: toward DSM-V. J Pers Disord. 2000;14:109–126. [PubMed] [Google Scholar]

7. Westen D, Shedler J, Bradley R. A prototype approach to personality disorder diagnosis. Am J Psychiatry. 2006;163:846–856. [PubMed] [Google Scholar]

8. Ortigo KM, Bradley B, Westen D. An empirically based prototype diagnostic system for DSM-V and ICD-11. In: Millon T, Krueger R, Simonsen E, editors. Contemporary directions in psychopathology: scientific foundations of the DSM-V and ICD-11. New York: Guilford; 2010. pp. 374–390. [Google Scholar]

9. DeFife JA, Peart J, Bradley B. The validity of prototype diagnosis in everyday practice. Submitted for publication [PubMed] [Google Scholar]

10. DeFife JA, Westen D, Bradley B. Prototype personality diagnosis in clinical practice: a viable alternative for DSM-5 and ICD-11. Prof Psychol Res Pr. 2010;41:482–487. [PMC free article] [PubMed] [Google Scholar]

11. Widiger T, Frances A. The DSM-III personality disorders: perspectives from psychology. Arch Gen Psychiatry. 1985;42:615–623. [PubMed] [Google Scholar]

12. Reed GM, Correia JM, Esparza P. The WPA-WHO global survey of psychiatrists’ attitudes towards mental disorders classification. World Psychiatry. 2011;10:118–131. [PMC free article] [PubMed] [Google Scholar]

13. International Advisory Group for the Revision of ICD-10 Mental and Behavioural Disorders. A conceptual framework for the revision of the ICD-10 classification of mental and behavioural disorders. World Psychiatry. 2011;10:86–92. [PMC free article] [PubMed] [Google Scholar]

14. Folstein JR, Van Petten C. Multidimensional rule, unidimensional rule, and similarity strategies in categorization: event-related brain potential correlates. J Exp Psychol Learn Mem Cogn. 2004;30:1026–1044. [PubMed] [Google Scholar]

15. Medin DL. Concepts and conceptual structure. Am Psychol. 1989;44:1469–1481. [PubMed] [Google Scholar]

16. Rosch E, Mervis CB. Family resemblances: studies in the internal structure of categories. Cogn Psychol. 1975;7:573–605. [Google Scholar]

17. Gigerenzer G, Goldstein DG. Reasoning the fast and frugal way: models of bounded rationality. Psychol Rev. 1996;103:650–669. [PubMed] [Google Scholar]

18. Gigerenzer G, editor. Gut feelings: the intelligence of the unconscious. New York: Penguin Group; 2008. [Google Scholar]

19. Simon HA. Information-processing theory of human problem solving. In: Estes WK, editor. Handbook of learning and cognitive processes. Hillsdale: Erlbaum; 1978. [Google Scholar]

20. Fava GA, Mangelli L. Assessment of subclinical symptoms and psychological well-being in depression. Eur Arch Psychiatry Clin Neurosci. 2001;251:47–52. [PubMed] [Google Scholar]

21. Marshall RD, Olfson M, Hellman F. Comorbidity, impairment, and suicidality in subthreshold PTSD. Am J Psychiatry. 2001;158:1467–1473. [PubMed] [Google Scholar]

22. Spitzer RL, Endicott J, Robins E. Research Diagnostic Criteria: rationale and reliability. Arch Gen Psychiatry. 1978;35:773–782. [PubMed] [Google Scholar]

23. Rottman B, Ahn W, Sanislow C. Can clinicians recognize DSM-IV personality disorders from five-factor model descriptions of patient cases? Am J Psychiatry. 2009;166:427–433. [PMC free article] [PubMed] [Google Scholar]

24. Spitzer R, First M, Shedler J. Clinical utility of five dimensional systems for personality diagnosis: a “consumer preference” study. J Nerv Ment Dis. 2008;196:356–374. [PubMed] [Google Scholar]

25. Westen D, Muderrisoglu S. Reliability and validity of personality disorder assessment using a systematic clinical interview: evaluating an alternative to structured interviews. J Pers Disord. 2003;17:350–368. [PubMed] [Google Scholar]

26. Westen D, Shedler J. Revising and assessing Axis II, Part 2: toward an empirically based and clinically useful classification of personality disorders. Am J Psychiatry. 1999;156:273–285. [PubMed] [Google Scholar]

27. Kim NS, Ahn W. Clinical psychologists’ theory-based representations of mental disorders predict their diagnostic reasoning and memory. J Exper Psychol. 2002;131:451–476. [PubMed] [Google Scholar]

28. Cantor N, Genero N. Psychiatric diagnosis and natural categorization: a close analogy. In: Millon T, Klerman GL, editors. Contemporary directions in psychopathology: toward the DSM-IV. New York: Guilford; 1986. pp. 233–256. [Google Scholar]

29. Ahn W. Effect of causal structure on category construction. Memory & Cognition. 1999;27:1008–1023. [PubMed] [Google Scholar]

Articles from World Psychiatry are provided here courtesy of The World Psychiatric Association