8
Pergamon 0957-4174(95)00033-X Expert Systems With Applications, Vol. 10, No. 1, pp. 55-62, 1996 Copyright © 1996 Elsevier Science Ltd Printed in Great Britain. All rights reserved 0957-4174/96 $15.00 + 0.00 A Knowledge-Based Support System for Psychological Analysis G. DARYL NORD* AND JERETTA HORN NORD Collegeof BusinessAdministration, OklahomaStateUniversity, Stillwater, OK 74078 Abstract--An expert system was designed and developed to assist a cfinician in the administration and preliminary analysis of the Minnesota Multiphasic Personality Inventory (MMPI) test. The inventory is a standardized test developed in the late 1930s designed to assess attitudes, character, behavior, and in general personality traits of individuals. Today the test is widely used not only for clinical purposes but also for employment screening and research. The expert system encompasses a computer program that: (1) administers the personality inventory via computer, (2) scores the test results based on standard norms, and (3) passes the scores to a rule-based expert system that renders a preliminary psychological profile of the test taker to the clinician. The system can be exploited for its educational value as well as provide valuable assistance to the clinician. 1. INTRODUCTION EXPERT SYSTEMSdesigned to mimic and replace human experts have attracted extensive attention in the past several decades. Their development advanced from efforts of artificial intelligence (AI) researchers to achieve the goal of building computer programs that could "think", that is, solve problems in a way con- sidered by human beings to be intelligent (Holsapple & Whinston, 1987; Pigford & Baur, 1990). As early as the 1950s, scientists began writing symbolic, domain specific computer programs. Then in 1956, Newell, Shaw and Simon developed an AI program named Logic Theorist that used reverse reason- ing to solve problems in propositional calculus (Olson & Courmey Jr, 1992; Quinlan, 1987). Dendral, a system that identifies molecular structures of materials using mass spectrometer data was later developed in the mid 1960s. AI researchers continued to initiate numerous expert system applications in the 1970s. Mycin, a rule- based system consisting of 400 heuristic rules that diagnoses blood diseases, is one of the more notable projects (Newell, Shaw & Simon, 1963). Beginning in the 1980s and continuing into this decade, many researchers decided that AI had matured to the point that it belonged in business, government, and related applications. This decision has accelerated the * Requests for reprints should be sent to: G. Daryl Nord, College of Business Administration, Oklahoma State University,Stillwater, OK 74078-0555. USA. research and development of AI and specifically expert system technologies. This paper discusses the development of an expert system designed to test and analyze psychological traits of individuals. First, the domain in the form of the psychological test will be examined; then the expert system development software that was utilized will be discussed; and, finally, the detailed architecture of the expert system will be presented. 2. THE PSYCHOLOGICAL TEST DOMAIN In the late 1930s, Starke R. Hathaway, a medical psychologist, and John C. McKinley, a neuropsychiatrist, developed a personality study called the Minnesota Multiphasic Personality Inventory (MMPI). The psycho- logical inventory consists of a test of over 500 questions designed to focus and identify specific personality traits of individuals. The test assesses character, attitudes, and behavior by the patterns of responses to true, false, or cannot say questions. Originally designed to help identify those with psychological problems, today the test is widely used in employment screening, clinical practice, and research. Prospective law enforcement officers, nurses, nuclear power plant personnel, and even the Russian cosmonauts are required to take the MMPI. The test is used in 46 countries around the world and has been translated into 124 languages. MMPI scores have been assembled on nearly every kind of medical and psychiatric population as well as on the young, the old, prisoners, religious 55

A knowledge-based support system for psychological analysis

Embed Size (px)

Citation preview

Page 1: A knowledge-based support system for psychological analysis

Pergamon

0957-4174(95)00033-X

Expert Systems With Applications, Vol. 10, No. 1, pp. 55-62, 1996 Copyright © 1996 Elsevier Science Ltd

Printed in Great Britain. All rights reserved 0957-4174/96 $15.00 + 0.00

A Knowledge-Based Support System for Psychological Analysis

G. DARYL NORD* AND JERETTA HORN NORD

College of Business Administration, Oklahoma State University, Stillwater, OK 74078

Abstract--An expert system was designed and developed to assist a cfinician in the administration and preliminary analysis of the Minnesota Multiphasic Personality Inventory (MMPI) test. The inventory is a standardized test developed in the late 1930s designed to assess attitudes, character, behavior, and in general personality traits of individuals. Today the test is widely used not only for clinical purposes but also for employment screening and research. The expert system encompasses a computer program that: (1) administers the personality inventory via computer, (2) scores the test results based on standard norms, and (3) passes the scores to a rule-based expert system that renders a preliminary psychological profile of the test taker to the clinician. The system can be exploited for its educational value as well as provide valuable assistance to the clinician.

1. INTRODUCTION

EXPERT SYSTEMS designed to mimic and replace human experts have attracted extensive attention in the past several decades. Their development advanced from efforts of artificial intelligence (AI) researchers to achieve the goal of building computer programs that could "think", that is, solve problems in a way con- sidered by human beings to be intelligent (Holsapple & Whinston, 1987; Pigford & Baur, 1990).

As early as the 1950s, scientists began writing symbolic, domain specific computer programs. Then in 1956, Newell, Shaw and Simon developed an AI program named Logic Theorist that used reverse reason- ing to solve problems in propositional calculus (Olson & Courmey Jr, 1992; Quinlan, 1987). Dendral, a system that identifies molecular structures of materials using mass spectrometer data was later developed in the mid 1960s. AI researchers continued to initiate numerous expert system applications in the 1970s. Mycin, a rule- based system consisting of 400 heuristic rules that diagnoses blood diseases, is one of the more notable projects (Newell, Shaw & Simon, 1963).

Beginning in the 1980s and continuing into this decade, many researchers decided that AI had matured to the point that it belonged in business, government, and related applications. This decision has accelerated the

* Requests for reprints should be sent to: G. Daryl Nord, College of Business Administration, Oklahoma State University, Stillwater, OK 74078-0555. USA.

research and development of AI and specifically expert system technologies.

This paper discusses the development of an expert system designed to test and analyze psychological traits of individuals. First, the domain in the form of the psychological test will be examined; then the expert system development software that was utilized will be discussed; and, finally, the detailed architecture of the expert system will be presented.

2. THE PSYCHOLOGICAL TEST DOMAIN

In the late 1930s, Starke R. Hathaway, a medical psychologist, and John C. McKinley, a neuropsychiatrist, developed a personality study called the Minnesota Multiphasic Personality Inventory (MMPI). The psycho- logical inventory consists of a test of over 500 questions designed to focus and identify specific personality traits of individuals. The test assesses character, attitudes, and behavior by the patterns of responses to true, false, or cannot say questions.

Originally designed to help identify those with psychological problems, today the test is widely used in employment screening, clinical practice, and research. Prospective law enforcement officers, nurses, nuclear power plant personnel, and even the Russian cosmonauts are required to take the MMPI. The test is used in 46 countries around the world and has been translated into 124 languages. MMPI scores have been assembled on nearly every kind of medical and psychiatric population as well as on the young, the old, prisoners, religious

55

Page 2: A knowledge-based support system for psychological analysis

56 G. D. Nord and J. H. Nord

groups, and ethnic subgroups. Additionally, over 5000 published research studies have been conducted on the MMPI (Holden, 1986; Parker, Hanson & Hunsley, 1988).

Items that compose the primary clinical scales of the MMPI were selected essentially by empirical means. Response patterns obtained from criterion groups with known psychological and behavioral characteristics were compared with contrast groups of normal people, medical patients, and persons with various psychiatric diagnoses. Thus, the various scales of the MMPI contain diverse item content; in addition, some test items are included on more than one scale because of the empirical selection procedures that were implemented. The origi- nal test included 13 clinical scale domains briefly described below (Colligan and Offord, 1988):

1. Social Maladjustment. The item content of this scale approximates the concept of introversion-extraversion. High scores tend to indicate that the person is socially bashful, shy, embarrassed, self-conscious, and extremely resereved; and low scores indicate that the person is probably gregarious, confident, assertive, and fun-loving and relates quickly and easily to others.

2. Depression. High scorers have classic depression with feelings of guilt, regret, worry, unhappiness, and loss of concentration and motivation; reduced self- esteem, anxiety, and apprehension are present; these persons are oversensitive and likely to feel misunder- stood, unworthy, and deserving of punishment.

3. Feminine Interest. High scorers describe liking femi- nine games, hobbies, and vocations and deny liking masculine games, hobbies and vocations. High scores also may result from expressing a preference for a broad range of interests and those viewed as socially desir- able.

4. Poor Morale. High scores indicate a lack of self- confidence, feelings of failure and despair, and tendencies to give up hope. Persons with high scores are likely to be oversensitive to the feelings and reactions of others, to feel misunderstood and useless, and to be socially suggestible.

Although there is overlap among Poor Morale, Social Maladjustment, and Depression, Poor Morale empha- sizes a lack of self-confidence and hypersensitivity to the opinions of others, whereas feelings of guilt and apprehension are stronger on Depression and a lack of social poise is prominent on Social Maladjustment.

5. Religious Fundamentalism. A high score indicates that the person is religious and churchgoing and has many fundamentalist beliefs and a view of his or her faith as the true one.

6. Authority Conflict. High scorers tend to view life as a jungle and to view other persons as unscrupulous, dishonest, hypocritical, and motivated only by personal profit. They are likely to be competitive, believe that rules are for others, have little respect for experts, and be generally mistrustful.

7. Psychoticism. Persons with high scores admit to hallucinations, strange experiences, loss of control, classic paranoid delusions of grandeur and persecution, feelings of unreality or daydreaming, and the sense that things are wrong and to feeling misunderstood by others when these views are expressed.

8. Organic Symptoms. High scorers describe symptoms of headache, nausea, dizziness, loss of mobility and coordination, poor concentration and memory, difficulty in speaking and reading, unusual sensations of hearing, smell, and touch, and loss of consciousness, which may be indicative of organic involvement.

9. Family Problems. High scorers believe that their home life is unpleasant and characterized by a lack of love. Their parents are viewed as unnecessarily critical, nervous, quarrelsome, and quick tempered.

10. Manifest Hostility. High scorers admit to sadistic impulses and tendencies to be cross, grouchy, com- petitive, argumentative, uncooperative, socially aggressive, and retaliatory in interpersonal relationships.

11. Phobias. High scorers admit to many fears of the classic phobic variety (e.g. heights, the dark, and close spaces).

12. Hypomania. High scorers are characterized by feel- ings of excitement, well-being, restlessness, tension, enthusiasm, broad interests, seeking change, cheerful- ness, great energy, and difficulty with temper control.

13. Poor Health. High scorers express concern about their health and admit to various gastrointestinal symp- toms, primarily related to an upset stomach and difficulty in elimination.

Over the 57 years of the test's history, dozens of specialized subscales have been added, including sub- scales that evaluate such items as basic trust, violence, dominance, worry and obsession, to name a few. Additionally, there are scales that measure validity factors on the answers to the main items. These validity scales take into account fabrication of test answers, incompetence, and defensiveness that may have an effect on the final test results. Each scale is scored separately but is meaningful only in the framework of the entire profile.

Page 3: A knowledge-based support system for psychological analysis

Psychological Analysis Support System 57

le Set)

FIGURE 1. Model of the PICTURES Expert Assistance System.

3. EXPERT SYSTEM DEVELOPMENT SOFTWARE

In the early development of computer programs for expert systems, it became evident that many of the programming tasks were duplicated in every program, while the coding of the problem-specific knowledge base changed from one program to another (Zahedi, 1993). Over the past 25 years, these tools have evolved from low-level system development shells to high-level knowledge engineering facilitators.

The EXSYS Professional system development soft- ware was used to develop our expert assistance system. EXSYS Professional is a generalized expert system designed for any problem that involves a selection from among a definable group of choices where the decision is based on logical rules. The rules can comprise relative probabilities of a choice being correct. Furthermore, the knowledge management facilities of EXSYS Profes- sional supports an excellent user interface, development tool function, and external program/data interface capa- bility. The external program/data interface component allows the system to communicate and retrieve data from external sources, which was a major consideration in the development of our expert assistance system.

4. ARCHITECTURE OF PICTURES

PICTURES is an acronym for P__ersonality _Inventory Calculated Through the U__se of a Rule-based E_xpert System. Figure 1, exhibits a model of the logical design of the PICTURES expert assistance system. The system (1) administers the psychological test, (2) scores and computes the scores for each of the 41 personality scales included, and (3) analyzes the final test results.

PICTURES includes approx. 500 test questions administered and scored exclusively by means of the computer. A Pascal program was designed, structured, and written that displays each test question on the computer monitor, scores the test, and passes the computed statistical results to the expert system compo- nent of PICTURES. The test taker answers the questions as they appear on the screen, and the individual's responses are scored objectively in accordance with set norms and standards. The raw scores from the standard validity and clinical scales are transformed to linear T- scores (mean = 50; SD = 10) using data provided in the MMPI Manual (Dahlstrom, Welsh & Dahlstrom, 1972; Duckworth, 1980; Graham, 1987; Hathaway & Heehl, 1972; McKinley & Hathaway, 1943). The responses of

the Minnesota normal group provide the basis for the T- score conversions. The normal population, on which the test is based, was represented by a group of about 1400 men and women randomly drawn from Minnesota, Iowa, and Wisconsin in the early 1940s (Colligan & Offord, 1988).

Scoring of the inventory is objective. Standard scoring procedures generate a test profile composed of four validity scales and 37 clinical or personality scales. PICTURES includes the 13 original scales and an additional 24 of the more interesting scales that have been added over the years. The inventory's purpose is to elicit a wide range of self-descriptions from each test subject and to provide, in quantitative form, a set of evaluations of his/her personality status and emotional adjustment (Colligan & Of-ford, 1988; Newell, Shaw & Simon, 1963). A final score is computed for each scale based on the test subject's responses on each inventory item of the MMPI. The formula used to obtain the final score is:

((((subjects score-mean) * 10)/sd) + 50),

where the subject's score is the score acquired on that specific scale, mean is the average score for that scale based on the normal group, and sd is the standard deviation for that scale (Dahlstrom, Welsh & Dahlstrom, 1972; Duckworth, 1980; Graham, 1987).

Table 1 is a complete list of the 41 scales included in the PICTURES knowledge-base assistance system. The scales range from emotional to physical problems, but predominantly evaluate one's personal responses to oneself and to others. These scales measure both positive and negative traits. Each scale is a quantifiable measure of one trait. For example, the positive trait for depression is cheerfulness, while the negative is called severe depression. As can readily be seen from this illustration, each scale is actually two traits. To classify the subject for a scale, a score will be generated from the test that will show whether the subject is ranked low or average or high, as compared to the norm response for that scale.

Figure 2 shows a representative example of an attribute chart generated for each of the scales. The attribute chart illustrates that abnormally low scores and abnormally high scores are usually considered malad- justed, while mid-range scores are considered within the normal range.

Page 4: A knowledge-based support system for psychological analysis

58 G. D. Nord and J. H. Nord

TABLE 1 Profile Scales Within PICTURES

Lie Scale Frequency Scale Correction Scale

? Scale Attitude Toward Self

Ambition Basic Trust Depression

Orderliness Originality Paranoia Phobias

Prejudice Psychasthenia

Psychopathic Deviate Quantitative-Linguistic Balance

Dominance Emotional Immaturity

Family Problems Female Masochism

Hypochondriasis Hypomania

Hysteria Impulsivity

Intellectual and Cultural Intelligence

Introversion-Extroversion Manifest Anxiety

Masculinity-Femininity

Religiousness Repression-Sensitization

Schizophrenia Self Criticism

Sensitivity Sexual Deviation Social Adjustment Social Introversion

Suicide Key Underachiever-Overachiever

Violence Worry and Obsession

Figure 3 depicts the specific characteristics of one of the 41 scales, the depression scale. The illustrative count- range preceding each choice shows examples of final test scores that will result in that trait being designated as true.

The PICTURES knowledge base is comprised of approx. 150 rules. The following five rules illustrate the range of traits accessible for selection within the depression scale.

RULE NUMBER: 20 IF:

[DSCALE] < 45 THEN:

Cheerful, enthusiastic, optimistic, active, and outgoing. Sometimes caused by a lack of inhibi- tion. (depression scale)-Probability = 1.

RULE NUMBER: 21 IF:

[DSCALE] > 44 and [DSCALE] < 60

THEN: Reflects a life perspective with an average balance of optimism and pessimism, energy and enthu- siasm. (depression scale)-Probability = 1.

RULE NUMBER: 22 IF:

[DSCALE] > 59 and [DSCALE] < 70

THEN: Likely to be mildly depressed, worrying, and pessimistic. This mood may be the result of transient situational pressures. (depression scale) -Probability = 1.

Midrange scores

Adjusted Personality

Personality Inventory

Final scores between 40 and 60

Abnormally low final scores

Maladjusted

Abnormally high final scores

FIGURE 2. Attribute Chart for each clinical scale.

Page 5: A knowledge-based support system for psychological analysis

Psychological Analysis Support System 59

Cheerful

(28-44) Cheerful 1 Enthusiastic, Optimistic, Active

Depression Level

(45-59) Balance of optimi.qm and pessimism

(60-69) Mildly depressed, worrying, pessimistic

Depressed ~70-79~ Clinically significant levels of depression

(80 and up) Sevvmly depressed, pessimistic and worrying, Socially withdrawn, Extreme apathy

FIGURE 3. Attribute Chart for the depression scale.

RULE NUMBER: 23 IF:

[DSCALE] > 69 and [DSCALE] < 80

THEN: Levels of depression that are clinically significant, tend to worry over minor issues, and evaluate for suicidal tendencies. (depression scale)-Probabil- i ty= 1.

RULE NUMBER: 24 IF:

[DSCALE] > 79 THEN:

Severe depression, pessimism, indecision, and worrying, with slowing psychomotor and extreme apathy. (depression scale)-Probability = 1.

Table 2, is an illustration of the output derived from a sample run of the PICTURES system. Only 10 of the scales are shown for illustrative purposes. The number following the scale name is the test subject's final score for that scale as computed in the Pascal program and passed to the knowledge base for processing.

5. PICTURES SESSION

As an illustration, a sample session with PICTURES is presented in this section. As indicated earlier, the PICTURES system is designed to administer the psycho- logical test, score the test based on set standards, and provides a preliminary profile for the clinician to examine and analyze.

Figure 4 illustrates, that once the test-taker calls PICTURES, a screen is displayed that provides a friendly welcome to the system and briefly explains the purpose of the system. After reading the introductory screen the test-taker advances to the next screen by

pressing any key. Figure 5, provides the test-taker with fundamental

instructions on the procedure for selecting an answer to each of the 500 questions administered by the computer. Then, the test-taker proceeds to the next screen and begins taking the test. Figure 6, depicts for illustration purposes the type of questions asked on the test. Figure 7, displays the screen that the test-taker sees upon completion of the psychological test.

TABLE 2 Sample PICTURES Output

1. The person is responding to the test in a rational and pertinent manner. (Frequency scale = 5)

2. Cheerful, enthusiastic, optimistic, active, and out- going. Sometimes caused by a lack of inhibition. (Depression Scale = 35)

3. Suggests conventional and conforming individual who may be seen as passive and nonassertive. (Psychopathic Deviate Scale = 38)

4. Suggests adequate regard and consideration for others, a flexible approach, and no undue sensitivity. (Paranoia Scale = 45)

5. Scores suggest a warm, sociable, outgoing, and rather gregarious individual. (Social Introversion Scale = 23)

6. Score reflects average worries and fears, but does not have a major effect on the individual. (Phobia Scale = 45)

7. Shows extremely naive and optimistic attitude; may be overly trusting and will tend to be sensitive and self confident. (Basic Trust Scale = 63)

8. An average degree of analytical logic with a normal amount of interest in culture and the arts. (Intellectual and Cultural Interests Scale = 48)

9. Individual may exhibit normal nervous tendencies with a minor amount of physical complaints. (Manifest Anxiety, Short Form Scale = 53)

10. Suggests a person who is very rigid and intolerant in his/her beliefs. This person generally has a cynical, distrustful attitude toward other people. (Prejudice Scale = 61)

Page 6: A knowledge-based support system for psychological analysis

60 G. D. Nord and J. H. Nord

P I C T U R E S

W e l c o m e to the Pe r sona l i t y I n v e n t o r y

C a l c u l a t e d t h r o u g h the U s e of a R u l e - B a s e d E x p e r t S y s t e m (PICTURES).

Th is s y s t e m is d e s i g n e d to ass is t a c l in ic ian in a n a l y z i n g t he resu l t s of t he p sycho log i ca l test a d m i n i s t e r e d b y the c o m p u t e r .

Press a n y key to start:

FIGURE 4. PICTURES main display.

Figure 8, displays an abbreviated sample of the results that the clinician would receive from PICTURES. After displaying the psychological profile results, the system advances to a conclusion screen, as shown in Figure 9.

6. STATUS, EVALUATION, AND FUTURE WORK

A typical cycle of activities recurs repeatedly during the development of a knowledge base for an expert system. Each iteration through the developmental cycle results in a more complete realization of the expert system than the former iteration. Each iteration may be thought of as

THE TEST

This test will be answered on a Yes=l, No=2, or Cannot Answer=3 scale for each question. For Example:

I work under a great deal of tension.

1) Yes 2) No 3) Cannot Answer Select the answer that best reflects your choice.

You must try to answer the question Yes/No if at all possible!

Press any key to start :

FIGURE 5. Test Information.

T E S T B E G I N S H E R E .............

I like mechanics magazines. 1) Yes 2) No 3) Cannot Answer

2 I wake up fresh and rested most mornings.

1) Yes 2) No 3) Cannot Answer

2 I am sure I get a raw deal from life.

1) Yes 2) No 3) Cannot Answer

2 My father was a good man.

1) Yes 2) No 3) Cannot Answer

1

FIGURE 6. Sample test questions.

;3

'7,

corresponding to a phase in the project's overall plan. Even the construction of the initial prototype will normally consist of an iteration through the development cycle. Thus the development of a knowledge base, such as for the PICTURES system, tends to be incremental and evolutionary in nature.

The PICTURES system has attained a high level of development after numerous iterations in the evolution of the prototype. The screening, scheduling, and imple-

STOP[I[!!

You have finished the test. The results have been calculated.

Your counselor will help you evaluate the results.

FIGURE 7. Test completed screen.

Page 7: A knowledge-based support system for psychological analysis

Psychological Analysis Support System 61

RESULTS

Profile

No significance is given. (Lie Scale) The Person is responding to the test in a rational and pertinent manner.

(Frequency Scale) Cheerful, enthusiastic, optimistic, active, and outgoing. Sometimes

caused by a lack of inhibition. (Depression Scale) Suggests conventional and conforming individual who may be seen as

passive and non-assertive. (Psychopathic Deviate Scale) Suggests adequate regard and consideration for others, a flexible

approach and no undue sensitivity. (Paranoia Scale) Tends to be without feelings, slow to feel or notice. (Sensitivity Scale) Intolerant of religious beliefs and practices. (Religiousness Scale) Average emotional maturity level. (Emotional immaturity Scale) A good aptitude for original ideas. (Originality Scale) Individual may exhibit normal nervous tendencies with a minor amount

of physical complaints. Manifest Anxiety, Short Form Scale)

~i ~.

FIGURE 8. PICTURES profile results.

mentation of knowledge acquired from clinicians and the wealth of published literature should be taken as standards against which the system generated results are evaluated. An on-going project currently exists that will further test the utility of the system through logical and empirical analysis. These techniques include the use of historical data to test the system, assessment of the validity of the system by clinicians, and extensive use of the system within a simulated environment. Presently,

CONCLUSION

The PICTURES system has now completed.

Thank you for using the psychological system.

To leave the system type Q for Quit. To run the system again, type R for Run.

II

FIGURE 9. PICTURES conclusion screen.

the system has gone through numerous iterative cycles in updating and refining the knowledge base.

7. SUMMARY

In this paper, we have described an expert assistance system that integrates a computerized psychological test with a knowledge base that aids in analyzing the test results. The psychological test is administered and graded solely via computer. The test results are then scored and passed to a knowledge base that is designed to evaluate the final test score on each of 41 clinical scales. The expert assistance system provides output that identifies specific psychological traits that the clinician can examine and undertake further advisement. The system was designed to mimic the analytical and logical processes that the clinician diligently implements during the administration and evaluation of a personality inventory.

It is important to emphasize that the PICTURES system was designed to: (1) aid the clinician during the initial screening of the test subject's personality profile, (2) save the clinician a significant amount of time in administering and evaluating the test results, and (3) allow the system to be used for its potential educational value. Based upon preliminary results, the system has functioned admirably in all three areas. However, as in all knowledge based systems, on-going and future research needs to be continually conducted to improve, update, and maintain PICTURES.

REFERENCES

Antoni, M., Levine, J., "fischer, E, Green, C. & Millon, T. (1986). Refining personality assessments by combining MCMI high-point profiles and MMPI codes, part IV: MMPI 89/98. Journal of Personality Assessment, 50(l), 65-72.

Colligan, R. C. & Offord, K. E (1988). Contemporary norms for the Wiggins Content Scales: a 45-year update. Journal of Clinical Psychology, 44(1), 23-32.

Dahlstrom, W. G., Welsh, G. S. & Dahlstrom, L. E. (1972). An MMPI Handbook, University of Minnesota Press, Minneapolis, MN.

Duckworth, J. C. (1980). MMPI Interpretation Manual for Counselors and Clinicians, Second Edition, Accelerated Development Inc., Muncie, IN.

Graham, J. R. (1987). The MMPI: A Practical Guide, Second Edition, Oxford University Press, New York, NY.

Hathaway, S. R. & Meehl, E E. (1972). An Atlas for the Clinical Use of the MMPI, University of Minnesota Press, Minneapolis, MN.

Hoiden, C. (1986). Researchers Grapple with Problems of Updating Classic Psychological Test. Science, 233, 1249-1251.

Holsapple, C. W. & Whinston, A. B. (1987). In Richard D. Irwin (Ed.), Business Expert Systems. IL: Homewood.

Hsu, L. M. (1988). Fixed versus flexible MMPI diagnostic rules. Journal of Consulting and Clinical Psychology, 56(I ), 458-462.

Jacob, V. S., Moore, J. C. & Whinston, A. B. (1988). Artificial Intelligence and the Management Science Practitioner: rational choice and artificial intelligence. Interfaces, 18(4), 24-35.

Kroeck, K., Kirs, E & Diedler, A. (1989). Cognitive biasing effects in information systems: implications for linking Real World Informa- tion with human judgement. Proceedings of the Twenty-second Annual Hawaii International Conference on System Sciences, IEEE Computer Society Press.

Page 8: A knowledge-based support system for psychological analysis

62 G. D. Nord and J. H. Nord

Luger, G. E & Stubblefield, W. A. (1989). Artificial Intelligence and the Design of Expert Systems. Redwood City, CA: Benjamin Cum- mings.

Martin, J. & Oxman, S. (1988). Building Expert Systems: A Tutorial. Englewood Cliffs, NJ: Prentice Hall.

McKinley, J. C. & Hathaway, S. R. (1943). The identification and measurement of the psychoneuroses in medical practice: the Minnesota Multiphasic Personality Inventory. Journal of the American Medical Association, 122, 161-167.

Moore, J. C. & Whinston, A. B. (1987). A model of decision making with sequential information acquisition--Part II. Decision Support Systems, 3(1), 47-72.

Newell, A., Shaw, J. C. & Simon, H. A. (1963). Empirical explorations with the logic theory machine: a case history in heuristics. In Computers and Thought. New York, NY: Feigenbaum and

Feldman. Olson, D. L. & Courtney, J. E Jr (1992). Decision Support Models and

Expert Systems. New York, NY: Macmillan Publishing Company. Parker, C. H., Hanson, R. & Hunsley, J. (1988). MMPI, Rorschach, and

WAIS: a meta-analytic comparison of reliability, stability, and validity. Psychological Bulletin, 103(3), 367-373.

Pigford, D. V. & Banr, G. (1990). Expert Systems for Business, Concepts and Applications. Boston, MA: Boyd & Fraser Publishing Company.

Quinlan, J. R. (1987). Applications of Expert Systems. New York, NY: Addison-Wesley Publishing Company.

Ward, C. L. (1986). MMPI item subtlety research: current issues and directions. Journal of Personality Assessment, 50(1).

Zahedi, E (1993). Intelligent Systems for Business: Expert Systems with Neural Networks. Belmont, CA: Wadsworth Publishing Company.