Radiologic automated diagnosis (RAD)

Computer Methods and Programs in Biomedicine 25 (1987) 157-168 157 Elsevier

CPB 00856

Radiologic automated diagnosis (RAD)

Gordon Banks, John K. Vries and Sean McLinden

Decision Systems Laboratory, University of Pittsburgh, Pittsburgh, PA, U.S.A.

RAD is a program currently being developed to interpret neuroimages. Given the clinical information usually available on the imaging request, R AD will analyze the scan directly from the data generated by the scanning machine to produce a differential diagnostic list explaining any lesions it discovers. R A D uses a computerized three-dimensional stereotaxic atlas of the nervous system as a model of normal structures in the analysis of scans.

Diagnosis; Neuroimaging; Radiology; Artificial intelligence

1. Background

During the past ten years, new techniques for imaging the brain have had a revolutionary effect on the practice of neurology and neurosurgery. Formerly, many structural abnormalities in the brain were inaccessible to radiologic investigation. Arteriography and pneumoencephalography were valuable procedures, but they are invasive. More- over, lesions that were avascular or too small to distort the ventricular system could only be studied through the functional changes which they caused in the clinical neurologic examination. Computed tomography (CT) has allowed direct imaging of the brain with a high degree of resolution, and is currently one of the most useful diagnostic tools of modern medicine. Yearly improvements in resolution and speed have continued to enhance its usefulness. Recently, magnetic resonance imaging (MRI) has reached the point in its development where it has begun to complement CT scanning.

The data gathered by these imaging techniques

Correspondence: G. Banks, Decision Systems Laboratory, Uni- versity of Pittsburg, 1360 Scaife Hall, Pittsburg, PA 15261, U.S.A.

consists of arrays of numeric values which are associated with regions in three-dimensional space inside the object being imaged. In the case of CT, these values, called Hounsfield numbers, are re- lated to the x-ray attenuating properties of the matter occupying the region. In order to facilitate interpretation by radiologists, this data is displayed using computer graphics techniques in the form of two-dimensional slices through the object, with points in space represented as 'pixels' of intensity proportional to the Hounsfield numbers. This produces an image similar in appearance to conventional radiographs. To facilitate the generation of these images, a dedicated computer is part of each imaging system.

2. Neuroimage interpretation

In order to successfully interpret neuroimages, the radiologist must have knowledge of the location, morphology, and x-ray attenuating properties (radiodensity and radiolucency) of many anatomic structures. He must know the relationships of anatomic regions to vascular territories, and how lesions in one anatomic location may produce morphologic changes in another region (for exam-

0169-2607/87/$03.50 © 1987 Elsevier Science Publishers B.V. (Biomedical Division)

158

pie, dilation of the lateral ventricles resulting from obstruction of the aqueduct of Sylvius by a posterior fossa tumor). While the images alone contain considerable differential diagnostic information, the radiologist is greatly aided by clinical information about the patient [1]. The patient's symptoms and signs can suggest a set of anatomic sites for closer scrutiny, as well as allow the radiologist to prioritize his differential diagnosis among those etiologic entities that present a similar radiologic picture to the one he has under current consideration.

Ideally, these anatomically based neuroimages are interpreted by radiologic subspecialists (neuro- radiologists) in conjunction with clinical information supplied by neurologists or neurosurgeons. These specialists characteristically have the knowledge of correlative neuroanatomy necessary for optimum interpretation of these scans. Because of the usefulness of CT, scanners are now present in many hospitals which do not have neuroradiolo- gists. In these hospitals, general radiologists are responsible for scan interpretation. Often, the community hospitals have the latest in CT hard- ware, but lack even neurological or neurosurgical support. Many times, the radiologist responsible for reading CT scans was trained before the ad- vent of CT and has to rely on continuing education and reading to achieve his competency with these techniques, and may have had little instruc- tion in neuroanatomy beyond medical school.

We believe that an expert system could be of value to physicians performing neuroradiologic diagnosis, whatever their training or competence. The general radiologist would benefit by reviewing a program-generated differential diagnosis for those diseases which he might not have considered. In addition, the explanatory features of an expert system could have value from the continuing education standpoint. The neurologist, neuro- surgeon, or neuroradiologist might find the program useful for screening scans for special attention, or for prompting consideration of the more arcane possibilities in difficult cases. In addition, such a system could facilitate analysis of the images in ways that are difficult for humans. For example, a program that could successfully extract regions automatically could make quantitative mea-

surements of the volume of lesions or ventricular size with more facility than can be done from the film. Dynamic processes could be investigated by studying serial changes in lesions over time. Cur- rently, these studies require tedious human inter- action, which precludes their widespread use. Also, standardized objective methods might be developed to perform cross-modality correlation (e.g. CT vs. MRI vs. PET). At the present state-of-the- art, there is no reason to believe that a computer capable of handling such an expert system could not be an integral part of the next generation of CT scanners without adding significantly to their cost.

3. Image analysis

The application of artificial intelligence (AI) techniques to image analysis is currently a topic of great interest in the fields of computer vision and robotics. The general field of AI applications to computer vision and image understanding is too broad to be covered here. The field is well surveyed in the textbook of Ballard and Brown [2], as well as in chapters on image understanding in Winston [3] and Cohen and Feigenbaum [4].

One of the most difficult steps in image understanding is segmenting the objects of interest from the background. A wide variety of techniques have been developed for this purpose, but all have had some drawbacks. Recently, AI techniques have been considered for this type of low-level processing where a priori knowledge of the expected properties of objects can be used to guide edge, surface, and boundary detectors. These techniques may also have a role to play in region merging and texture analysis. In the field of medical imaging, Ballard and coworkers [5,6] and Lesgold [7] have applied AI techniques to the extraction of features from digitized chest films.

In addition, AI techniques open the possibility of evaluating images with respect to analogic models, and for providing procedural knowledge for top-level control of analytical processes. AI methods can also be useful in choosing forms for representing higher-level knowledge about normal and pathologic features that may be present in the

image. Kumar [8] demonstrates how a rule-based system can be used to analyze objects which have been extracted from CT scans using conventional image analysis techniques such as edge detectors.

4. RAD

The goal of the RAD project is to develop a program using AI techniques which will take data from medical imaging devices along with a suc- cinct list of the clinical findings and demographic information, such as would be provided on a request for an imaging study, and produce a ranked differential diagnostic list. The list should be as short as possible, and it should contain the diagnosis that eventually proves to be the correct one.

The basic approach used in RAD is to analyze the scan data by comparing the pixel densities of regions of the scan with an analogic atlas-based three-dimensional model which is normalized to the patient. A similar approach using two-dimensional models is being pursued simultaneously and independently by Newell [9]. Focussing is done by using clinical manifestations and also by identify- ing asymmetries and regions of the image which are statistically abnormal in terms of pixel histogram. Initial hypotheses are identified based on the location and histogram characteristics of the abnormali ty and the clinical information. Hy- potheses are tested against the image data, and those that cannot be ruled out are reported on the differential diagnostic list. The program is reported as work-in-progress. In the following sections, we will report in detail on the parts of the system that are operational, some of which have interesting applications, and outline the planned approach for future work.

5. Image representation

Raw data is entered into the system by means of digital tapes produced by imaging devices. These tapes contain processed information in archive format with image data represented as arrays of binary numbers. The first step in analysis is to convert these arrays into Lisp vectors to make the

159

image information accessible to RAD. In the case of the GE 9800 CT scanner, programs to accom- plish this step have been completed. For this scanner, the images are stored as 512 x 512, 384 x 384, or 256 x 256 arrays of 12-bit numbers representing radiodensities (Hounsfield numbers) cor- responding to the coordinates { x, y } in each CT slice. The various CT slices may be thought of as taken at different points along a z axis.

Udupa [10] has surveyed the large number of image representation schemes in current use, including chain codes, boundary and surface representations, B-spline functions, generalized cylin-

t

l object

i

Fig. 1. Octree representation.

@

level 1

level 2

level 3

160

ders, and quadtree and octree encoding. For our application, we have chosen the latter technique, which we feel offers distinct advantages. Two-dimensional image representation using quadtree encoding was first proposed by Klinger [11] and elaborated by Samet [12]. In this scheme, binary images extracted from raw data are represented by a tree of order (or degree) 4 (quadtree). The root level can be conceptualized as a block large enough to enclose all of the pixels making up the image or region. A single number can be assigned to the root node, denoting its 'state'. If it is 0, the entire image is background (white). If it is 2, the entire image is object (black). If the entire image cannot be classified as all object or all background, the number is 1 (gray) and the tree is then subdivided into quadrants. For each of these subquadrants, the same process is reapplied, characterizing their states as 0, 1, or 2. This process may thus be applied recursively. When the process terminates, we are left with a quadtree structure whose termi- nal nodes are all 0 or 2. One computational ad- vantage immediately becomes apparent when it is considered that large section is of a typical image will be all background or all image and processing only need continue for those subregions which contain borders. The quadtree concept can easily be extended to three dimensions (or indeed, gener- ally into n dimensions) [13]. The three-dimensional representation, octree (tree of order 8) encoding, was elaborated by Meagher [14]. Here the three-dimensional regions are divided recursively into octants until the desired resolution is obtained. Fig. 1 shows how a simple object (a rectan- gular parallelopiped) in a region can be com- pletely represented by a three-level octree.

Some of the advantages of quadtree and octree encoding are: (1) The encoding process can be carried to any level resolution, including the single-pixel level. If high resolution is not necessary, examination of the top few levels of the tree saves orders of magnitude of computer processing time. (2) Image regions are naturally organized into hierarchical relationships which facilitates context analysis. (3) An extensive library of algorithms for conver- sion between quadtrees and other forms of image

representation, and for calculating the geometric properties of images represented as quadtrees has been developed by Samet. (4) Three-dimensional octree images can easily be constructed from two-dimensional slices in quadtree format. (5) The operations of translation, scaling, rota- tion, and hidden-surface elimination, which are computationally intensive for most image representation schemes, can be accomplished without multiplication or division. Rotations at angles of 90 ° simply require changing the order of traversal of the tree. Arbitrary rotations are more com- plicated but still more economical than with the conventional trigonometric methods [14]. (6) Composite images can be quickly constructed by logical union or intersection of source images. In this fashion, octree-encoded images of anatomic structures from the atlas can be combined with images from CT scans.

6. Regional segmentation

The CT cuts correspond to tomographic slices through the head at a given level and a preset angle with respect to the base. The data in each CT cut is a two-dimensional array of Hounsfield numbers in the x and y axes. The depth of the stack of slices representing all the tomographic cuts corresponds to the z coordinate. The Houns- field number therefore represents the radiodensity of a region of space located at a given set of { x, y, z } coordinates. The approximate Houns- field numbers for various substances found in the head are shown in Table 1. It should be noted that CT scanners are often calibrated with respect to cerebrospinal fluid (csf) density, and that these ranges may vary considerably in individual cases. Nevertheless, they serve as useful initial values prior to histogram optimization.

For each substance shown in Table 1, the number of pixels falling into each range can be computed for the whole scan or for any subdivision thereof. Image segmentation is begun by classify- ing each pixel in a given CT cut according to its Hounsfield number. Performing simple statistical analysis on the tabulated results of the pixel counts

TABLE 1

Approximate Hounsfield ranges

Tissue Range

Fat - 100 to - 10 Cerebrospinal fluid - 10 to 15 White matter 28 to 32 Gray matter ( - contrast) 33 to 38 Gray matter ( + contrast) 39 to 45 Tumor 50 to 150 Blood ( - contrast) 50 to 80 Blood ( + contrast) 50 to 100 Bone 100 to 4000

and their ( x , y } coord ina tes for each slice can yie ld useful focussing in fo rma t ion for the image analys is process . Tab le 2 is a t abu la t ion of the pixels for a C T slice con ta in ing a con t ras t - enhancing tumor (g l iob las toma) which is in the left p a r i e t a l - o c c i p i t a l region. The or igin of coor- d ina tes is in the u p p e r left corner . F o r each Houns f i e ld par t i t ion , the x and y coord ina tes of the ' cen te r of gravi ty ' for that pa r t i t i on is t abu- la ted. Inspec t ion of the table shows tha t the bone pa r t i t i on is symmetr ic , while 26% of the pixels lie in the hype rdense (con t ras t -enhanc ing) range, in- d ica t ing gross abnorma l i ty . These hyperdense pixels are cen te red in the left lower q u a d r a n t ( the cen te r of the slice is a t {252, 248}). G r a y and whi te ma t t e r densi t ies a re shif ted to the right, while there are more hypodense pixels in the left lower q u a d r a n t than there ought to be. This al- mos t i n s t an taneous analysis can focus a t t en t ion on the lower left q u a d r a n t as well as given some

TABLE 2

Statistical analysis of Hounsfield ranges

Density Area % cgx cgy

Air 32 0 243 436 Hypodense 16 0 242 346

White 527 12 268 231 Gray 2041 49 268 226

Hyperdense 1090 26 217 267 Heme 1176 4 100 250 Bone 263 6 250 236 Total 4145 100 252 248

161

idea of the character is t ics of the lesion (large, enhancing , with associa ted hypodense areas).

7. Computerized stereotaxic atlas

U n d e r s t a n d i n g of an image of the nervous system requires the in te rpre te r (whether h u m a n or machine) to have a knowledge of neu roana tomy . Then regions of the scan co r r e spond ing to neuro- a n a t o m i c s t ructures which are visual ized by the technique can be ident i f ied. We use s t ructures represen ted in a compute r i zed th ree-d imens iona l n e u r o a n a t o m i c at las as mode l s for compar i son with ex t rac ted objects , as well as tools for faci l i ta t- ing the ext rac t ing process. These models are represented in te rna l ly in the same oc t ree -encoded for- ma t as the objects ex t rac ted f rom CT. The oct ree m e t h o d faci l i ta tes compar i sons since it is s imple to make logical A N D s and ORs ( in tersect ions and unions) be tween octrees represent ing at las objects and ex t rac ted objects .

The sys tem can normal ize objects der ived f rom at lases to ind iv idua l C T scans and correct for skew of the pa t i en t in the scanner. Three -d imen- s ional images combin ing a rb i t r a ry at las objec ts and objects ex t rac ted f rom scans can be p roduced . The th ree-d imens iona l represen ta t ions of a t las objec ts can be sect ioned in any a rb i t r a ry p lane , and these sect ions can be pro jec ted upon a two-d imen- s ional CT slice m a d e in any plane. This component of the R A D system is useful in its own right for p l ann ing s tereotaxic neurosurgica l p rocedures and teaching neu roana tomy .

Fig. 2 shows the modules in R A D for gener- a t ing and man ipu la t i ng th ree -d imens iona l images ex t rac ted f rom C T scans and atlases. Image files f rom the G E 9800 scanner are ex t rac ted f rom tape and conver ted to 512 × 512 a r rays of Houns f i e ld numbers . These a r rays are averaged and inter- po l a t ed to p roduce a 256 × 256 × 256 cube of 16-bi t numbers . The O R T H O and C T V I E W routines are used to ob ta in a rb i t r a ry two-d imens iona l slices in the axial, coronal , or sagit tal p lanes . Oct ree da t a s t ructures are genera ted b y recurs ively app ly ing the rout ines in O C T A G E N unti l the daugh te r oc tan ts are ent i re ly full or empty . If this has not occurred by the t ime the octree is eight

162

I,ram. bo,for I

I octroo d0,abose

Fig. 2. Image manipulation and display.

law of geometrical optics. Color map indexing is controlled by the STOA and ATOS routines. Composite images are generated using the LOGI- CAL_OR and L O G I C A L A N D routines. These programs perform a parallel traversal of two objects in octree format to produce a third octree object which is the logical OR or logical A N D of the original two object. Fig. 3 shows a view of the base of the skull from above. This picture was made from the bone-window octree of a CT scan using the technique described above.

Fig. 4 shows the tools for extracting and manipulating objects for the computerized atlas. Sources may be conventional printed atlases, CT scans displayed on the video monitor, or interac- tively edited existing data structures. Plates from printed atlases are placed on the graphics tablet (HIPAD) and structures are outlined using a mouse-like device. Alternatively, CT scans can serve as a source for objects. Their borders can be traced from CT sections on the Sun computer

levels deep, the pixels are forced to be full or empty.

Octrees are created for each appropriate Hounsfield range (bone, csf, white matter, etc.). Although average values for these ranges are known, individualized ranges can be obtained prior to octree generation by manually thresholding using CTVIEW. These octree data structures are the basis for the remaining operations. The X F R M routine is used to translate, scale, and rotate the octree objects as well as to perform hidden surface elimination. The output of this routine is a 256 × 256 array of 16-bit numbers that represent the distance from the surface of the viewing plane to the object at each point (z-buffer). The S H A D E R routine generates pseudo three-dimensional shad- ing from the z-buffer. The z-buffer is interpreted as a contour map, and imaginary light vectors are reflected from its surface. The intensity of reflected light is calculated according to Lambert ' s Fig. 3. Base of skull viewed from above.

.r? HIPAD

transformed I octree

f

Fig. 4. Tools for extracting and manipulat ing objects from an atlas.

163

graphic d i sp lay screen using the mouse. The F M modu le records the z coord ina t e of the two-di- mens iona l outl ines. The stacks of out l ines f rom serial a t las or C T sect ions are processed by the P O L Y H E D R O N modu le to form th ree -d imen- s ional objects. I t pe r fo rms l inear in t e rpo la t ion to c rea te a un i fo rm series of out ines at 1-mm intervals f rom the b o t t o m of an objec t to the top. These out l ines are then processed to yield a set of { x, y, z } coord ina tes for all po in t s in a three-di-

mens iona l region be longing to the object . The p r o g r a m A T L A S takes these po in t s as input and p roduces an octree represen ta t ion of the object in the same format as the octrees used for the objects ex t rac ted f rom the CT scans. This means that all of the oct ree t r ans fo rma t ion ope ra t ions used on C T scans can also be used on these a t las -der ived objects . This includes scaling, t rans la t ion , and rota t ions as well as the logical O R and logical A N D opera t ions . Compos i t e objects may be bui l t f rom subuni t objects and at las objects may be com- b ined with regions ex t rac ted f rom CT scans. Fig. 5 shows a th ree -d imens iona l view of the tha lami as seen f rom the top. These tha lami were der ived f rom Scha l t enbrand ' s s tereotaxic at las [15] by trac- ing with the H I P A D .

Fig. 5. Thalami extracted from an atlas.

8. Normalization

The norma l i za t ion p r o b l e m may be b roken down in to three par ts . The first involves t r ans fo rma t ion of all the s t ructures in the compute r i zed at las in to a s t anda rd projec t ion , regardless of the source of their der ivat ion. The second involves normal iz ing c o m p o n e n t s in a n e u r o a n a t o m i c da t abase to struc-

164

tures in patient CT scans. This requires the estab- lishment of stable reference points that are relatively invariant from patient to patient. The third part involves correction in three dimensions for the obliquity of the alignment of the patient with the scanner. We have found that it is easier to normalize the neuroanatomic database to the scan than the converse. Since there are fewer data points in the database than in the scan, we apply a homogeneous coordinate transformation to the three-dimensional atlas in order to fit it to the coordinates of the CT under consideration.

To normalize for the different orientations of the stereotaxic atlas plates or CT projections used to generate the neuroanatomic database, the axial view as seen from the top was defined as the standard view. Defining the orientation of the atlas plates for the ATLAS program (i.e. axial, coronal, sagittal) automatically generates a transform matrix to allow a homogeneous coordinate transformation into the standard view.

The normalization of structures from the neuroanatomic database to patient CT scans is based on the line from the anterior commisure to the posterior commisure ( A C - P C line). This line has been used extensively by stereotaxic neurosurgeons and has been found to be relatively invariant [16]. It has been used as a reference line for the construction of coordinate systems for several stereotaxic atlases. To determine the A C - P C line, the midline sagittal CT cut is generated from the interpolated three-dimensional CT scan using the O R T H O program. This view is displayed on the video monitor, and the position of the anterior and posterior commisures are marked using a mouse. The midpoint of the A C - P C line determines the origin of the x and y coordinates. The z coordinate is determined by the slice selected to represent the midline in the sagittal plane. The transformation matrix necessary to normalize the neuroanatomic database with respect to the origin of the (x , y, z ) coordinate system is automatically generated when the A C - P C line is marked.

Correction for obliquity involves the de- termination of an orientation vector in the sagittal, coronal, and axial planes. In the case of the sagittal view, the orientation of the A C - P C line is used.

In the coronal plane, the line connecting the floor and the roof of the third ventricle is traced with the mouse on a coronal view produced by OR- THO. The axial vector is obtained by mousing the line between the aqueduct of Sylvius and the most anterior point of third ventricle on an axial section containing both. While these operations are currently done by hand, we hope to be able to automate the process. This will require the program to be able to locate the third ventricle as well as to determine the landmarks for the commisures.

Fig. 6 shows the results of combining the octree of the normalized central ganglionic masses (caudate, putamen, globus pallidus, and thalamus) extracted from Schaltenbrand's atlas with the skull octree from a patient CT scan using the LOGI- CAL_OR routine. The front of the skull has been cut away to reveal those gray-matter structures within.

The PROJECT module allows two-dimensional back projections of the objects in the atlas to be made upon any arbitrary CT slice generated with the O R T H O routine. Fig. 7 shows a back-projected section of the central ganglionic masses extracted from the atlas on a coronal section of a patient CT. Fig. 7 is a black-and-white reproduc- tion of the color display of the Sun, with each component of the ganglionic masses assigned a

Fig. 6. Atlas derived central ganglionic masses ORed with patient skull.

Fig. 7. Back projection of coronal section of ganglionic masses.

different number in the color map. This technique is useful in stereotaxic surgery. Fig. 8 shows a projection of the VL nucleus of the thalamus and the whole thalamus from the atlas on an axial

Fig. 8. Thalamus and VL nucleus projected in axial section.

165

section of a patient CT. The VL nucleus is a frequent target for the placing of stereotaxic lesions. The ability to superimpose the target from the atlas upon the patient's CT scan can help the surgeon plan his approach. Stereotaxic surgery is usually done in an operating room which is fitted with a CT scanner, and so the computerized stereotaxic atlas has potential for becoming a useful tool in this realm.

9. Clinical correlation

Clinical information is requested as a part of every requisition for neuroimaging studies. This clinical information is useful to the neuroradiologist in focussing his attention on anatomic regions where involvement would explain the clinical manifestations. The demographic information (sex, age, race, etc.) is useful because the different etiologic entities causing the radiographic lesions have varying predictions for certain patient 'ecologies'. Taken together, the ecology and the neurologic manifestations serve to focus attention on a few probable explanations and to narrow the differential diagnosis. For example, ataxia and vomiting in a 6- year-old will bring to mind processes involving the posterior fossa, likely meduloblastoma or astrocy- toma of the cerebellum. In an adult, cerebellar hemorrhage or metastatic carcinoma would be more likely with the same manifestations. Sub- dural hematomas are more likely in the very young and the very old, as well as in alcoholics. The scan of a patient with a left hemiparesis needs special attention to the right motor cortex, internal capsulate and corticospinal tracts.

We have constructed a database of neuroanatomic objects linked to the symptoms and signs produced when the objects are involved with pathologic processes. The neuroanatomic objects are also linked into the atlas representations, so the spatial geometric relations are known. The vascular territories are represented so that constella- tions of symptoms and signs can be localized by connectivity (tracts), spatial, or vascular rela- tionship. An example of a manifestation data structure is the following:

166

(setq~ALEXIA WITHOUT AGRAPHIA[ (make-manf

:comment 'NIL :definition " 'Abili ty to write, but not read, even what the

patient has written." :source " 'Kertesz, Chap. 13, 14." :anatomy '(]any 1 of I

(I L A N G U L A R GYRUS I (lull of I

( ]*SPLENIUM OF CORPUS CAL- LOSUM I

(]any 1 of I C * L OCCIPITAL CORTEX I I*L O C C I P I T A L W H I T E MATTER])))))

: diagnostic- anatomy

: aka

((ANATOMIC (lany 1 of I

(]L OCCIPITAL LOBE]))) (VASCULAR

( ]any 1 of I ([L POSTERIOR CEREBRAL[))))

'(]PURE ALEXIA I [PREANGULAR ALEXIA[)))

Manifestations implicate involvement of neuroanatomic objects. These objects (e.g. thalamus) are represented by Lisp structures which contain their spatial coordinates and extent, pointers to the data structures necessary for imaging their models, their subunits (for thalamus, these include the different thalamic nuclei, such as VL), their super- units, and their vascular supply. Through the use of these data structures, a set of manifestations can be explained by hypotheses involving anatomic structures, regions, or vascular territories. Such information will serve to focus the attention of the image analysis on certain regions as well as help order the differential diagnostic list.

10. Identification of imaged regions

While this part of the program is still developmen- tal, we can outline the approach. The basic prob- lem is to first identify known neuroanatomic structures which image well based on comparison with their models. The regions which are candi- dates for identification with a specific anatomic structure can be selected by expected volume, tissue density, and the coordinates of the expected

location. We have found experimentally in two-dimensional CT cuts, using a crude triangle shape of proper orientation as a model, the pu tamen-pa l - lidal complex is easily identified, even when only the central region of the scan is specified as the location. Once the more easily identified structures (ventricles, ganglionic masses) are located, they can be used to assist in the identification of more obscure structures. Several approaches are useful in the analysis of these regions, given a model. It is not to be expected that simply normal- izing the model to the patient CT will always ensure congruency between model and corre- sponding patient structures in the image. The model may have to be deformed in certain ways in order to match the patient [17,18]. Certain structures, for example the ventricles, are quite varia- ble. However, the variability is usually manifest in a few simple stereotyped ways, which can be cap- tured in a knowledge base, or in a set of models. Pizer [19,20] and coworkers are attempting to develop a method based on symmetric axis trans- forms for defining neuroanatomic objects by hierarchical shape descriptors. While their method is mathematically intensive, there are similarities (conceptually, at least) between the construction of their Gaussian-blurred hierarchic shapes and the octree representation. Another tool that may be quite useful in comparing regions in the scan with the model is the Levenshtein distance [21]. This technique is a method of comparing strings for closeness of fit. String-like structures can be generated from the images and the models for any arbitrary point by sending out radial vectors from that point and computing a histogram of the radially encountered changes of Hounsfield number. For example if we restricted ourselves for simplicity to densities of csf (C), white matter (W), gray matter (G), and bone (B), and plotted the histogram at 2-mm intervals along some imaginary radial vector from center of pallidus through caps- ule, caudate, ventrical, and frontal lobe to the skull we might have something like:

GG WGGC C C C W W W W W W G G G G B B B B.

Sets of vectors about a point might then be com- pared to the expected vectors from the database to

determine how close the unknown was to the model.

11. Neuroradiologic knowledge base

Abnormalities in a scan consist of regions which represent known structures which are distorted, displaced or missing, and those which represent objects for which there is no normal analogue, such as hematomas, tumors, aneurisms, and bul- lets. This second class of lesion cannot be handled exclusively by a model-based approach and requires a knowledge base of neuroradiology. In order to decide if a given diagnosis is consistent or inconsistent, the program must be able to access knowledge of radiologic features which are ex- plainable by that diagnosis. The neuroradiologic knowledge base will contain a profile of each etiologic lesion which is detectable using neuroradiologic imaging techniques. Each profile will have knowledge of the radiographic features which may be observed with each lesion, demographics, favored anatomic sites, and specific radiologic features such as presence of calcifications, character- istic densities, edema, multiplicity, cavitation, etc. There will also be procedural knowledge for de- termining the presence or absence of the char- acteristic features in the form of functions that know how to determine the presence of calcifications within the lesion, surrounding edema, etc. Both the classic appearance of a particular lesion and the freq~ency of known variations in the appearance of each lesion will be entered into the database. It is estimated that between 500 and 1000 lesions will be needed for this database.

References

[1] P. Doubilet and P.G. Herman, Interpretation of radiographs: effect of clinical history, AJR 137 (1981) 1055-1058.

[2] D.H. Ballard and C.M. Brown, Computer Vision (Pren- tice-Hall, Englewood Cliffs N J, 1982).

[3] P.H. Winston, Artificial Intelligence, 2nd edn. (Addison- Wesley, Reading MA, 1984).

[4} P.R. Cohen and E.A. Feigenbaum, Handbook of Artificial Intelligence (W. Kaufman, Los Altos CA, 1982).

[5] D.H. Ballard, Hierarchic Recognition of Tumors in Chest Radiographs (Birkhauser-Verlag, Basel, 1976).

167

[6] D.H. Ballard, U. Shani and R.B. Schudy, Anatomical models for medical images, in: Proc. 3rd COMPSAC, IEEE Computer Society International Computer Software and Applications Conference (1979) 565-570.

[7] A.M. Lesgold, Acquiring expertise, in: Tutorials in Learn- ing and Memory: Essays in Honor of Gordon Bower, J.R, Anderson and S.M. Kosslyn, eds. (W.H. Freeman, San Francisco CA, 1984).

[8] R. Kumar, and S. Srihari, An expert system for the interpretation of cranial CT scan images, in: Proceedings of the Expert Systems in Government Symposium, K. Karan, ed., McClean, VA, October 1985.

[9] J.A. Newell and E. Sokolowska, Model based recognition of CT scan images, in: MEDINFO 86 Proceedings, R. Salamon, B. Blum and M. Jorgensen, eds,, pp. 619-623 (Elsevier, Amsterdam, 1986).

[10] J.K. Udupa, Display of 3D information in discrete 3D scenes produced by computerized tomography, Proc. IEEE 71 (1983) 420-431.

[11] A. Klinger and C.R. Dyer, Experiments in picture representation using regular decomposition, Comput. Graph. Image Proc. 5 (1976) 68-105.

[12] H. Samet, Region representation: quadtrees from boundary codes, Commun. ACM 23 (1980) 163-169.

[13] J. Udupa, S.N. Srihari and G.T. Herman, Boundary detec- tion in multidimensions. IEEE Trans. PAMI 4 (1982) 41-50.

[14] D. Meagher, Geometric modelling using octree encoding, Comput. Graphics Image Proc. 19 (1982) 129 147.

[15] G. Schaltenbrand and W. Wahren, Atlas for Stereotaxy of the Human Brain (Thieme, Stuttgart, 1977).

[16] J. Talairach. VI Congreso Latin Americano de Neuro- chirurgia, Montevideo (1955) 865 925.

[17] F.L. Bookstein. Size and shape spaces for landmark data in two dimensions. Stat. Sci. 1 (1986) 181-242.

[18] R. Bajcsy and C. Broit, Matching of deformed images, Proc. Int. Conf. Pattern Recogn., Munchen, October 1982, pp. 351-353.

[19] L.R. Nackman and S.M. Pizer, Three-dimensional shape description using the symmetric transform. I. Theory, IEEE Trans. PAMI 7 (1985) 187 202.

[20] S.M. Pizer, W.R. Oliver, J.M. Gauch and S.H. Bloomberg, Hierarchical figure-based shape description for medical imaging, Technical report 86-026 (University of North Carolina, Department of Computer Science, Chapel Hill NC, 1986).

[21] V.I. Levenshtein, Binary codes of capable of correcting deletions, insertions, and reversals, Soviet Physic-Doklady 10 (1966) 707-710.

Copyright notice

This article is based on "Radiologic automated diagnosis (RAD)", by G. Banks, J.K. Vries and S. McLinden, Decision Systems Laboratory, appearing in Tenth Annual Symposium on Computer Applications in Medical Care, Washington, DC, 25-26 October 1986, pp. 228-239, © 1986 IEEE.

Documents

Radiologic automated diagnosis (RAD)