IEEE TRANSACTIONS ON COMPUTER GRAPHICS AND …vis.cs.brown.edu/docs/pdf/Jianu-2011-EBJ.pdf · 2012-01-12 · IEEE TRANSACTIONS ON COMPUTER GRAPHICS AND VISUALIZATIONS 1 Exploring

IEEE TRANSACTIONS ON COMPUTER GRAPHICS AND VISUALIZATIONS 1

Exploring brain connectivity withtwo-dimensional neural maps

Radu Jianu, Member, IEEE, Cagatay Demiralp, and David H Laidlaw, Senior Member, IEEE

Abstract—We introduce two-dimensional neural maps for exploring connectivity in the brain. For this, we create standardstreamtube models from diffusion-weighted brain imaging data sets along with neural paths hierarchically projected into the plane.These planar neural maps combine desirable properties of low-dimensional representations, such as visual clarity and ease oftract-of-interest selection, with the anatomical familiarity of 3D brain models and planar sectional views. We distribute this type ofvisualization both in a traditional standalone interactive application and as a novel, lightweight web-accessible system. The webinterface integrates precomputed neural-path representations into a geographical digital-maps framework with associated labels,metrics, statistics, and linkouts. Anecdotal and quantitative comparisons of the present method with a recently proposed 2D pointrepresentation suggest that our representation is more intuitive and easier to use and learn. Similarly, users are faster and moreaccurate in selecting bundles using the 2D path representation than the 2D point representation. Finally, expert feedback on theweb interface suggests that it can be useful for collaboration as well as quick exploration of data.

Index Terms—DTI fiber tracts, Abstraction, Filtration, Path Immersion, Interaction, Coloring

✦

1 INTRODUCTION

LOW-DIMENSIONAL point representations have beenproposed recently for better interaction with fiber

tracts obtained from diffusion-weighted magnetic resonanceimaging (DWI) datasets [1], [2]. Driven by known embed-ding methods, these representations provide an interestingwindow into the manifold space of neural connectivity andhelp in fine selection of tracts. A significant drawback ofpoint representations is, however, that coordinate axes in thelow-dimensional space lack an anatomical interpretation. Itis clear from evaluations in [1], [2] that having a frameof reference, anatomical or otherwise, is important forusers. Motivated by this problem, we present here two-dimensional neural maps that have the desirable propertiesof low-dimensional representations while preserving mean-ingful and familiar coordinates.

DWI enables neural pathways in the in vivo brain tobe estimated as a collection of space curves, called atractogram. The study of tractograms (i.e., tractography)has important applications in both clinical and basic neu-roscience research on the brain. Tractograms have visualcomplexity proportional to the intricacy of the axonal brainconnectivity and, with increasing DWI resolutions, thiscomplexity is becoming greater and greater. It is thus oftendifficult for practitioners to see tract projections clearlyor identify anatomical and functional structures easily inthese dense curve collections. This is important because, forexample, a clinical study into a neurodegenerative diseasetypically involves selecting more than 30 TOIs manuallyacross different datasets. Therefore, it is necessary for

• R. Jianu, C. Demiralp and D. H. Laidlaw are with the Department ofComputer Science, Brown University.E-mail: jr,cad,[email protected]

tractography visualization tools to provide means to reduceand help cope with visual complexity at interaction and rep-resentation levels. We believe that two simple but powerfulideas, abstraction and filtration, can help users overcomethe difficulties of visual complexity. While abstractioninvolves simplification and generalization, filtration hereentails clustering and hierarchization. Applying these twoideas without sacrificing the intuitiveness of representationsis a theme of the present work. To this end, we first obtaina hierarchy of two-dimensional neural diagrams from awhole-brain tractogram, inspired in part by illustrationsin medical textbooks [3]. These can be considered im-mersions of neural paths in the plane. We then link thetwo-dimensional neural maps with the three-dimensionalstreamtube representations. We also create a web interfacefor our representation by integrating it into a standardgeographical digital-map framework [4] and enhancing thiswith labels, statistics, and links (see Figure 1). We assessthe usefulness of our approach in comparison to the 2Dpoint representation in two studies: one anecdotal and onequantitative. Anecdotal study results indicate that our newrepresentation is more intuitive and easier to use and learn.Results of the quantitative study shows that users are fasterand more confident with the 2D path representation thanwith the 2D point representation. Also, users characterizedthe web interface with its digital-map-like interaction asuseful for collaboration and quick exploration of data.

In this context, our work has three main contributions,which are primarily in the field of tractography visual-ization. Our first contribution is a two-dimensional pathrepresentation of tractography datasets that, in contrast tothe previously proposed 2D point representation, preservesanatomically meaningful and familiar coordinates whilepossessing the advantages of low-dimensional representa-tions. By construction, this representation is conducive to

Digital Object Indentifier 10.1109/TVCG.2011.82 1077-2626/11/$26.00 © 2011 IEEE

This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication.


Fig. 1: Schematic planar projections of DTI tractograms as part of a standalone interactive system (left) and as a web-accessible digital map (right). The digital map interface easily incorporates any tract-associated information, includinglabels, links, metrics, and statistics. Shown in the pop-up window on the right is the “brain view” of the selected tract.

be navigated and interacted with like a digital geographicmap. Consequently, the second contribution is a web-interface that allows the exploration and annotation oftractography datasets like online geographic maps. Thisopens up exciting possibilities of, not just sharing, but alsoenriching tractography datasets by tapping into the vastknowledge base of the Web. Our third contribution is ananecdotal and quantitative comparison of the 2D point andpath representations, results of which provide a handle onthe relative merits of each representation.

DWI is the source of all the data used in the workpresented here. We first give an elementary introduction toDWI and then briefly discuss related work on tractographyvisualization.

2 RELATED WORK

Diffusion-weighted magnetic resonance imaging (DWI)measures the diffusion rate of water molecules in biologicaltissues in vivo [5]. Since tissue characteristics, geometricor otherwise, at a given point affect the diffusion rate,measured diffusion-rate information is an indicator of thetissue characteristics at the point. In particular, water infibrous tissues such as brain white matter (i.e., a collectionof myelinated axons) diffuses faster along fibers than or-thogonal to them. Therefore, it is possible to estimate fibertrajectories computationally using diffusion models such asthe tensor model that quantify anisotropic diffusion. Diffu-sion imaging based on fitting second-order tensors to DWIsequences is known as diffusion-tensor magnetic resonanceimaging (DTI) [6]. Fiber trajectories are computed fromDTI data by integrating bidirectionally along the principaleigenvector of the underlying tensor field. This process,called fiber tracking, yields a dense collection of integral

curves (i.e., a tractogram). All the tractograms used in ourwork were obtained by fiber-tracking in DTI volumes.

Tractograms are often visualized with streamlines or varia-tions of streamlines in 3D [7], [8]. Reflecting the intricacyof the connectivity in the brain, these 3D models are gener-ally visually dense. Consequently, typical interaction tasksover tracts, such as fine bundle selection, are often difficultto perform and have been a focus of recent research [9],[10]. Planar point representations have been proposed forimproving interaction with DTI fiber tracts [1], [2]. Therepresentation introduced here is also a projection of fibertracts into a plane, but as planar curves rather than points.

The complex structure of tractography datasets has moti-vated earlier work to apply clustering techniques to trac-tograms (e.g., [11], [12], [13], [14], [15]). The success of atract clustering is often determined by the degree to whichthe similarity measure used in the process can captureanatomical features that are of interest to a specific user. Acomparison of some of the existing tractography clusteringmethods and similarity measures can be found in [16].Clustering is an important tool for creating abstractions andfiltrations in general and is central to our representation inparticular. We use an agglomerative hierarchical clusteringalgorithm. The output of the clustering algorithm is a hier-archical tree called dendrogram. The height of the tree canbe thought as the radius of the bounding ball of the dataset–in the units of the similarity measure used. Any horizontalcut on this tree provides a clustering of the dataset. Toexemplify, the root node can represent a clustering whereall data-points are contained in a single cluster. Conversely,the leaf nodes correspond to a clustering where each datapoint is a cluster. The similarity measure parametrizes thenested space of clusterings. In this sense, the dendrogramembodies a filtration of the dataset.



Use of simplified, multiscale, and multiview representationshas a long history in interaction and visualization research(e.g., [17], [18], [19]). For example, multiple orthogonal3D views is fairly common in computer aided design(CAD), modeling, and animation applications. Similarly,use of 2D orthogonal cross-sectional views is a de-factostandard in volumetric medical data visualization [20]. Oursystem interface of multiscale orthogonal 2D neural pathprojections linked with each other and with a view of the3D model can be considered an implicit combination ofthese earlier research ideas as well as the current standardpractices.

Our 2D rendering of neural paths is partly inspired by2D illustrations of white-matter structures in medical text-books [3]. The appeal of hand-drawn illustrations can beattributed to the power of abstraction based on perceptualsimplification and generalization. Therefore, there havebeen several works in illustrative visualization [21], [22],[23]. Also, stylized line renderings of general streamlinesand tractograms in 3D have been used in the past [24],[25]. Recently, Otten et al. applied illustrative renderingtechniques to visualize white-matter bundles in 3D to easethe exploration of complex datasets [26]. We use simple,contoured 2D spline curves to draw our neural pathsrepresentations.

One of the advantages of 2D projections is that they canbe naturally integrated into a web-based digital geographicmap framework. Basic data visualization has been availableon the web for many years but was usually limited totraditional techniques such as bar graphs and charts. Morerecently, however, visualization research started targetingthis environment and advanced applications have emerged.ManyEyes [27] paved the way for everyday data visu-alization, with subsequent studies such as [28] and [29]proving the need for accessible web visualization. Whileweb-development toolkits such as [30] greatly aid webvisualization development, large-scale web-visualization islimited by inherent browser capabilities, as demonstratedin [31]. Alternatively, stand-alone systems have been madeavailable in applet form or can be run as client applicationsdirectly from websites. However, users still must controlthe parameters involved in producing visualizations, specifytheir data queries and learn the system features. This oftenconstitutes an undesirable overhead. Yet another approach,most similar to our work from an implementation stand-point, is to use Ajax (asynchronous JavaScript and XML)technology to perform the rendering on the server side andserve images asynchronously to the client browser. The es-sential difference between the present work and traditionaloffline visualization systems is that the former separatesinteraction and display from rendering and computation.Our brain maps differ by eliminating users’ efforts increating visualizations and assigning this task to expertsand by using the Google Maps API, which is an Ajaxframework for interactive display of pre-rendered images.Closest to our work in this latter aspect are X:MAP [32]and Genome Projector [33], which present genome browser

Fig. 2: Schematic tract-cluster representation. (Top) 2Dprojections of a tract-bundle, with an associated centroidcurve (orange), are determined from a hierarchical clus-tering of initial 3D tracts. (Middle) The centroid curve issmoothed by a spline and the endpoints of non-centroidcurves are clustered using their initial 3D coordinates (fourclusters); for each cluster, three control points linking thecenter of the cluster to the centroid spline are computed.(Bottom) Splines are run from each curve endpoint throughthe control points of its corresponding cluster.

tools implemented using the Google Maps API. We extendthis idea to a new domain and demonstrate its usefulnessfor tractography datasets.

3 METHODS

We create neural map representations in five basic steps.First, we obtain a whole-brain tractogram by fiber trackingin a diffusion-tensor volume fitted to a given DWI brainsequence. Second, we compute similarities between allpairs of tracts within the tractogram, obtaining a similarity(or affinity) matrix. Third, using the similarity matrixfrom the previous step, we run a hierarchical clusteringalgorithm on the tractogram, obtaining a clustering tree(i.e., dendrogram). Fourth, primarily using cluster cen-troids at different levels of the clustering tree as pivots,we create hierarchical projections of tractograms onto themajor orthogonal planes. Finally, we render these 2D curvesstylistically using heuristics determined by the topologyand geometry of the corresponding tracts and tract clusters,creating neural maps. We give details on these steps insubsequent sections.



Fig. 3: Depth ordering of 2D paths. For each segment ofa 2D spline, we locate a corresponding segment on the 3Dcurve from which the spline was derived by traveling thesame fractional distance along both curves. The depth ofthe 2D segment is the same as the depth of the middle ofits corresponding 3D segment.

3.1 Image Acquisition and Fiber Tract Generation

DWI brain datasets used in this paper were acquired fromhealthy volunteers on a 1.5T Siemens Symphony scannerwith the following acquisition parameters in twelve bipolardiffusion-encoding gradient directions: thickness = 1.7 mm,FOV = 21.7 cm × 21.7 cm, TR = 7200 ms, TE = 156ms, b = 1000, and NEX = 3. For each DWI sequence,the corresponding DTI volume was then obtained by fittingsix independent parameters of a single second-order tensorat each voxel to the twelve measurements from the DWIsequence [6]. We generate fiber-tract models of the wholebrain by integrating (second-order Runge-Kutta integration)the major eigenvector field of the diffusion tensor fieldbidirectionally starting at seed points. We integrate witha constant step size of 0.5 mm and stop integration whena gray-matter area is reached or the linear anisotropy orsignal-to-noise (SNR) ratio at the current point passespredetermined thresholds. Details of our integration schemecan be found in [8].

3.2 Similarities Between Fiber Tracts

We quantify the similarity between two tracts using theweighted chamfer distance discussed in [34]. This measuretries to capture how much any given two tracts follow asimilar path, while giving more weight to the points closerto tract ends. We compute distances between each pair offiber tract using λ = 0.5 as described in [34] and assemblethe measures to create a distance matrix used in clusteringand projection.

While our approach is independent of a particular similaritymeasure, good results in practice require a good similaritymeasure – one that reflects users’ understanding of thesimilarity between data points (i.e., tracts) and works wellfor the task at hand.

3.3 Clustering and Projection

For a given 3D DTI streamtube model we create schematicviews of major tract bundles projected on a few selectedplanes. In this work we choose the sagittal, coronal, andtransverse planes as our main modes of representation.

We first compute a clustering tree using an average-linkagehierarchical clustering algorithm on the tract distance ma-trix (e.g., [35]). We choose the average-linkage criterionbecause it is less sensitive than the minimum-linkage tobroken tracts due to tracking errors. We obtain a clusteringof tracts by manually setting a cut threshold on the dendro-gram. This threshold can be also interactively changed byuser to control the coarseness of the clustering. A constantcut at 60% of the clustering tree’s height gave consistentresults across the six datasets we experimented with.

Next, we create simple orthogonal projections of tractson each plane. We cull out tracts that do not contributesignificantly to the projection. If the ratio of projected tractlength to true tract length is under a threshold value, weremove the tract from the corresponding cluster. We set theculling threshold to 0.65 for the projections used in ourexperiments.

Finally, we compute a centroid for each cluster by choosingthe tract with the smallest maximum distance to any othertract in the cluster. We found that for illustration purposesit is desirable to avoid broken tracts. We therefore weighthe centroid selection to favor longer tracts by dividing themaximum distance from each tract to any other tract by thetract’s length.

3.4 Rendering

We opted for an illustrative rendering of brain projections.Illustrative visualization uses abstraction to reveal structurein dense visualizations and to harness scientists’ familiaritywith textbook representations [36]. Both criteria apply towhite matter tractograms: fiber bundles provide a natu-ral abstraction of 3D anatomy that avoids the clutter oflarge streamtube collections, while textbook illustrations [3]shape the intuition of neuroscientists. These advantageshave also been recognized and explored by Otten et al [26].

The rendering assumes a given clustering with assignedcentroid tracts, which can be computed as described inthe previous section. Our approach is inspired by Holten’shierarchical edge bundles [37] in attempting to group allfiber-tracts from a bundle into one, visually salient struc-ture. However, hierarchical edge bundles operate on abstractconnections that are unconstrained by concrete geometrical



Fig. 4: An interactive analysis system using linked views and planar tract-bundle projections. Three planar representations,along the coronal, transverse and sagittal planes (bottom panels), are linked to a 3D streamtube model (upper left) and a2D point embedding of tract similarities (upper right). Selections in the projection views can be performed by clicking orcutting across cluster curves and are mirrored in the 3D view. Points corresponding to the selected tracts are interactivelyembedded into the plane and used to refine selections at tract level.

shapes. They can therefore be drawn according to visualaesthetics principles alone. Conversely, fiber tract paths playimportant anatomical functions and should be preservedin tractogram visualizations. To this end, we perform ourbundling by routing tracts along the path of the mostrepresentative tract in their bundle. Thus, the centroid tractswill define a schematic neural skeleton on top of which thenon-centroid tracts are scaffolded.

Projections of centroid curves are smoothed prior to ren-dering to achieve a schematic representation and to reduceclutter. This is done by sampling a number of evenlydistributed control points (five in our implementation) alongthe tract projection and using them as control points for aspline. In our implementation the spline is piecewise cubicand consists of 30 segments. The thickness of a centroidcurve is proportional to the square root of the number oftracts in the bundle.

Once centroid tracts are represented as 2D splines, end-points of non-centroid curves are linked to their cluster’scentroid spline following the procedure illustrated in Fig-ure 2. First, the end-points of non-centroid curves in abundle are clustered based on the end-points’ initial 3Dcoordinates. Two endpoints are placed in the same clusterif the distance between them is less than 2 mm. Then, foreach such endpoint cluster we compute three control pointsthat link the geometrical center of the endpoint cluster to thecentroid spline: the first point is the center itself, the secondis a point on the centroid spline closest to the center point,

and the third is determined by traveling from the secondpoint down the centroid spline, towards each curve’s otherendpoint, for a predefined distance (e.g., half of the distancebetween the first two points). Ultimately, splines are runfrom each tract endpoint through its cluster’s three controlpoints, thus linking each endpoint to the centroid path.The thickness of these endpoint linkage splines graduallyincreases from unit thickness (i.e., single-tract thickness) atthe tract endpoint to a thickness proportional to the squareroot of the endpoint cluster size, where it merges with thecentroid spline.

We depth-order spline segments so that 2D centroid splinescrossings can indicate the depth ordering of their corre-sponding 3D shapes. The depth ordering is done differentlyfor centroid splines and non-centroid splines, since whilecentroid curves are close representations of actual 3D tracts,non-centroid curves are abstract representations obtainedthrough the process described above. Furthermore, thedepth ordering is approximate (as discussed in the followingparagraph) and may produce artifacts.

For centroid splines, the depth of a spline segment iscomputed by finding a matching segment on the 3D tractfrom which the spline was derived, and taking the depthof that segment’s center (Figure 3). The matching segmenton the 3D tract has its endpoints at the same fractionaldistance from the start of the 3D tract as the 2D segment’sdistance from the start of the 2D spline. This per-segmentordering was chosen because of the intricacy of white mat-



ter tractograms. Tracts often wrap around each other suchthat a correct per-tract depth ordering cannot be determined.Treating each curve segment independently maximizes theprobability that the 2D rendering remains truthful, at leastwithin the resolution of the tract segmentation. Conversely,non-centroid splines are completely abstract 2D represen-tations. The depth of any non-centroid spline is determinedby averaging the depth of the corresponding 3D tract.Wrapping fiber tracts are therefore not captured by thislatter process.

Finally, bundle-color, texture or thickness can be used asadditional depth cues. While we have not fully integratedand evaluated such encodings in our current prototypes, wehave experimented with color cues and found those to beuseful.

In the following two sections, we give details on how weuse 2D neural path representations as part of an interactiveapplication and as standalone digital maps.

3.5 Interactive Application

We expect a typical use of low-dimensional representationsto be as part of interactive applications where views andinteractions of conventional representations are linked withthat of low-dimensional representations. Therefore, we havedeveloped an interactive visualization system using the 2Dpath representation to demonstrate this mode of use (seeFigure 4). We link the three views of the 2D path represen-tations with a view of the 3D streamtube representation. Wecolor streamtubes by embedding the tract similarity matrixin the L*a*b* color space [2].

The clustering cut threshold that defines the specificity ofthe projected bundles can be altered interactively duringvisualization. Tract clusters in the planar projections can beselected by drawing line segments that select intersectingbundles. A selection in any of the planar views is mirroredin the 3D model view as well as all other 2D projections.

In addition to the standard 3D viewing interactions, we havetwo basic selection/deselection interactions on streamtubemodels: sphere-selection and brushing. Sphere-selection,like box-selection, enables user to select the intersectingtracts by moving a sphere of desired radius. Brushing letsuser draw 2D curve on the viewing plane and select theintersecting tracts.

Although both sphere-selection and brushing can be used tofurther prune the current selection, they cannot be used togrow it. For this purpose, we provide a selection-growinginteraction that gradually adds tracts closest to the currentselection. Proximity is determined again by the distancemeasure discussed above. Selections can be saved for futureanalysis. Moreover, statistics such as average tube length,number of tubes or average fractional anisotropy can becomputed interactively on sets of selected tracts.

3.6 Digital Map Interface

Brain mapping is one of the quintessential problems in neu-rosciences. We believe that a geographical map metaphor iswell-suited to the visualization and analysis of results ob-tained in this direction. Therefore, having a representationof the brain that is viewed, interacted, queried, and enrichedlike an online geographical map was one of the motivationsbehind our creation of the 2D path representation.

For this, we use the Google Maps API, an Ajax frameworkused to render large maps, to interactively display our trac-togram maps on the web. The Google Maps API receivesinput image data in the form of a set of small images,called tiles, that when assembled together form the differentzoom levels of the map. Each zoom level, z, consists of arectangular grid of tiles of size 2z× 2z. The API decodesthe zoom level and coordinates of the currently viewed mapregion to retrieve and display the visible tiles. The developercan load a custom set of tiles in the API by implementing acallback function that translates numerical tile coordinatesand zoom level into unique paths to the custom tiles.

The API provides basic functionality such as zoomingand panning and allows programmatic extension or cus-tomization with markers and polyline overlays, informationpop-ups and event management. The API can easily beintegrated into any webpage supporting Javascripts.

Our visualization system can render our 2D projections intoa set of image tiles instead of the screen. For each cluster,including both tract-bundled and endpoint clusters, weexport information required for interaction and browsing.Selection information consisting of evenly spaced pointsalong splines and thickness radii for splines contained in acluster is exported. In line with the tile paradigm, instead ofexporting this information to a single large file, we divideit geometrically across corresponding tiles and write it asmultiple tile-content text files. Upon user selection, thecontent file of a clicked tile is fetched from the server andits data analyzed for an intersection. This approach avoidsloading and searching through large files. A valid clusterselection is marked on the map with polyline overlaysrunning over tract splines contained in the selected cluster(see Figure 5). For this purpose, spline coordinates for eachcluster are exported to files indexed by a unique clusteridentifier.

Finally, for each tract cluster we export a variety of meta-data accessible during map browsing in information boxes,as shown in Figure 5. A short description and links tothe most relevant publications or research can be manuallyadded for major tracts. A few 3D poses of each tract bundleare prerendered and exported as animated GIF images,indexed by the cluster identifier. Statistical data, in bothtextual and graphical form, are computed for each clusterand written as HTML content to cluster indexed files. Thisinformation is loaded and displayed in tabbed informationboxes at the user’s request.



Fig. 5: DTI tractography data projected onto the sagittal, coronal and transverse planes. Major tract bundles are representedschematically by their centroid tract; individual tracts in bundles are linked from the centroid bundle to their projectedend points. Zooming in allows access to smaller clusters of tracts. Bundles can be selected and pre-computed statisticaldata along with 3D views of the tract bundle (“brain view”) can be displayed.

3.7 Implementation

We have implemented a prototype of our method in C++using G3D and Qt libraries [38], [39], [4]. The webinterface based on the Google Maps API of this prototypecan be accessed via the url link [40]. Note that the currentweb interface implementation provides only a single viewat a time. It is, however, possible to have linked multipleviews, as in the standalone application, using a cookie-pooling mechanism.

4 USER EVALUATION

We compared our method to a 2D point-based representa-tion both anecdotally and quantitatively.

4.1 Anecdotal Study: Methods and Results

In the anecdotal study we showed a prototype that im-plements both 2D point and neural map representationsto three neuropsychologists. They were all interested inthe relationship between fiber tracts and cognitive andbehavioral function in the brain. Similarly, all have usedcomputational tools for analyzing DTI data, though onlyone of them had used fiber-tract visualization tools in hisclinical research. The participants had research interests invascular cognitive impairment, early Alzheimer’s disease,and HIV, focusing on specific tracts such as the corpuscallosum (CC), frontal lobe, basal ganglia, cingulate bundle,superior and inferior longitudinal fasciculi, anterior internalcapsule, and the uncinate fasciculus.

Our anecdotal evaluation protocol was straightforward: wedemonstrated the prototype while asking questions andcollecting participant’s feedback. Two of the experts alsotried both interfaces themselves by selecting a set of majorTOIs, the CC, cingulate bundle, uncinate anterior internalcapsule, and the corticospinal tract. There was agreementthat our new interface is more intuitive and easier to useand learn than the 2D point representation.

Our experts also found the web interface with the digitalmap interaction useful. Although they believed that thestandalone application with linked representations wouldremain necessary for quantitative analyses that requirefine selection, they thought the web accessibility openedup interesting possibilities. They were particularly excitedabout browsing through datasets while commuting or athome, of quickly inspecting unfamiliar datasets, and ofsharing such visualizations with collaborators.

4.2 Quantitative Study

In this experiment we compared the point and path rep-resentations by measuring user performance on a bundleselection task with the two representations.

4.2.1 Application

We compared our path-visualization system (Section 3.5)against the one presented in [1], which consists of a3D streamtube model view and a linked 2D point rep-resentation. The system offers a brush tool that workssimilarly to ours in 3D and as a lasso tool on the 2D point



representation. Users can select tracts or points and thenremove them or, conversely, remove everything else froma current selection.

4.2.2 Participant Pool

The four subjects were all familiar with neuroanatomy andtractography. They also had experience with one or moretractography visualization tools. Our first subject was aneuroscience graduate student, working on tracing white-matter tracts from frontal subregions to basal ganglia andthe medial temporal lobe. Our second user was a postdocin neuropsychology and had five years of experience withDWI in clinical research. This user, who participated inthe anecdotal study as well, studied white matter variationwith neurodegenerative diseases as specified above. Ourthird subject was a biomedical engineering graduate studentand had significant tract-selection experience working as arater for a neuroscientist. Our last subject was a computer-science graduate student doing research on computationalDWI algorithms. Two of the users were male and twofemale.

4.2.3 Task

We measured user performance on bundle selection, atypical real world task in tractography tools. Users wereasked to select three major bundles, the cingulate bundle(cb), corticospinal tract (cst), and right superior longitu-dinal fasciculus (slf), in two different brain datasets. Wechoose these bundles because they represent the easy-to-hard selection-difficulty range well and were used forevaluation in [1].

For each system, we explained to users the underlyingvisualization concepts and demonstrated the basic inter-actions, mainly involving brushing on 2D and streamtuberepresentations. After this introduction, users were asked toselect the bundles (cb, cst, and slf) on two different trainingdatasets. Following training, the users performed the taskon two different test datasets while we collected theirtask-completion times. After each selection they providedsubjective confidence estimate in the range 1-5 (1: notconfident, 5: very confident) for their selection. They couldgive fractional estimates. After completing the task on bothsystems, users were asked to complete a post-questionnairegiving qualitative feedback on their experience. Half theusers performed the task first on the 2D point-representationtool and the other half on the 2D path tool.

4.2.4 Factors and Measures

The sole factor considered in our quantitative experimentwas the type of low-dimensional representation: 2D pointand 2D path. All subjects used both types of representation.We recorded the users’ bundle-selection times and subjec-tive confidence values as measures of performance.

0

20

40

60

80

100

120

140

160

�me (secs)

user1

user2

user3

user4

mean

(a)

-0.6

-0.5

-0.4

-0.3

-0.2

-0.1

0

subjec�ve

confidence

user1

user2

user3

user4

mean

(b)

Fig. 6: Per-user differences between (a) time and (b) con-fidence measurements with the two tools. Differences areobtained by subtracting 2D-point-tool performance valuesfrom 2D-path-tool performance values. Red squares showthe mean performance difference between the tools. Errorsbar around the red squares indicate the standard error ofper-user differences.

time (secs) confidence

cb cs slf mean cb cst slf mean

2D point 227 361 234 274 4.1 3.3 3.1 3.52D path 136 165 215 172 4.1 3.8 3.7 3.9

TABLE 1: User performance on bundle selection task.

4.2.5 Results

In order to understand if the differences between userperformances on the two tools were significant, we ran thepaired t-test on our measurements. Results show that userswere significantly faster on the 2D path tool than the 2Dpoint tool (p = 0.02). Users were also significantly moreconfident with using the 2D path representation than the2D point representation (p = 0.01). Table 1 summarizesusers’ overall and per-bundle mean performances on eachtool. Figure 6 shows the difference between the meansof performance measures per user (2D-path-performancevalues are subtracted from 2D-point-performance values)and the mean over users. Errors bars indicate the standarderror of per-user differences.

We observed some interaction patterns worth reporting. Wenoticed two distinct selection strategies used with the 2Dpath tool. Two of the users consistently brushed over largeareas of the projection to ensure that the targeted bundlewas selected and then relied on the 3D view to clean upthe selection. The other two users aimed for fine selectionsin the 2D projections and then inspected the 3D view todetermined whether any fibers the selection. They addedthe missing tracts using short, targeted brushstrokes andthen removed tubes that were erroneously added duringthis operation. These users seemed to have a better under-



standing of the mapping between the 3D view and the 2Dprojections, perhaps explaining the difference in strategies.

All subjects used the 2D point representation relativelyrarely. The most common operation was to remove pointsthey were completely confident were not part of the selec-tion (e.g., half of the brain, or peripheral U-shaped bundles).However, in the absence of a clear contextual mappingbetween the 2D point and streamtube views, subjects werehesitant to perform bold operations in 2D, at least in theshort run.

5 DISCUSSION

It is important to note that our representation relies on theanatomical fidelity of the intermediate results at each step.For example, broken trajectories due to fiber tracking errorsor noise can reduce the effectiveness of the representation.In general, the overall robustness of our representation tonoise will be determined by the fiber tracking algorithmused to generate the tractograms.

Our method also expects the clustering algorithm and sim-ilarity measure to provide anatomically plausible results.However, it is difficult for a single distance measure tocapture the anatomical similarity completely. Furthermore,on the same data, a good similarity measure for onepurpose can be entirely irrelevant for another. While thechoice of similarity measure makes clustering a subjectivetask, clustering algorithms themselves also have intrinsiclimitations [41].

On the other hand, we believe that our representation, whichis driven by cluster centroids, is stable. It is importantto observe that our cluster centroids are existing tracts indatasets. They are not, for example, the medial-axes ofthe corresponding cluster surfaces, which tend to be lessstable [42]. Therefore, we expect small perturbations tothresholds or datasets to cause only ignorable changes inthe representation.

One of the limitations of our visualization pipeline isits reliance on adjustable thresholds. Each computationalstep introduces variability that is hard to account forautomatically. More robustness in tractography, similar-ity measures and clustering could lead to anatomicallymotivated thresholds. However, we note that variabilityis also present in neuroscientists’ assessments of whatconstitutes a valid bundle and that different tasks oftenrequire different clustering granularity. This suggests thatmanually adjustable thresholds are perhaps unavoidable.Still, one can imagine providing adaptive tools based onpersistence that can help users to adjust these the thresholdsmore efficiently, minimizing the number of trial-and-erroriterations.

Another potential limitation of the current method is thatbundles surrounded by other bundles similar in orientationand shape may not be clearly visible. While we have notfound this to be an issue in practice, moving projection

planes along major axes while restricting the projectingtracts to a volumetric window around the projection planecan help solve potential problems.

Also, results of user studies should be taken with a grainof salt. In general, it is difficult to run experiments thatvary one factor while keeping all the other factors constant.For example, an earlier study [1] compared the 2D pointrepresentation tool with other tractography applications andreported that users were faster with the former. However, inour user evaluation, we observed that users rarely used the2D point representation and the brush tool dominated theirinteraction. This brings up an open question of whetherthe performance difference in the reported evaluation wasdue mainly to the brush tool or to the 2D point represen-tation. An experiment that replaced the brush tool with amore standard box-selection tool, say, might resolve thisquestion. In either case, we believe that abstract represen-tations, including the 2D point representation, are usefulin the long run, as users gain more experience with themapping between brain tractograms and low-dimensionalrepresentation primitives.

While DWI is the only imaging protocol to estimatethe brain neural architecture in vivo, there are in vitroimaging techniques, such as the three-dimensional elec-tron microscopy particularly used in the emerging field ofconnectomics, with which it is possible to extract neuralstructures on much smaller scales (e.g., individual axonbodies) [43]. We believe that the general ideas as well asspecific techniques presented in this paper can extend tothe visualization and analysis of visually complex axonalstructures originating from these high-throughput imagingtechniques.

6 CONCLUSIONS

We have presented 2D neural maps, an intuitive, low-dimensional representation for tractograms, that facilitatesexploration and analysis of brain connectivity. Our repre-sentation is essentially an application of abstraction andfiltration concepts to tractograms with preservation ofanatomical context in mind. We achieve abstraction byboth geometrically and topologically simplifying and gen-eralizing fiber tracts with 2D schematic curves. We createfiltrations of tractograms by computing hierarchical clus-tering trees. These help create better abstractions as wellas provide a multiscale view of data, which is importantin reducing visual complexity and noise. Both qualitativeand quantitative user evaluation results suggest that our newrepresentation can significantly improve user performancein fiber bundle selection.

We also introduced a novel way of making tractographydata accessible by publishing neural maps online through adigital map framework. Our representation is conducive tosuch a geographic map interface by construction. This in-terface leads to new possibilities for enriching tractographydatasets using the mass knowledge base available on the



web. User feedback indicates that our web interface can beparticularly useful for browsing unfamiliar datasets quickly,for analysis tasks that do not require fine selection and forsharing data in collaborative settings.

ACKNOWLEDGMENT

We thank Song Zhang for generously providing us the2D point representation tool used for comparison in ourquantitative evaluation. This work was supported by NIHgrant 1R01EB00415501A1.

REFERENCES

[1] W. Chen, Z. Ding, S. Zhang, A. MacKay-Brandt, S. Correia, H. Qu,J. A. Crow, D. F. Tate, Z. Yan, and Q. Peng, “A novel interfacefor interactive exploration of DTI fibers,” IEEE TVCG (Proc. ofVisualization), 2009.

[2] R. Jianu, C. Demiralp, and D. Laidlaw, “Exploring 3D DTI fibertracts with linked 2D representations,” IEEE TVCG (Proc. of Visu-alization), vol. 15, no. 6, pp. 1449–1456, 2009.

[3] H. Gray, Anatomy of the Human Body. Lea & Febiger, 1918.

[4] GoogleMapsAPI, http://code.google.com/apis/maps/.

[5] J. E. Tanner, “Transient diffusion in system partitioned by permeablebarriers. Application to NMR measurements with a pulsed fieldgradient,” Journal of Chemical Physics, vol. 69, no. 4, pp. 1748–1754, January 1978.

[6] P. J. Basser, J. Mattiello, and D. LeBihan, “Estimation of the effectiveself-diffusion tensor from the NMR spin echo.” J Magn Reson B,vol. 103, no. 3, pp. 247–254, March 1994.

[7] S. Mori and P. Van Zijl, “Fiber tracking: principles and strategies–atechnical review,” NMR in Biomedicine, vol. 15, no. 7-8, pp. 468–480, 2002.

[8] S. Zhang, C. Demiralp, and D. Laidlaw, “Visualizing diffusion tensorMR images using streamtubes and streamsurfaces,” IEEE TVCG,vol. 9, no. 4, pp. 454–462, 2003.

[9] D. Akers, “Wizard of Oz for participatory design: Inventing an inter-face for 3D selection of neural pathway estimates,” in Proceedingsof CHI 2006 Extended Abstracts, 2006, pp. 454–459.

[10] D. Akers, A. Sherbondy, R. Mackenzie, R. Dougherty, and B. Wan-dell, “Exploration of the brain’s white matter pathways with dynamicqueries,” in Proc. of Visualization, 2004, pp. 377–384.

[11] J. S. Shimony, A. Z. Snyder, N. Lori, and T. E. Conturo, “Automatedfuzzy clustering of neuronal pathways in diffusion tensor tracking,”in Proc. Intl. Soc. Mag. Reson. Med., 2002.

[12] A. Brun, H. Knutsson, H. J. Park, M. E. Shenton, and C.-F. Westin,“Clustering fiber tracts using normalized cuts,” in Seventh Inter-national Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI’04), ser. Lecture Notes in ComputerScience, Rennes - Saint Malo, France, September 2004, pp. 368–375.

[13] L. O’Donnell and C.-F. Westin, “White matter tract clustering andcorrespondence in populations,” in Eighth International Conferenceon Medical Image Computing and Computer-Assisted Intervention(MICCAI’05), ser. Lecture Notes in Computer Science 3749, PalmSprings, CA, USA, October 2005, pp. 140–147.

[14] S. Zhang, S. Correia, and D. H. Laidlaw, “Identifying white-matter fiber bundles in dti data using an automated proximity-basedfiber-clustering method,” IEEE Transactions on Visualization andComputer Graphics, vol. 14, pp. 1044–1053, 2008.

[15] M. Maddah, W. E. L. Grimson, S. K. Warfield, and W. M. Wells, “Aunified framework for clustering and quantitative analysis of whitematter fiber tracts,” Medical Image Analysis, vol. 12, no. 2, pp. 191– 202, 2008.

[16] B. Moberts, A. Vilanova, and J. J. van Wijk, “Evaluation of fiberclustering methods for diffusion tensor imaging.” in Procs. of Vis’05,2005, pp. 65–72.

[17] R. Stoakley, M. Conway, and R. F. Pausch, “Virtual reality on awim: Interactive worlds in miniature,” in CHI, 1995, pp. 265–272.

[18] S. K. Card, J. D. Mackinlay, and B. Shneiderman, Eds., Readingsin information visualization: using vision to think. San Francisco,CA, USA: Morgan Kaufmann Publishers Inc., 1999.

[19] C. Ware, Information visualization: perception for design. SanFrancisco, CA, USA: Morgan Kaufmann Publishers Inc., 2000.

[20] W. Schroeder, K. M. Martin, and W. E. Lorensen, The VisualizationToolkit, 2nd ed. Prentice-Hall, Inc., 1998.

[21] D. Ebert and P. Rheingans, “Volume illustration: non-photorealisticrendering of volume models,” in Proceedings of Visualization 2000,Oct 2000, pp. 195 –202.

[22] A. Lu, C. J. Morris, J. Taylor, D. S. Ebert, C. Hansen, P. Rheingans,and M. Hartner, “Illustrative interactive stipple rendering,” IEEETransactions on Visualization and Computer Graphics, vol. 9, pp.127–138, 2003.

[23] W. Li, L. Ritter, M. Agrawala, B. Curless, and D. Salesin,“Interactive cutaway illustrations of complex 3d models,” ACMTrans. Graph., vol. 26, July 2007. [Online]. Available: http://doi.acm.org/10.1145/1276377.1276416

[24] C. Stoll, S. Gumhold, and H.-P. Seidel, “Visualization with stylizedline primitives,” Visualization Conference, IEEE, vol. 0, p. 88, 2005.

[25] M. H. Everts, H. Bekker, J. B. Roerdink, and T. Isenberg, “Depth-dependent halos: Illustrative rendering of dense line data,” IEEETransactions on Visualization and Computer Graphics, vol. 15, pp.1299–1306, 2009.

[26] R. Otten, A. Vilanova, and H. Van De Wetering, “Illustrative WhiteMatter Fiber Bundles,” Computer Graphics Forum, vol. 29, no. 3,pp. 1013–1022, 2010.

[27] F. Viegas, M. Wattenberg, F. Van Ham, J. Kriss, and M. McKeon,“ManyEyes: a site for visualization at internet scale,” IEEE Trans-actions on Visualization and Computer Graphics, vol. 13, no. 6, p.1121, 2007.

[28] F. Viegas, M. Wattenberg, M. McKeon, F. Van Ham, and J. Kriss,“Harry Potter and the meat-filled freezer: A case study of sponta-neous usage of visualization tools,” in Proc. HICSS, 2008.

[29] C. Danis, F. Viegas, and M. Wattenberg, “Your place or mine?Visualization as a community component,” in Proceedings of CHI,2008.

[30] M. Bostock and J. Heer, “Protovis: A graphical toolkit for visualiza-tion,” IEEE Transactions on Visualization and Computer Graphics,vol. 15, no. 6, pp. 1121–1128, 2009.

[31] D. Johnson and T. Jankun-Kelly, “A scalability study of web-nativeinformation visualization,” in Proceedings of Graphics Interface,2008, pp. 163–168.

[32] T. Yates, M. Okoniewski, and C. Miller, “X: Map: annotationand visualization of genome structure for Affymetrix exon arrayanalysis,” Nucleic Acids Research, vol. 36, no. Database issue, p.D780, 2008.

[33] K. Arakawa, S. Tamaki, N. Kono, N. Kido, K. Ikegami, R. Ogawa,and M. Tomita, “Genome Projector: zoomable genome map withmultiple views,” BMC bioinformatics, vol. 10, no. 1, p. 31, 2009.

[34] C. Demiralp and D. H. Laidlaw, “Similarity coloring of DTI fibertracts,” in Proceedings of DMFC Workshop at MICCAI, 2009.

[35] R. O. Duda, P. E. Hart, and D. G. Stork, Pattern Classification,2nd ed. Wiley-Interscience, 2000.



[36] I. Viola, M. Groller, M. Hadwiger, K. Buhler, B. Preim, M. Sousa,D. Ebert, and D. Stredney, “Illustrative visualization,” in IEEEVisualization, 2005, p. 124.

[37] D. Holten, “Hierarchical edge bundles: Visualization of adjacencyrelations in hierarchical data,” IEEE Transactions on Visualizationand Computer Graphics, pp. 741–748, 2006.

[38] G3D, http://g3d-cpp.sourceforge.net/.

[39] Qt, http://www.qtsoftware.com/.

[40] http://graphics.cs.brown.edu/research/sciviz/newbraininteraction/.

[41] J. M. Kleinberg, “An impossibility theorem for clustering,” in NIPS,2002, pp. 446–453.

[42] D. Attali, J.-D. Boissonnat, and H. Edelsbrunner, “Stabilityand Computation of Medial Axes - a State-of-the-ArtReport,” in Mathematical Foundations of Scientific Visualization,Computer Graphics, and Massive Data Exploration. Springer BerlinHeidelberg, 2009, ch. 6, pp. 109–125. [Online]. Available: http://hal.archives-ouvertes.fr/docs/00/46/86/90/PDF/04-medial-axes.pdf

[43] W.-K. Jeong, J. Beyer, M. Hadwiger, A. Vazquez, H. Pfister, andR. T. Whitaker, “Scalable and interactive segmentation and visual-ization of neural processes in EM datasets,” IEEE Transactions onVisualization and Computer Graphics, vol. 15, pp. 1505–1514, 2009.

Radu Jianu Radu Jianu received the Dipl.Ing. degree in Computer Science in 2005from Polytechnic University in Timisoara, Ro-mania. In 2007 he received the MS degree atthe Computer Science Department at BrownUniversity where he currently pursues a PhDdegree in Computer Science. His researchinterests are mainly in graphics and visual-ization with specific applications in genomicand proteomic biology.

Cagatay Demiralp Cagatay Demiralp is aPhD student in computer science at BrownUniversity. His research interests are in char-acterizing the pattern and structure in data.Computational brain connectivity is the focusof his current research.

David H. Laidlaw David H. Laidlaw receivedthe PhD degree in computer science from theCalifornia Institute of Technology, where healso did post-doctoral work in the Division ofBiology. He is a professor in the ComputerScience Department at Brown University. Hisresearch centers on applications of visualiza-tion, modeling, computer graphics, and com-puter science to other scienti?c disciplines.He is a senior member of the IEEE and theIEEE Computer Society.


Documents

IEEE TRANSACTIONS ON COMPUTER GRAPHICS AND …vis.cs.brown.edu/docs/pdf/Jianu-2011-EBJ.pdf · 2012-01-12 · IEEE TRANSACTIONS ON COMPUTER GRAPHICS AND VISUALIZATIONS 1 Exploring