Embed Size (px)
Citation preview
48 Copublished by the IEEE CS and the AIP 1521-9615/06/$20.00 © 2006 IEEE COMPUTING IN SCIENCE & ENGINEERING
Editors: Jim X. Chen, [email protected]
R. Bowen Loftin, [email protected]
V I S U A L I Z A T I O N C O R N E R
cause it. Given that a key process ismuscle development, researchers at aconsortium of 10 institutions arestudying 1,000 men and women, ages18 to 40 years, to see how their musclesenlarge with exercise. The 150 vari-ables collected for each participant willmake this data analysis task challeng-ing for users of traditional statisticalsoftware tools.
However, a new approach to visualdata analysis is helping these re-searchers speed up their work. At theUniversity of Maryland’s Human–Computer Interaction Library, wedeveloped an interactive approach tolet researchers explore high-dimen-sional data in an orderly manner, fo-cusing on specific features one at atime. The rank-by-feature frameworklets them adjust controls to specifywhat they’re looking for, and then,with only a glance, they can spotstrong relationships among variables,find tight data clusters, or identifyunexpected gaps. Sometimes surpris-ing outliers invite further study as towhether they represent errors or anunusual outcome.
Similar data analysis problems comeup in meteorology, finance, chemistry,and other sciences in which complexrelationships among many variables
govern outcomes. The rank-by-featureframework could be helpful to manyresearchers, engineers, and managersbecause they can then steer their analy-ses toward the action.
Understanding the Rank-by-Feature FrameworkWe can use a playful analogy to helpexplain this new way of thinking aboutdata. Imagine you parachute into anunknown landscape. You might lookaround to find something interesting,or maybe you climb a nearby hillside togain an overview. You notice a largerock and some sandy soil, admire abright green fern, smell a rose bush,and spot an oak tree. You might get an-noyed by a swarm of bees, recognize ared-tailed hawk flying above, and spotdeer tracks. You might also notice thesloping terrain, remember jagged cliffs,and become aware of the noisy streamin the valley below. In short, there’s alot to look at, just as there is in a high-dimensional database.
Recording “interesting features” isuseful, but if you have specific goals,such as cataloging the area’s plants toidentify potential pharmaceuticals, youmust be systematic and thorough. Ifyou’re determining the area’s suitabil-ity for rural development or trying to
protect endangered species, an orderlyand comprehensive approach is neces-sary. Professionals in botany, geology,civil engineering, and urban planninghave developed step-by-step ap-proaches to ensure thorough and stan-dardized evaluations that are consistentacross time and evaluators.
Such orderly approaches and toolsare being developed to support high-dimensional data as well. The prolificstatistician John Tukey proposed anapproach he called scagnostics in 1985,by which he meant scatterplot diag-nostics.1 He believed that scatterplotswith two of the many dimensionsalong the x- and y-axes were a compre-hensible way to look at data, but heknew that there were often too manysuch 2D projections to examine inlarge data sets. He proposed a few cri-teria for ranking scatterplots, but noone has implemented his ideas untilnow. The rank-by-feature frameworkimplements Tukey’s vision in an open-ended manner to allow easy addition ofnew criteria.
At the University of Maryland, doc-toral student Jinwook Seo added therank-by-feature framework to soft-ware he developed for his dissertationresearch on genomic data analysis.That software tool, the HierarchicalClustering Explorer (HCE),2 is avail-able for download at www.cs.umd.edu/hcil/hce (see the sidebar on p.52). HCE handles clustering of largedata sets to help biologists find simi-lar genes that might participate inprocesses related to muscular dystro-phy, cancer, cell death, and so on. The
A TELESCOPE FORHIGH-DIMENSIONAL DATABy Ben Shneiderman
M USCULAR DYSTROPHY IS A DEGENERATIVE DISEASE
THAT DESTROYS MUSCLES AND ULTIMATELY KILLS ITS
VICTIMS. RESEARCHERS WORLDWIDE ARE RACING TO FIND A
CURE BY TRYING TO UNCOVER THE GENETIC PROCESSES THAT
MARCH/APRIL 2006 49
addition of the rank-by-feature frame-work dramatically speeds up explo-ration of high-dimensional data.
A Simple Data SetWe can use a simple data set about au-tomobiles to help describe the rank-by-feature framework. Figure 1 shows fivevariables: horsepower (HP), maximumspeed (SP), weight (WT), volume ofcab space (VOL), and miles per gallon(MPG). The bright red coding showsthe highest correlation, and brightgreen indicates the lowest. The or-dered list shows all 10 pairs of variablesin ranked order. The user selects thehighest correlation, which is speed ver-sus horsepower, showing that to in-crease speed, users must ensure higher
horsepower. Figure 2 shows that it isequally easy to find the strongest neg-ative correlation, which is weight ver-sus miles per gallon, meaning that asweight goes up, mileage goes down.
We conducted an email surveyamong known HCE users, which pro-duced 57 completed responses. Thecorrelation coefficient was the most de-sired feature, but these users also usedthe ranking criteria to find strong out-liers, tight clusters, uniform distribu-tions, and large gaps. Figure 3 showsthat the strongest quadratic relation-ship in this simple data set was MPGand HP. In one of our eight-week casestudies, a meteorologist found quad-racity to be central to revealing an un-known pattern of how aerosols (small
particles) affect absorption and reflec-tion of infrared and ultraviolet light,potentially leading to improvedweather prediction models.
Outliers—data points that differstrongly from most others—can revealunusual items that either deserve moreattention or uncover mistaken values inlarge data sets. Figure 4 reveals six strongoutliers that deserve further study—first,to verify that the data are correct and,second, to understand how the automo-bile designers achieved high interior vol-ume while keeping the weight low.
Some rankings are more helpful in one-dimensional (1D) histogram,whereas others are well suited for 2Dscatterplots. The two separate but con-sistently designed interfaces let users
Figure 1. The automobile data set. The order-by criteria (left panel) is the correlation coefficient, producing a color-coded scores overview with five variables: horsepower (HP), maximum speed (SP), weight (WT), volume of cab space(VOL), and miles per gallon (MPG). The bright red coding shows the highest correlation, and bright green indicates thelowest. The ordered list shows all 10 pairs of variables in ranked order from bright red down to bright green. The userhas selected the highest correlation, which is speed vs. horsepower. The scatterplot in the far right panel shows that toincrease speed, users must ensure higher horsepower.
Figure 2. The automobile data set, ordered by the strongest negative correlation. In this case, it’s weight (WT) vs. milesper gallon (MPG).
50 COMPUTING IN SCIENCE & ENGINEERING
explore 1D features first, and then turntheir attention to the more complex2D features. In each case, the rank-by-feature framework provides a visualoverview, with red for high and greenfor low values, but users can set colorsas they see fit.
For many users, finding the 1D dis-tributions that were close to normal(bell-shaped curve) or furthest fromnormal was important. Figure 5 showshow easily users can discover that themaximum speed of cars was closest tobeing normally distributed.
Finding gaps is also a common taskfor data explorers. Figure 6 shows howto find the biggest gaps for this data,which occurred in the volume variable.
This simple data set demonstratesthe basic ideas, but the rank-by-feature framework’s substantial value
becomes apparent with much largerdata sets, such as a muscle data setwith 150 variables.
Muscle Exercise DataDealing with large data sets is moredifficult, and the time needed foranalysis much longer. It might takehours or days to check out the interest-ing features, but using the rank-by-fea-ture framework can give users amethod and tool to guide them and en-sure that their analysis has been com-prehensive and systematic.
The 150 variables in the muscle ex-ercise study noted earlier include age,gender, muscle strength before and af-ter, fat levels, and much more. Thesevariables produce more than 22,000possible relationship pairs, whichwould take weeks to analyze using tra-
ditional methods. In this case, re-searchers set the controls on the rank-by-feature framework to produce atriangular matrix (see Figure 7) inwhich a few hundred bright redsquares show strong positive relation-ships.3 Only a few dozen blue squaresshow strong negative relationships(when one variable increases, the otherdecreases). Thus, with one compactoverview, researchers can narrow theirfocus and explore the strongest andmost surprising relationships, eitherpositive or negative.
Many relationships might be obvi-ous, like the strong positive correlationbetween height before and after exer-cise. Surprises are immediately visible,however, such as participants who dif-fered in height substantially before andafter—clearly a result of measurement
V I S U A L I Z A T I O N C O R N E R
Figure 3. The automobile data set, ordered by quadracity. The scores overview shows the highest quadratic relationshipfor horsepower (HP) vs. miles per gallon (MPG), which users select to produce the scatterplot on the right. In this case,MPG drops sharply as HP increases.
Figure 4. The automobile data set, ordered by number of outliers. The scores overview shows large numbers of outliersfor weight (WT) vs. volume (VOL) and for horsepower (HP) vs. weight (WT). The scatterplot on the right show weightvs. volume, with the six outliers highlighted by small triangles.
MARCH/APRIL 2006 51
errors. An interesting negative rela-tionship emerged in this study: thosewho started with weaker arms im-proved the most, whereas the strongera person’s arms were initially, the lessthey gained from exercise. The mostsignificant finding was an unknownstrong association between the AKT1gene and male body composition thataffects strength, bone size, and subcu-taneous fat, while protecting the meta-bolic syndrome.
Relationships between variables areonly one feature of 2D scatterplots.Users of the rank-by-feature frameworkcan set the controls to look for featuressuch as clusters, gaps, and outliers.
Information Visualization:New Telescopes for DataJust as the telescope let Galileo see
Jupiter’s moons, new visual analyticsoftware tools are enabling discoveriesin genomics, pollution control, and ter-ror-threat detection. The recent re-port, Illuminating the Path, celebratedthe importance of visual analytic ap-proaches4 and called for substantial in-creases in research, especially forhomeland security applications.
The emergence of information vi-sualization user interfaces that com-bine data mining and statisticalmethods is bringing novel methodsand potent tools to many analysts’desktops. Some improvements resultfrom faster processors, larger displays,and improved databases, but the ma-jor advances are coming from innova-tive user interface features and novelfiltering strategies. Designers of re-search prototypes and successful com-
mercial products are finding imagina-tive ways to apply the widely statedinformation-seeking mantra: overviewfirst, zoom and filter next, and thendetails-on-demand.
This simple strategy suggests thatusers should begin by seeing anoverview of all their data, even if thisvisual representation must aggregateand summarize millions or billions ofitems. This lets them see the range anddistribution of data items, immediatelydiscover surprising gaps, and identifymissing values or incorrect data items.Then they can zoom in on what theywant and filter out what they don’t, us-ing animation and user-controlledsliders. Finally, when they’ve focusedtheir attention on a few items, userscan click to get details-on-demand,send the result set via email, or save a
Figure 5. The automobile data set, ordered by normality. Shifting to the histogram ordering tab, the score overviewshows that the variable whose distribution is closest to being normal is speed (SP). The histogram on the right shows analmost symmetric normal distribution.
Figure 6. The automobile data set, ordered by size of biggest gap. The score overview shows that the biggest gap occursin the volume variable. The histogram on the right shows the biggest gap in a peach-colored rectangle. Several cars haveextremely small volumes of interior seating space, but this analysis reveals that an error exists in the data set.
52 COMPUTING IN SCIENCE & ENGINEERING
presentation to a Web page to invitecolleagues’ comments.
These advances, made during thepast decade, are available through re-search prototypes and a growingnumber of commercial tools. Multi-
dimensional information visualizationproducts include Spotfire (www.spotfire.com), Datadesk (www.datadesk.com), and SAS/GRAPH (www.sas.com). Research tools are also maturingrapidly, with toolkits such as GeoVista
from Penn State University and Polarisfrom Stanford University.
Although multidimensional data arewidely used, many scientific, business,demographic, and other data sets are hi-erarchically organized. For these datasets, tree-structured visualizations suchas treemaps are rapidly gaining accep-tance. They are prominent in Web-based applications such as Smartmoney’sMarketmap (www.smartmoney.com/marketmap/), which shows 600 stockscolor-coded to highlight rising (green)or falling (red) prices, with area denot-ing market capitalization—that is, big-ger companies have larger rectangles.Recently successful treemaps have in-cluded the US Marine Corps’ supply-chain management and Matrikon’sprocess-monitoring tools, both based onthe HiveGroup’s (www.hivegroup.com)software. Other treemap applications in-clude the SequoiaView free browser forPC hard drives (www.win.tue.nl/sequoiaview/) and Newsmap, whichgives an overview of the world’s news(www.marumushi.com/apps/newsmap/),with new products coming soon fromMicrosoft and Oracle. Scientificapplications, such as viewing gene ex-pression data according to the tree-structured gene ontology, are alsogaining prominence.5
N ot every data set lends itself toexploration by the rank-by-
feature framework, but this new ap-
V I S U A L I Z A T I O N C O R N E R
Figure 7. Correlation coefficient ranking. Missing values are excluded for the150 variables in a study of muscle size and strength. Red squares indicate highpositive correlations, whereas blue squares indicate negative correlations.
THE HEIRARCHICAL CLUSTERING EXPLORER
Y ou can access the Hierarchical Clustering Explorer (HCE 3.0), free technical papers, and an extensive user manual for ed-ucational and research use at www.cs.umd.edu/hcil/hce/. HCE provides powerful features for multidimensional data
analysis, including interactive exploration of hierarchical clustering results (with dendrograms and color-coded mosaics), andsupports the rank-by-feature framework. During an eight-week evaluation period, researchers we observed found the frame-work useful in looking at all variables’ distributions and test normality quickly and simultaneously, as well as for finding out-liers and unique values.1 HCE’s additional capabilities include parallel coordinates and tabular data viewers. HCE 3.0 is astandalone Microsoft Windows application written in C++ that runs on a general PC environment. We have successfully in-stalled it with Windows emulators on Macintosh computers.
Reference
1. J. Seo and B. Shneiderman, “Knowledge Discovery in High-Dimensional Data,” IEEE Trans. Visualization and Computer Graphics, vol. 12, no. 3, 2006.
MARCH/APRIL 2006 53
proach provides a rapid understandingof key features and patterns in manyhigh-dimensional data sets. Readerscan learn about the rank-by-featureframework and try our implementa-tion in the Hierarchical ClusteringExplorer (see the sidebar). We realizethat the concepts are difficult for someusers to follow, so our next step will beto simplify the interface and provideguidance to new users. Of course,those with deep knowledge of theirdomains will be best prepared to rec-ognize the importance of the featuresthey find in their data. Galileo’s dis-covery of Jupiter’s moons dependednot only on his making the telescope,
but also on his capacity to understandwhat he was seeing.
References1. J.W. Tukey and P.A. Tukey, “Computer
Graphics and Exploratory Data Analysis: AnIntroduction,” Proc. Ann. Conf. and Exposition:Computer Graphics, National Micrograhics As-soc., vol. 3, 1985, pp. 773–785.
2. J. Seo and B. Shneiderman, “Interactively Ex-ploring Hierarchical Clustering Results,” Com-puter, vol. 35, no. 7, 2002, pp. 80–86.
3. J. Seo and B. Shneiderman, “A Rank-by-Fea-ture Framework for Interactive Exploration ofMultidimensional Data,” Information Visual-ization, vol. 4, no. 2, 2005, pp. 1–16.
4. J. Thomas and K. Cook, eds., Illuminating thePath: The Research and Development Agendafor Visual Analytics, IEEE Press, 2005;http://nvac.pnl.gov/agenda.stm.
5. E.H. Baehrecke et al., “Visualization andAnalysis of Microarray and Gene OntologyData with Treemaps,” BMC Bioinformatics, vol.5, June 2004, p. 84; www.biomedcentral.com/1471-2105/5/84/.
Ben Shneiderman is a professor of com-
puter science at the University of Maryland
and founding director of its Human-
Computer Interaction Lab. His research in-
terests include information visualization and
human-computer interaction. Shneiderman
has a PhD in computer science from the
State University of New York, Stony Brook.
He is a senior member of the IEEE, a fellow of
the ACM, and a fellow of the American As-
sociation for the Advancement of Science.
Contact him at [email protected].
Mid Atlantic (product/recruitment)Dawn BeckerPhone: +1 732 772 0160Fax: +1 732 772 0164Email: [email protected]
New England (product)Jody EstabrookPhone: +1 978 244 0192Fax: +1 978 244 0103Email: [email protected]
New England (recruitment)Robert ZwickPhone: +1 212 419 7765Fax: +1 212 419 7570Email: [email protected]
Connecticut (product)Stan GreenfieldPhone: +1 203 938 2418Fax: +1 203 938 3211Email: [email protected]
Midwest (product)Dave JonesPhone: +1 708 442 5633Fax: +1 708 442 7620Email: [email protected] HamiltonPhone: +1 269 381 2156Fax: +1 269 381 2556Email: [email protected] DiNardoPhone: +1 440 248 2456Fax: +1 440 248 2594Email: [email protected]
Southeast (recruitment)Thomas M. FlynnPhone: +1 770 645 2944Fax: +1 770 993 4423Email: [email protected]
Southeast (product)Bill HollandPhone: +1 770 435 6549Fax: +1 770 435 0243Email: [email protected]
Midwest/Southwest (recruitment)Darcy GiovingoPhone: +1 847 498-4520Fax: +1 847 498-5911Email: [email protected]
Southwest (product)Josh MayerPhone: +1 972 423 5507Fax: +1 972 423 6858Email: [email protected]
Northwest (product)Peter D. ScottPhone: +1 415 421-7950Fax: +1 415 398-4156Email: [email protected]
Southern CA (product)Marshall RubinPhone: +1 818 888 2407Fax: +1 818 888 4907Email: [email protected]
Northwest/Southern CA (recruitment)Tim MattesonPhone: +1 310 836 4064Fax: +1 310 836 4067Email: [email protected]
JapanTim MattesonPhone: +1 310 836 4064Fax: +1 310 836 4067Email: [email protected]
Europe (product) Hilary TurnbullPhone: +44 1875 825700Fax: +44 1875 825701Email: [email protected]
A D V E R T I S E R / P R O D U C T I N D E X M A R / A P R 2 0 0 6
IEE Inspec Cover 4
University of Kentucky 5
Boldface denotes advertisements in this issue.
Advertising PersonnelAdvertiser Page Number
Marion DelaneyIEEE Media, Advertising DirectorPhone: +1 212 419 7766Fax: +1 212 419 7589Email: [email protected] AndersonAdvertising CoordinatorPhone: +1 714 821 8380Fax: +1 714 821 4010Email: [email protected]
Sandy BrownIEEE Computer Society,Business Development ManagerPhone: +1 714 821 8380Fax: +1 714 821 4010Email: [email protected]
Advertising Sales Representatives
