8/12/2019 Visualization as a Science Discovery ToolIssues and Concerns

1/18

Visualization as a Science Discovery ToolIssues and Concerns

Kelly GaitherDirector of Visualization/Sr. Research Scientist

Texas Advanced Computing Center

September 8, 2011

8/12/2019 Visualization as a Science Discovery ToolIssues and Concerns

2/18

Issues and Concerns Maximizing Scientific Impact Managing Data at Scale Providing Resources at Scale Ensuring Broad Accessibility/Developing

Ubiquitous Tools

8/12/2019 Visualization as a Science Discovery ToolIssues and Concerns

3/18

Visualization Mantra

The purpose of computing is insight not numbers.

-- R. W. Hamming (1961)

The purpose of visualization is insight not pictures. -- Ben Schneiderman (2005)

8/12/2019 Visualization as a Science Discovery ToolIssues and Concerns

4/18

Maximizing Scientific Impact

Image: Greg P. Johnson, Romy Schneider, TACC Image: Adam Ku bach, Karla Vega, Clint Daws on

Image: Karla Vega, Shaol ie Hossain , Tho mas J .R., HughesGreg Abram , Cars ten Bu rstedde, Georg Stadler, Lucas C. Wilcox,James R. Mart in , Tobin Isaac , Tan Bui-Thanh,and Om ar Ghat tas

8/12/2019 Visualization as a Science Discovery ToolIssues and Concerns

5/18

Managing Data at Scale

Large-ScaleVisualization

Resource

HPC

System

DataArchive

Display

Remote SiteWide-AreaNetwork Local Site

Pixels

Mouse

8/12/2019 Visualization as a Science Discovery ToolIssues and Concerns

6/18

Where Does Technology Fit In?

We have always used technology to createvisualizations of what we see in our minds eye.

What changes over time is the technology we use

to do the visualization.

8/12/2019 Visualization as a Science Discovery ToolIssues and Concerns

7/18

LonghornFirst NSF XD Visualization Resource

256 Dell Dual Socket, Quad Core Intel NehalemNodes

240 with 48 GB shared memory/node (6 GB/core) 16 with 144 GB shared memory/node (18 GB/core) 73 GB Local Disk 2 Nvidia GPUs/Node (FX 5800 4GB RAM)

~13.5 TB aggregate memory QDR InfiniBand Interconnect Jobs launched through SGE

~6GB/s to scratch filesystem

~6GB/s to Ranger filesystem

256 Nodes, 2048 Cores, 512 GPUs, 14.5 TB Memory

Kelly Gaither (PI), Valerio Pascucci, Chuck Hansen,David Ebert, John Clyne (Co-PI), Hank Childs

8/12/2019 Visualization as a Science Discovery ToolIssues and Concerns

8/18

Hadoop on LonghornPI: Weijia Xu (UT LIFT Grant)

Local Storage Expansion 192 500GB 7.2k drives are installed on 48 R610

nodes on Longhorn (96 usable TB) 112 146GB 15k drives are installed on 16 R710

nodes on Longhorn. (16 usable TB) /hadoop file system Went in to production December 2010 6 early user projects (text mining and

information retrieval) 142k co re ho urs f rom Dec to Feb

8/12/2019 Visualization as a Science Discovery ToolIssues and Concerns

9/18

Longhorn Usage Modalities:

Remote/Interactive Visualization Highest priority jobs Remote/Interactive capabilities facilitated through VNC Run on 3 hour queue limit boundary

GPGPU jobs Run on a lower priority than the remote/interactive jobs Run on a 12 hour queue limit boundary

CPU jobs with higher memory requirements Run on lowest priority when neither remote/interactive nor GPGPU

jobs are waiting in the queue Run on a 12 hour queue limit boundary

8/12/2019 Visualization as a Science Discovery ToolIssues and Concerns

10/18

Longhorn Queue Structure

Example: qsub -q normal -P vis

8/12/2019 Visualization as a Science Discovery ToolIssues and Concerns

11/18

Software Available on Longhorn Programming APIs: OpenGL, vtk (Not natively parallel)

OpenGL

low level primitives, useful for programming at arelatively low level with respect to graphics VTK (Visualization Toolkit) open source software system for 3D

computer graphics, image processing, and visualization IDL

Visualization Turnkey Systems VisIt free open source parallel visualization and graphical

analysis tool ParaView free open source general purpose parallel visualization

system

VAPOR

free flow visualization package developed out of NCAR EnSight commercial turnkey parallel visualization package

targeted at CFD visualization Amira commercial turnkey visualization package targeted at

visualizing scanned medical data (CAT scan, MRI, etc..)

8/12/2019 Visualization as a Science Discovery ToolIssues and Concerns

12/18

Longhorn Visualization Portal portal.longhorn.tacc.utexas.edu

Developed to provide easy access toLonghorn and abstract away complexitiesinvolved with command line access

Leverages TeraGrid user portal codebaseand employs a fraction of TGUP developersto ensure continuity

Used for all in-person Longhorn training

8/12/2019 Visualization as a Science Discovery ToolIssues and Concerns

13/18

8/12/2019 Visualization as a Science Discovery ToolIssues and Concerns

14/18

Longhorn Visualization Portal portal.longhorn.tacc.utexas.edu

Specify type ofsession

Specify resolution of

vnc session Specify number of

nodes needed andthe wayness of thenodes

Provides graphic ofmachine load

8/12/2019 Visualization as a Science Discovery ToolIssues and Concerns

15/18

Longhorn Visualization Portal portal.longhorn.tacc.utexas.edu

Vnc session opensup in java enabledbrowser

Behaves as if userhad gotten a remotedesktop intoLonghorn.

8/12/2019 Visualization as a Science Discovery ToolIssues and Concerns

16/18

EnVisionGreg Johnson, Brandt Westing

Web-based visualizationsoftware that allowsresearchers to developinteractive visualizationsintuitively.

Currently integrated into theLonghorn VisualizationPortal but can runindependently.

Began collaborations withParaView team.

8/12/2019 Visualization as a Science Discovery ToolIssues and Concerns

17/18

Summary

Close collaborations with the sciencepartners are key

User support Minimize data transfers if possible

Data stays in single location Scale resources effectively based on use

cases Easy accessibility to and interaction with

technologies encourages diversecommunities participation

8/12/2019 Visualization as a Science Discovery ToolIssues and Concerns

18/18

Questions?

kelly@tacc.utexas.edu