View
31
Download
6
Category
Tags:
Preview:
DESCRIPTION
VAPOR Visualization and Analysis Platform for Ocean, atmosphere, and solar Research SC06 Ultra-Scale Visualization Workshop. John Clyne & Alan Norton Scientific Computing Division National Center for Atmospheric Research Boulder, CO USA. - PowerPoint PPT Presentation
Citation preview
Ultra-Scale Visualization WorkshopNovember 13, 2006clyne@ncar.ucar.edu
Supercomputing • Communications • Data
NCAR Scientific Computing Division
VAPORVisualization and Analysis Platform for Ocean, atmosphere, and solar Research
SC06 Ultra-Scale Visualization Workshop
John Clyne & Alan Norton
Scientific Computing Division
National Center for Atmospheric Research
Boulder, CO USA
This work is funded in part through a U.S. National Science Foundation, Information Technology Research program grant
Ultra-Scale Visualization WorkshopNovember 13, 2006clyne@ncar.ucar.edu
Supercomputing • Communications • Data
NCAR Scientific Computing Division
[Numerical] models that can currently be run on typical supercomputing platforms produce data in amounts that make storage expensive, movement cumbersome, visualization difficult, and detailed analysis impossible. The result is a significantly reduced scientific return from the nation's largest computational efforts.
Mark RastUniversity of Colorado, LASP
Problem
• Numerical simulations in the earth sciences have reached such extraordinary sizes that researchers can no longer effectively extract insight from their simulation outputs.
• Result: loss of scientific productivity!!!
Ultra-Scale Visualization WorkshopNovember 13, 2006clyne@ncar.ucar.edu
Supercomputing • Communications • Data
NCAR Scientific Computing Division
Dichotomy of simulation and analysis needs and resources in today’s HPC environments
Simulation Analysis
Large systems (O(1000) processors)
Small systems (O(10) processors)
Batch computing model Interactive computing model
Modest on-line storage requirements
Large on-line storage requirements
CPU/interconnect bound IO bound
Highly tuned, custom, parallel codes
COTS serial software applications
Historical focus of centers Emerging focus of centers
Ultra-Scale Visualization WorkshopNovember 13, 2006clyne@ncar.ucar.edu
Supercomputing • Communications • Data
NCAR Scientific Computing Division
A sampling of various technology performance curves
Performance gains from 1980 to present
1
10
100
1000
10000
100000
1980 1984 1988 1992 1996 2000 2004
Improvement
Disk Drive Internal DataRate
Disk Drive InterfaceData Rate
Ethernet NetworkBandwidth
Intel MicroprocessorClock Speed
Drive Capacity
• Not all technologies advance at same rate• Impact of parallelization not shown
Ultra-Scale Visualization WorkshopNovember 13, 2006clyne@ncar.ucar.edu
Supercomputing • Communications • Data
NCAR Scientific Computing Division
Communication limits for volume rendering assuming theoretical peak performance
2563 5123 10243 20483
GFX Memory
76 GB/sec(GeForce 7900 GX2)
4551 568 71 9
CPU Memory
10 GB/sec(AMD Opteron 1000 series)
599 75 9 1
PCI Express 16x
4 GB/sec
240 30 4 0.5
SATA 3.0
0.3 GB/sec
18 2 0.3 0.04
• Table shows limits expressed as frames per second imposed by communication alone
• Assumes only 8-bit data quantities
Ultra-Scale Visualization WorkshopNovember 13, 2006clyne@ncar.ucar.edu
Supercomputing • Communications • Data
NCAR Scientific Computing Division
Visualization and Analysis Platform for Ocean, atmosphere, and solar Research (VAPOR)
Key components1. Domain specific application focus: simulated earth sciences fluid flow2. Coupled Visualization and quantitative data interrogation and
manipulation capabilities 3. Multiresolution enabled terascale data exploration on the desktop
Combination of visualization with multiresolution data representation that provide sufficient data reduction to enable interactive work on terascale data from a desktop
Visual data browsing
Datamanipulation
Quantitativeanalysis
Refine
Coarsen
Ultra-Scale Visualization WorkshopNovember 13, 2006clyne@ncar.ucar.edu
Supercomputing • Communications • Data
NCAR Scientific Computing Division
Fluid flow in the geosciences
• E.g. Numerically simulated turbulence– Cartesian grids (usually)
• 5123 to 10243
• Up to 40963 “hero” calculations
– 5 to 8 variables• Temperature & Pressure
• Velocity field components
• Magnetic field components (MHD calculations)
– Hundreds of time steps saved• Terabytes of data per experiment
– Numerical “experiments”• Substantial analysis requirements
Yannick Ponty, CNRS 2006
Ultra-Scale Visualization WorkshopNovember 13, 2006clyne@ncar.ucar.edu
Supercomputing • Communications • Data
NCAR Scientific Computing Division
Key Component (1) : Domain specific supportOnly limited support for:
– Grid & data types• Cartesian grids, stretched and uniform sampling• AMR grids• Scalar and vector quantities
– Visualization algorithms• Volume rendering, flow visualization, cutting
planes/probe
– Misc.• Publication quality graphics• Filters• File formats (one!)
• Extensive support for:– Time varying data
• Uniform as well as non-uniform sampling• Missing time steps
– Quantitative investigation• Mathematical operators and data manipulators
– Science driven specialized features
Keep it simple!Keep it focused!
Make it scientist friendly!
Ultra-Scale Visualization WorkshopNovember 13, 2006clyne@ncar.ucar.edu
Supercomputing • Communications • Data
NCAR Scientific Computing Division
Interactive exploration of time varying data
• Reduce bandwidth requirements– Regions of interest– Multiresolution– Caching
2563 5123 10243 20483
GFX Memory
76 GB/sec
(GeForce 7900 GX2)
4551 568 71 9
CPU Memory
10 GB/sec
(AMD Opteron 1000 series)
599 75 9 1
PCI Express 16x
4 GB/sec
240 30 4 0.5
SATA 3.0
0.3 GB/sec
18 2 0.3 0.04
Ultra-Scale Visualization WorkshopNovember 13, 2006clyne@ncar.ucar.edu
Supercomputing • Communications • Data
NCAR Scientific Computing Division
Future???VAPORInteractive visual browsing
IDLData manipulation &
analysis
VAPOR Data Collection
Disk Array
Multi-resolution access and rapid sub-region extraction
Key Component (2) : Coupled visualization, quantitative analysis and manipulation capabilities
• IDL - array based 4GL for scientific data processing– Thousands of mathematical functions
– Basic 2D plotting
– Array manipulation
Ultra-Scale Visualization WorkshopNovember 13, 2006clyne@ncar.ucar.edu
Supercomputing • Communications • Data
NCAR Scientific Computing Division
Key component (3) : Multiresolution data access
• Wavelet transformed data– Two parameter linear function
decomposition
– Hierarchical data representation
– Invertible and lossless
– Numerically efficient (O(n))
• forward and inverse transform
– No additional storage cost
• Enable speed/quality tradeoffs
504x504x2048
Full
252x252x1024
1/8
126x126x512
1/64
63x63x256
1/512
€
f t( ) = a j ,k
j
∑k
∑ ψ j ,k t( )
Ultra-Scale Visualization WorkshopNovember 13, 2006clyne@ncar.ucar.edu
Supercomputing • Communications • Data
NCAR Scientific Computing Division
Visual comparison of a 5123 compressible convection simulation
1283
coarsened
M. Rast, 20025123
native
Ultra-Scale Visualization WorkshopNovember 13, 2006clyne@ncar.ucar.edu
Supercomputing • Communications • Data
NCAR Scientific Computing Division
Performance of forward and inverse Haar wavelet transform
Inverse Data Transform
0
10
20
30
40
50
60
70
80
90
100
128 3̂ 256 3̂ 512 3̂ 1024 3̂
Resolution
Time (seconds)
Read
Transform
Forward Data Transformation
0
10
20
30
40
50
60
70
80
90
100
128 3̂ 256 3̂ 512 3̂ 1024 3̂
Resolution
Time (seconds)
Write
Transform
System
• Linux RHEL 3.0
• 2 x Intel 3.4 GHz Xeon EMT64
• 8 GBs RAM
• 1Gb/sec Fibre Channel storage
Data
• Scalar
• Single precision
Gains in microprocessor technology enable transforms at very low cost
Ultra-Scale Visualization WorkshopNovember 13, 2006clyne@ncar.ucar.edu
Supercomputing • Communications • Data
NCAR Scientific Computing Division
VAPOR Demo
Ultra-Scale Visualization WorkshopNovember 13, 2006clyne@ncar.ucar.edu
Supercomputing • Communications • Data
NCAR Scientific Computing Division
Summary
• VAPOR is a domain-specific platform for analysis, not a general purpose visualization tool
• Target users: fluid flow researchers in earth sciences– Limited value for medical, oil & gas, aerospace, etc.
• Desktop data exploration of terabyte data possible– Visualization enables rapid ROI identification
– Multiresolution enables speed/quality tradeoffs
Ultra-Scale Visualization WorkshopNovember 13, 2006clyne@ncar.ucar.edu
Supercomputing • Communications • Data
NCAR Scientific Computing Division
Acknowledgements
• Steering Committee– Nic Brummell - CU
– Yuhong Fan - NCAR, HAO
– Aimé Fournier – NCAR, IMAGe
– Pablo Mininni, NCAR, IMAGe
– Aake Nordlund, University of Copenhagen
– Helene Politano - Observatoire de la Cote d'Azur
– Yannick Ponty - Observatoire de la Cote d'Azur
– Annick Pouquet - NCAR, ESSL
– Mark Rast - CU
– Duane Rosenberg - NCAR, IMAGe
– Matthias Rempel - NCAR, HAO
– Geoff Vasil, CU
• Developers– Alan Norton – NCAR, SCD
– John Clyne – NCAR, SCD
– Kenny Gruchalla - CU
• Research Collaborators– Kwan-Liu Ma, U.C. Davis
– Hiroshi Akiba, U.C. Davis
– Han-Wei Shen, Ohio State
– Liya Li, Ohio State
• Systems Support– Joey Mendoza, NCAR, SCD
Ultra-Scale Visualization WorkshopNovember 13, 2006clyne@ncar.ucar.edu
Supercomputing • Communications • Data
NCAR Scientific Computing Division
Questions???
www.vapor.ucar.edu
Ultra-Scale Visualization WorkshopNovember 13, 2006clyne@ncar.ucar.edu
Supercomputing • Communications • Data
NCAR Scientific Computing Division
Inverse Haar transform with 1/8th volume subregion extraction
System
• Linux RHEL 3.0
• 2 x Intel 3.4 GHz Xeon EM64
• 8 GBs RAM
• 1Gb/sec Fibre Channel storage
Data
• Scalar
• Single precision
Inverse Data Transform with Subregion Extraction
0
2
4
6
8
10
12
14
128 3̂ 256 3̂ 512 3̂ 1024 3̂
Resolution
Time (seconds)
Read
Transform
Data blocking permits rapid subregion extraction
Ultra-Scale Visualization WorkshopNovember 13, 2006clyne@ncar.ucar.edu
Supercomputing • Communications • Data
NCAR Scientific Computing Division
The Lifting Method of wavelet construction in the spatial domain[Sweldens, 95]
{ } Z∈<≤≡ kjk jkjj ,for 20|,λλ
kkjkj ∀= +− 12,,1 λγ
( ) kkjkjkj P ∀−= −+− ,112,,1 λλγ
kjkj 2,,1 λλ =−
( ) kkjkjkj U ∀+= −−− ,1,1,1 γλλ
kj ,λ SplitPredict
Update
kj ,1−γ
kj ,1−λkj ,2−γ
kj ,2−λk,0γ
k,0λk,1λTransform
1
Transform 2
Transform j
1) Split:
2) Predict:
3) Update:
Split signal into even (λ) and odd (γ) coefficients. λ will contain low frequency information, γ will contain high frequency information.
Local correlation permits prediction of odd samples by even using a prediction operator, P. Capture difference between prediction and actual coefficient value.
Update λ coefficients to preserve a property (e.g. mean) of original signal.
A signal λj consisting of 2j samples
Ultra-Scale Visualization WorkshopNovember 13, 2006clyne@ncar.ucar.edu
Supercomputing • Communications • Data
NCAR Scientific Computing Division
Example: Lifting Method with the Haar Wavelet
1 7 3 1 6 0 9 5 ( )kkk P ,212,3,2 λλγ −= +
4 2 3 7
3 5
λ3,k
λ2,k
λ1,k
λ0,k
6 -2 -6 -4γ2,k
γ0,k
γ1,k
Haaroperators xxU
xxP
2
1)(
)(
=
=
( )kkk U ,2,2,2 γλλ +=
-2 4
4 2
Ultra-Scale Visualization WorkshopNovember 13, 2006clyne@ncar.ucar.edu
Supercomputing • Communications • Data
NCAR Scientific Computing Division
NCAR Historical Estimated Sustained GFLOPS (Batch Production Systems)
0
100
200
300
400
500
600
700
800
900
1000
Jan-97 Jan-98 Jan-99 Jan-00 Jan-01 Jan-02 Jan-03 Jan-04 Jan-05 Jan-06
IBM p5-575/HPS(bluevista)
IBM Opteron/Linux(lightning)
IBM POWER4/Federation(thunder)
IBM POWER4/Colony(bluesky)
IBM POWER4 (bluedawn)
SGI Origin3800/128
IBM POWER3(blackforest)
IBM POWER3 (babyblue)
Compaq ES40/32(prospect)
SGI Origin2000/128 (ute)
HP SPP-2000/64 (sioux)
CRI Cray C90/16 (antero)
CRI Cray J90 series
ARCS Phase 4
Cray C90/16
HP SPP2000
SGI Origin2000
blackforest (WH-1)
SGI Origin3800
lightning
bluesky
blackforest
ARCS Phase 3
ARCS Phase 2
ARCS Phase 1
Linux
blackforest (WH-2/NH-2)
bluevista
Ultra-Scale Visualization WorkshopNovember 13, 2006clyne@ncar.ucar.edu
Supercomputing • Communications • Data
NCAR Scientific Computing Division
NCAR Historical Estimated Sustained GFLOPS (Interactive Production Systems)
• Current NCAR visualization and analysis resources– ~32 processors
• 8 nodes (6 with gfx)
– ~100 TB on-line storage
– ~800 MBs/sec aggregate storage bandwidth
– ~100 users (99 of which will not leave office)
Recommended