XLAB Research 2003 – 2006Ljubljana / Slovenia "River Socha Project" Visualization of massive amount of data with Grid based engine Authors: Marjan Šterk,

XLAB Research 2003 – 2006 Ljubljana / Slovenia

"River Socha Project""River Socha Project"Visualization of massive amount of dataVisualization of massive amount of data

with Grid based enginewith Grid based engine

Authors: Marjan Šterk, Ivan Leben, Gregor Pipan

Organisation: XLAB ResearchE-mail: [email protected]

Cracow Grid Workshop 2006

mailto:[email protected]


XLAB Company, Ljubljana,Slovenia• Medical Imaging• Network & Communication• Distributed and grid

computing research

Main grid-related projects: • XtreemOS (FP6 IP)• DeDiSys (FP6 STREP)• Interactive visualization of flooding areas (national)• Computational Grid technologies for better resource

utilisation (national)

XLAB - ResearchXLAB - Research


IntroductionIntroduction

Three goals of the project:

– Interactive visualization of landscape based on very large data sets and photographic material,

– development of partitioning algorithms for optimal resource utilization,

– implementation on a grid, with an front-end implementation in OpenGL.


2401997.48 5066594.99 -1.312401992.00 5066586.59 -1.322401986.50 5066578.16 -1.332401980.88 5066569.52 -1.572401980.16 5066568.38 -2.062401979.39 5066567.23 -1.662401976.94 5066563.48 -1.462401964.22 5066544.02 -1.392401958.51 5066535.26 -1.512401956.88 5066532.75 -1.792401950.47 5066522.97 -1.432401941.48 5066509.20 -1.372401935.75 5066500.42 -1.612401933.96 5066497.65 -2.122401933.30 5066496.66 -1.652401929.95 5066491.55 -1.412401923.48 5066481.64 -1.372401917.27 5066472.14 -1.422401912.46 5066464.77 -1.612401910.84 5066462.26 -2.042401910.46 5066461.71 -1.742401907.44 5066457.12 -1.562401910.65 5066462.63 -2.202401911.97 5066464.70 -1.622401913.97 5066467.78 -1.40

2401997.48 5066594.99 -1.312401992.00 5066586.59 -1.322401986.50 5066578.16 -1.332401980.88 5066569.52 -1.572401980.16 5066568.38 -2.062401979.39 5066567.23 -1.662401976.94 5066563.48 -1.462401964.22 5066544.02 -1.392401958.51 5066535.26 -1.512401956.88 5066532.75 -1.792401950.47 5066522.97 -1.432401941.48 5066509.20 -1.372401935.75 5066500.42 -1.612401933.96 5066497.65 -2.122401933.30 5066496.66 -1.652401929.95 5066491.55 -1.412401923.48 5066481.64 -1.372401917.27 5066472.14 -1.422401912.46 5066464.77 -1.612401910.84 5066462.26 -2.042401910.46 5066461.71 -1.742401907.44 5066457.12 -1.562401910.65 5066462.63 -2.202401911.97 5066464.70 -1.622401913.97 5066467.78 -1.40

Available datasetsAvailable datasets

• Based on GIS– data does not provide high enough resolution

• Based on LiDAR (laser scan)– high volume– mesh resolution: ~ 10cm x 10cm– high memory and cpu requirements

• Obtained from a simulation– high volume– highly dynamic


DemoDemo

Demonstration of the LiDAR dataset visualization

Live demo


full mesh (half-edges)

subset of points

VisualisationVisualisation

Delaunay triangulationusing Fortune’s sweepline alg.

O(N log N)

full LiDAR data

(bag of points)selection

currentview

mesh simplificationusing edge decimation

O(N log N)

simplified mesh

(half-edges)parallelizable


Socha-Executer

simplified mesh

Socha-Executer

Socha-Executer

Visualisation (cont.)Visualisation (cont.)

full LiDAR data

(bag of points)

selectiondistribution of computation

currentview

OpenGL rendering


Visualisation distributionVisualisation distribution

• Data (mesh) depends on the selected view, and cannot be pre-computed

• Considerable workload and memory requirements can be distributed to Grid nodes

• Process can be pipelined

• Depending on algorithmfor combining (stitching)the sub-meshes, inter-nodecommunication may be required.

Front-endPartitioning agent

Socha-Executer

Socha-Executer

Socha-Executer


Domain decompositionDomain decomposition

Trivial (i.e. slicing)• Uneven load, • varies greatly with the selected

view,• not tunable for different CPU

speeds, network bandwidths etc.

Proposal 1: checkerboard• Even load, independent of view,• requires fine decomposition,• fine-grained internode

communication during stitching,• statically tunable.


Domain decomposition (cont.)Domain decomposition (cont.)

Proposal 2: view-based• Every node stores all

points,• dynamic scheduling,• even load, independent of

view,• decomposition can be

coarser,• tunable in real-time.


Planned resultsPlanned results

Planned test

• Load distribution test based on pre-recorded fly paths

• Comparison of the proposed decompositions

• Nodes used:- cluster-like,- heterogeneous, higher network latency.


Conclusions & Future workConclusions & Future work

High-performance Grid visualization engine• Mesh construction and simplification is distributed,• two domain decompositions proposed,• both allow even and tunable node load.

Further development• Grid engine:

– automatic adaptation to node speed and load,– further optimization of decomposition and

communication.• Visualization:

– textures,– meshless surface construction.


ContactContact

XLAB d.o.o.Teslova 30SI-1000 Ljubljana

Tel: 01 477 6777

http://www.xlab.sihttp://research.xlab.si

[email protected]

Documents

XLAB Research 2003 – 2006Ljubljana / Slovenia "River Socha Project" Visualization of massive amount of data with Grid based engine Authors: Marjan Šterk,