Visualization III-Interactive 3D GraphicsNicholas F. Polys Ph.D.VT Research Computing

GoalsUnderstand Networked Interactive 3D standardsExperience authoring environments (Flux Studio)Consider the impacts of immersion for computational science

Spatial Domain RepriseWhat is the difference and when would you use one or the other?Finite DifferenceFinite Element

Spatial DomainsWhere the simulation happens (e.g. FEM)Meshing the simulation spaceGranularityRuntimeExample: Delaunay Triangulationhttp://www.cse.unsw.edu.au/~lambert/java/3d/delaunay.html

ExampleSalome http://www.salome-platform.org/home/presentation/overview/

Visualization OverviewRaw data

Tables

Visual structures

Views

UserData transforms | Visual attribute | View transforms | Rendering assignmentFigure 2.1: Processing in a typical visualization pipeline (from Card et al, 1999)

Usability: Information VisualizationVisual display of abstract informationVisual Markers(Cleveland & McGill, 1984; Mackinlay, 1986; Card et al, 1999)Multiple Views (North, 2001; North et al, 2002; Convertino et al, 2003)Zoom-able Interfaces (Bederson et al, 1996, 2000; Woodruff et al, 1998a-c)

Once upon a time

Usability: Virtual EnvironmentsVisual display of spatial and perceptual dataImmersive and Desktop PlatformsConceptual Learning (Salzman et al, 1999)Navigating space (Darken et al, 1996, 2002)Naturalism & Performance (Bowman, 2002; Bowman et al, 2004)Image Plane (Pierce et al, 1997)

State of the ArtSuccessive layers of abstraction allow developers to design and build at higher levelsShaders and new rendering algorithms improve realism & performanceConcrete benefits of large format, high-res, and immersive displays

Graphics EnginesGames e.g. Unreal, Delta3D, Consumer solutionse.g. Cortona, Bitmanagement, Octaga, FreeWrl, Xj3D, Flux, H3D Multi-User spacese.g. ActiveWorlds, Blaxxun,BeThere, SecondLife, Industrial grade tookitse.g. DIVERSE, Paraview & VTK, Amira

Services & ServersIntegrated databases Interoperable file formatsReferenced resources across the webVisualization middleware servicesMulti-user & persistent worlds

Proprietary vs. OpensourceWho owns the data?Who owns the tools to access that data?How are bugs/new features accomplished?How much does it cost?

Games, Google, Second Life vs. open standards

Data FormatsVRML & X3D: expressive data and runtime behaviorInteroperation with WWWImport and export from many commercial and free toolsA capable common denominator

Open Standardswww.web3d.orgPortabilityDurabilityIP independenceInternational recognition and support

FoundationsISO standard, openly publishedMultiple implementations including open source codebasesX3D includes Transformation graph and behavior graph

Operating SystemOpen GL, etcVRML, X3DApplication

Source of Specs, Models, Links, Bulleting boards, Blogs, Mailing lists,

http://www.web3d.org

Viewers, Browsers, PlayersPlug-ins vs. Stand Alone

Deep Media Capabilities Real-time 3D scene graph Meshes, lights, materials, textures, shaders Integrated video, audio Animation Interaction Behaviors Scripts Application programming interfaces (API)

Source of Specs, Models, Links, Bulleting boards, Blogs, Mailing lists,

http://www.web3d.org

Web-enabledInternet and Local ResourcesMIME TypesFor VRMLx-world/x-vrml.wrlContent-encoding: x-gzip.gz

For X3DAddType model/x3d+xml .x3dAddType model/x3d+vrml .x3dvAddType model/x3d+binary .x3dbAddEncoding gzip .x3dvzAddEncoding gzip .x3dbz

Anchor, Inline

Application ExamplesPathsimCarilionDARPAMolecular Dynamics

Authoring Web3DFlux Studio (mediamachines.com, )White Dune (mac)Med text editor, X3D-Edit, VRMLPadViewers (many, see list at end).

Building VEs Navigation - many control options - walk, fly, examine or none (engines may also support proprietary modes) Viewpoints - pre-defined camera positions

VRML & X3D Scenegraph BasicsHierarchy of nodes (transformation graph)Nodes / Elements

Events and attribute Data types (behavior graph)

DAG!(Directed Acyclic Graph)Important nodes - (Grouping in parent/children hierarchy):Transform { }Group { }Anchor { }Switch { }LOD { }Billboard { }

Transform Node:translation (x y z coordinates, +Y is up, +Z is toward the user)rotation around an axis (x y z theta; language in rads, Flux in deg)scale (factor in x y z )children (other nodes)Boundingbox (helps rendering optimizations)

Building VEsModels - primitives (box, sphere, cone, cylinder), extrusions, indexed face set (mesh), line set, point set, elevation grid and text (engines may also support proprietary spline and NURBS geometry)Materials - diffuse colour, specular, emissive, ambient, shininess, transparency, colour per vertex

ShapesContain:geometryappearance

geometryPrimitivesBox, Cone, Cylinder, SphereExtrusionElevationGridIndexed Face and Line setsNURBS

Indexed FacesCoordinate {} coordIndex []creaseAngle : shading across polygons edges of the meshNormal [] s (for shape-dependent lighting control) solidcolorPerVertex []

Shapes & AppearancesAppearance {} and Materials {} :Color:specular, emissive, and diffuse Colors in RGB, shininesstransparencyambientIntensity

RGB Material {}diffuseColor 0.678, 0.169, 0.07 specularColorshininess

Building VEsTextures - support for JPEG, GIF, PNG and MPEG1 video. (engines may also have proprietary support for Flash, RealMedia, AVI, multi-texturing and environment mapping)Lighting - directional, point or spotlightEnvironments - background, and fog, hyperlinks (anchor), inlines

Building VEsAudio - fully spatialized 3D audio in WAVE or MIDI formatTextures - support for JPEG, GIF, PNG and MPEG1 video. (engines may also have proprietary support for Flash, RealMedia, AVI, multi-texturing and environment mapping)Lighting - directional, point or spotlightEnvironments - background, switches, hyperlinks (anchor), billboards and fog

Auditory PerceptionSound {}AudioClip {}MovieTexture {}

pitchintensitySpatialized Audio (doppler effect) Standard formats: .wav, .midi, .mp3, mpeg-1

TexturesImageTexture {} with (or without) alpha channels can be applied and mapped to geometry as fixed or animated maps. Standard formats: .png, .jpg, MovieTexture {} TextureTransform {} PixelTexture {}

LightingLighting Nodes: on, intensity, ambientIntensity, color Pointlight {attenuation}DirectionalLight {}Spotlight {direction, beamWidth, cutOffAngle}AMD 1: SFBool global

LightingLights have color!Directional LightsScoped by scenegraphsibling rule

webfolder/vrml/lightingbasis.wrl

Building VEsPerformance - LODs (levels of detail), visibility distance culling Animation - animate position, rotation, scale, points, colour and much more. Scope for many separate animations in one world all with different time lines and triggered by different events. Almost every attribute can be animated.Sensors - sense user activity such as touch, drag, keypress (plane, cylinder, sphere, and key sensors). Environmental sensors include time, proximity, and visibility.

Time & InteractivityKeyframed animationFunctional animationEvents are ROUTEd between nodes

InterpolatorsPositionOrientationScaleOthers:ColorsCoordinates

SensorsTouchSensorDragsensorsPlaneCylinderSphere ProximitySensorVisibilitySensor

Building VEsScripting - Interfaces directly with ECMAScript; also with Java, the web browser (DOM) and any programming language residing on the client/serverRoutes - scripts, animations and object properties can be "wired" together in an infinite number of ways to create any effect

ScriptingClient sideECMA ScriptLoosely typedNon-compiledBasic objects such as Math, Date, BrowserJavaFully typed CompiledIndustrial-strength classes

ImmersionTechnology that helps you be there to be presentStereo renderingView surroundHead and device tracking

VR Devices

Viewers

VRML / X3D Authoring Tools

Authoring / translation