Anita Havele

Executive Director, Web3D Consortium

www.web3d.org

anita.havele@web3d.org

March 19, 2012

Extending X3D for Augmented Reality

Fifth AR Standards Group Meeting

• Overview

• AR WG Update

• ISO SC24 AR Update

• OGC 3DPIE Report

• Upcoming events

March 19, 2012 Extending X3D for AR

Open Standards www.web3d.org

• Portability

• Durability

• IP-independence

• ISO Approved

March 19, 2012 Extending X3D for AR

About the Web3D Consortium

We are an International non-profit, member-funded, industry group Our members span business, academia, government and the military We maintain a royalty-free, ISO standard “X3D” for web based 3D.

March 19, 2012 Extending X3D for AR

X3D - ISO Standard

X3D an event and runtime model build for the Web

March 19, 2012 Extending X3D for AR

: the Standard Scene graph

Scene graph for real-time interactive delivery of virtual environments over the web:

• Meshes, lights, materials, textures, shaders • Integrated video, audio • Animation • Interaction • Scripts & Behaviors

• Multiple encodings (ISO = XML, VRML-Classic, Binary) • Multiple Application Programming Interfaces (ISO = ECMA, Java)

• X3D Version 3.3 in draft mode includes Volume rendering, CAD and

Geospatial components.

Sensor

Event ROUTE

March 19, 2012 Extending X3D for AR

• VRML and X3D are important parts of other specifications • We extend our standards to meet the needs of new use groups

Mpeg 4

WebGL

Collada

OPenGL

Web3D Collaboration & Convergence

Web3DS

CityGML

KML

N-D Presentation State Volume data

XML

HTML5

AR

POI

March 19, 2012 Extending X3D for AR

Adoption

March 19, 2012 Extending X3D for AR

Motivation to extend X3D for AR/MR

VR AR

Continuum Real Objects

MR

Virtual Reality (VR) – Virtual objects/ data X3D is a well established application description language to express Virtual objects and their behaviors in 3D virtual environments

Augmented Reality (AR) – Virtual objects augmented by sensors X3D has partial functionality for AR (Sensor Nodes, Viewpoint node, Camera node) Mixed Reality (MR) – Continuum between VR and AR X3D currently lacks features needed for MR - Extend X3D accommodate “real” world objects and represent MR contents

Extend X3D capabilities to address the larger “2D + 3D” space for AR

Going beyond basic geo-location based AR Virtual Objects

March 19, 2012 Extending X3D for AR

• Extensible to existing frameworks - X3D (Scene graph)

Extend Virtual Environment to implement AR

- KML, OpenGIS, ARML, X3dom…

Location Representation

• General/Flexible – Different AR platforms (platform independence)

Mobile, Desktop and HMD

– Sensors and devices

Vision based, Marker based and Location based

Our Approach to extend X3D for AR

March 19, 2012 Extending X3D for AR

Real Environment

Augmented Reality

Virtual Reality

Virtual Environment

X3D existing

Nodes

New/Extended X3D

Nodes for AR/MR

Extending X3D for AR

Mixed Reality

March 19, 2012 Extending X3D for AR

Making X3D AR/MR Capable

AR Working Group started in June 2011 Goal: Extend X3D specification for AR and MR applications Co-chairs:

Gun Lee (HIT Lab NZ), Yvonne Jung (Fraunhofer IGD) Tasks

Requirement and Use cases – August, 2011 Comparison of X3D AR proposals - March, 2012 Development to merge proposals

http://www.web3d.org/x3d/wiki/index.php/X3D_and_Augmented_Reality

March 19, 2012 Extending X3D for AR

X3D-AR Requirements

Functional requirements • Supporting image (camera) sensors • Live video • Background and textures • Tracking camera motion • Camera calibration • Correct image synthesis

Non-functional requirements Simplicity and extensibility of physical objects to be placed within the 3D scene.

March 19, 2012 Extending X3D for AR

X3D-AR Use Cases

• Augmented Reality Applications Where live video stream is shown on the background and the 3D scene is shown as registered in the physical space of the live video stream.

• Augmented Virtuality (or virtual studio) Applications Where live video stream of physical objects can be placed within the 3D scene.

http://www.web3d.org/x3d/wiki/index.php/X3D_AR_Requirements_and_Use_cases

March 19, 2012 Extending X3D for AR

X3D-AR Comparison of Proposals

Trade off between simplicity Vs flexibility/extensibility • Korea Chapter proposes on higher level AR/MR features • Instant Reality has lower-level multi-purpose nodes

X3D AR proposals from Korea Chapter and Fraunhofer IGD

http://www.web3d.org/x3d/wiki/index.php/Comparison_of_X3D_AR_Proposals

Based on this comparison the AR WG is discussing how to merge the X3D

AR proposals without compromising the AR requirements and use cases.

March 19, 2012 Extending X3D for AR

ISO Activities on AR Standardization JTC 1 Approval of SC24 Scope Reorganization

• Widen the scope and not limit it to computer graphics,

• Include various forms of media in the spectrum of mixed

reality

• Virtual Reality

• Augmented Reality

• Augmented Virtuality

Hence the term “Augmented Reality Continuum (ARC)”

March 19, 2012 Extending X3D for AR

ISO Activities on AR Standardization

SC24 Goals on ARC

WG9 Work Items

• ARC Concepts and Reference Model

• Terms and Definitions

• Architecture Model for Generic ARC System

Independent of implementation

• Proper ARC content representation

ARC content representation Specific file format

March 19, 2012 Extending X3D for AR

ISO Activities on AR Standardization

First WG9 Meeting was held in Seoul, Korea, Nov. 2011

Defined work items for WG9

• Standard ARC Reference Architecture

• Terms and Definitions for ARC

These work items (for SC24) will be submitted for approval at the next

SC24 Plenary Meeting (Brussels, Aug 2012)

Working Group 9 homepage: http://dxp.korea.ac.kr/SC24-WG

Basic information and news

Bulletin board for on-line discussion about the work items

Hope to coordinate above activities with the ARS community

March 19, 2012 Extending X3D for AR

• X3D aligns with 3DPIE visualization goals

• 3DPIE report demonstrates X3D Value

• Web3D Member Contributions • Virginia Tech – 3D Blacksburg Project • Bitmanagement – BS Contact Browser • Fraunhofer – Instant Reality Browser • MBARI

3DPIE and X3d

March 21, 2011 Extending X3D for AR

Next Steps To Align X3D with OGC

• Align with HTML5 approach using GeoLocation APIs

• Consider diverse location inputs to X3D scenes

• Consider X3D geospatial portability on mobile devices

• ARML WG collaboration in comparing AR proposals

• Write proposals and papers to gain external sponsor funding

Propose new X3D work to further OGC

3D portrayal Requirements

March 21, 2011 Extending X3D for AR

Web3D Conference 2012

• Co-sponsored by Web3D Consortium, ACM

SIGGRAPH and Eurographics

• 4th and 5th of August 2012 in Los Angeles.co-

Located with SIGGRAPH 2012

• Highlights current Capabilities and trends in

interactive 3D graphics. See CFP – Deadline April 3rd

• Several Workshops ( AR Workshop) and Tutorials

• Looking forward to participation from the AR Community

March 21, 2011 Extending X3D for AR

www.web3d2012.org

Visit us at: www.web3d.org

To Join: www.web3d.org/join

Executive Director: Anita Havele

Email: anita.havele@web3d.org

Web3D Consortium

Castro Street, Suite #120-490

Mountain View, CA 94041, USA

Phone: +1 248 342 7662

Fax: +1 248 457 8018