AugmentedAugmented Reality Reality

Overview and Current SystemsOverview and Current Systems

Motivation: A Brief ScenarioMotivation: A Brief Scenario

Motivation: A Brief ScenarioMotivation: A Brief Scenario

The Problems:The Problems:

The surgeon can’t look at the patient and the screen The surgeon can’t look at the patient and the screen at the same time.at the same time.

Looking at the screen while working on the patient Looking at the screen while working on the patient isn‘t ergonomic.isn‘t ergonomic.

The screen is 2D whereas the patient‘s body is 3D. The screen is 2D whereas the patient‘s body is 3D.

Motivation: A Brief ScenarioMotivation: A Brief Scenario

The VisionThe Vision

The information should be provided where it is The information should be provided where it is needed. needed.

The user should be enabled to access and interact The user should be enabled to access and interact with the information ergonomically.with the information ergonomically.

The information should be in 3 dimensions like the The information should be in 3 dimensions like the application’s subject.application’s subject.

RoadmapRoadmap

Defining Augmented RealityDefining Augmented Reality

A Taxonomy of Reality and VirtualityA Taxonomy of Reality and Virtuality

TechnologiesTechnologies

Fields of ApplicationFields of Application

Current SystemsCurrent Systems

Defining Augmented RealityDefining Augmented Reality

Combining Real and VirtualCombining Real and Virtual

Interactive in Real TimeInteractive in Real Time

Registered in 3-DimensionsRegistered in 3-Dimensions

Defining Augmented Reality:Defining Augmented Reality:Combining Real and VirtualCombining Real and Virtual

Attention: Attention:

Combining does not Combining does not neccesarily mean neccesarily mean adding.adding.

Combining does not Combining does not only include visual only include visual information (Navi-information (Navi-Project).Project).

Defining Augmented Reality:Defining Augmented Reality: Interactive in Real Time Interactive in Real Time

„„Using AR technology, Using AR technology, users can interact users can interact with a combination of with a combination of real and virtual real and virtual objects in a natural objects in a natural way.“way.“

In „Confluence of Computer Vision and Interactive In „Confluence of Computer Vision and Interactive Graphics for Augmented Reality“Graphics for Augmented Reality“

Definining Augmented Reality:Definining Augmented Reality: Registered in 3-Dimensions Registered in 3-Dimensions

Lightning reacts to theposition of the tangible sun.

A Taxonomy A Taxonomy of Reality and Virtualityof Reality and Virtuality

The Reality-Virtuality ContinuumThe Reality-Virtuality Continuum

A Taxonomy of A Taxonomy of Reality and VirtualityReality and Virtuality

Centricity: Exocentric or Egocentric?Centricity: Exocentric or Egocentric?

                

A Taxonomy of A Taxonomy of Reality and VirtualityReality and Virtuality

Control-Display CongruenceControl-Display Congruence Question: How intuitive is the control?Question: How intuitive is the control? In AR: When I move the real object to the right hand In AR: When I move the real object to the right hand

side, in what direction does the virtual one go?side, in what direction does the virtual one go?

Conclusion: Conclusion: AR is situated near the real-role of the continuum.AR is situated near the real-role of the continuum. AR uses an egocentric viewpoint in most cases.AR uses an egocentric viewpoint in most cases. AR tries to maximize the control-display congruence.AR tries to maximize the control-display congruence.

TechnologiesTechnologies

User InterfacesUser Interfaces

VisualizationVisualization

TrackingTracking

AR-System

User

User Interface

Tra

cking

Virtual World

Real World

User InterfacesUser Interfaces

VisualVisual

AcousticAcoustic

Haptic/TangibleHaptic/Tangible

User Interfaces: VisualUser Interfaces: Visual

Gesture RecognitionGesture Recognition

Example:Example: MIT‘s KidsRoom: MIT‘s KidsRoom:

Temporal integrationTemporal integration

User Interfaces: AcusticUser Interfaces: Acustic

2 Parts2 Parts Speech SynthesisSpeech Synthesis Speech Speech RecognitionRecognition

Odyssey 2001‘s dream Odyssey 2001‘s dream has not come true, yet.has not come true, yet.

So, Acoustic UIs are So, Acoustic UIs are combined with other combined with other modalities.modalities.

Example:Example: SHEEPSHEEP

Insert!

User Interfaces: Haptic/TangibleUser Interfaces: Haptic/Tangible

Joystick, 3D-MouseJoystick, 3D-Mouse

Tangible ObjectsTangible Objects Examples: Examples:

CAR, SHEEP and the CAR, SHEEP and the Studierstube‘s PIPStudierstube‘s PIP

Touch glovesTouch gloves

Visualization: Ways of VisualizingVisualization: Ways of Visualizing

Visualization DevicesVisualization Devices HMDsHMDs PDAsPDAs ProjectionProjection MonitorsMonitors

3-Dimensionality3-Dimensionality Separeted displaysSepareted displays Polarized projectionPolarized projection Anaglyphic projectionAnaglyphic projection Time divisionTime division

Visualization:Visualization:Mixing Real and VirtualMixing Real and Virtual

Optical see throughOptical see throughVideo see throughVideo see through Chroma keying: Mixing with video hardwareChroma keying: Mixing with video hardware Mixing in the RendererMixing in the Renderer

What about occlusion?What about occlusion? Tracking all real objectsTracking all real objects Depthmaps construction via stereo imagesDepthmaps construction via stereo images Special Projection TechniquesSpecial Projection Techniques

Visualization: ChallengesVisualization: Challenges

ContrastContrast

BrightnessBrightness

ResolutionResolution

Field of ViewField of View

SafetySafety

Tracking: Ways of TrackingTracking: Ways of Tracking

Optical TrackingOptical Tracking

MarkersMarkers IR-reflecting spheresIR-reflecting spheres CardsCards

Types of optical Types of optical TrackingTracking Inside outInside out Outside inOutside in

Tracking: Ways of Tracking Tracking: Ways of Tracking

Time of FlightTime of Flight GPS, DGPS, GalileoGPS, DGPS, Galileo UltrasoundUltrasound

Magnetic Field Magnetic Field SensingSensing

Inertial TrackingInertial Tracking Gyro MetersGyro Meters AccelerometerAccelerometer

Special: Eye TrackingSpecial: Eye Tracking

Tracking: ChallengesTracking: Challenges

Static ErrorsStatic Errors Optical distortionOptical distortion Improper parameter Improper parameter

configurationconfiguration Mechanical Mechanical

misalignmentmisalignment

StrategiesStrategiesCarefullnessCarefullness

Algorithms for Algorithms for autocalibrationautocalibration

Dynamic ErrorsDynamic Errors Hardware related Hardware related

delaysdelays Software related delaysSoftware related delays

StrategiesStrategiesHigher processing speedHigher processing speed

Video stream matchingVideo stream matching

Rendering in advanceRendering in advance

Predicting the futurePredicting the future

Fields of ApplicationFields of Application

Annotating the EnvironmentAnnotating the Environment

Displaying Auxiliary Sensor InformationDisplaying Auxiliary Sensor Information

Visualizing Artificial DataVisualizing Artificial Data

Annotating the EnvironmentAnnotating the Environment

NavigationNavigation Columbia‘s Columbia‘s

„Touring Machine“„Touring Machine“

Police and MilitaryPolice and Military BARSBARS

„… the success of a military operation in an urbanized environment depends crucially on being able to provide navigation and coordination information to the individual marine level.“

The US-Army's Concepts Division of the Marine Corps Combat Development Command

Annotating the EnvironmentAnnotating the Environment

Repair, Maintenance Repair, Maintenance and Productionand Production

BMW: The „Intelligent BMW: The „Intelligent Welding Gun“Welding Gun“

Boeing: „Augmented Boeing: „Augmented wiring“wiring“

Displaying Auxiliary Sensor Displaying Auxiliary Sensor InformationInformation

MedicalMedical University of North Carolina: University of North Carolina:

Ultrasound images of a fetus.Ultrasound images of a fetus. The heARt-Project: Minimal The heARt-Project: Minimal

invasive heart surgery.invasive heart surgery. MIT: MRI and EEG to support MIT: MRI and EEG to support

neurosurgery. neurosurgery.

Successfully usedSuccessfully used

Visualizing Artificial DataVisualizing Artificial Data

DesignDesign Studierstube‘s Virtual ShowcaseStudierstube‘s Virtual Showcase

Visualizing Artificial DataVisualizing Artificial Data

Scientific Scientific Visualization Visualization

EducationEducation

Studierstube: 3D-Studierstube: 3D-Construct ApplicationConstruct Application

Current SystemsCurrent Systems

LibrariesLibraries ARToolKitARToolKit

FrameworksFrameworks StudierstubeStudierstube DWARFDWARF

Current SystemsCurrent SystemsDWARF a Sample FrameworkDWARF a Sample Framework

DWARF, a network of distributed DWARF, a network of distributed dynamically cooperating services.dynamically cooperating services.

DistributionDistribution HeterogenityHeterogenity ModularityModularity

Current SystemsCurrent SystemsDWARF a Sample FrameworkDWARF a Sample Framework

ServicesServices NeedsNeeds AbilitiesAbilities Service DescriptionsService Descriptions

MiddlewareMiddleware Service ManagerService Manager Visual FrontendVisual Frontend

Current SystemsCurrent SystemsDWARF a Sample FrameworkDWARF a Sample Framework

SubsystemsSubsystems TrackingTracking PresentationPresentation InteractionInteraction ContextContext World ModelWorld Model ApplicationApplication

The FutureThe Future

HMDs in the near future? HMDs in the near future?

AR in sports and televisionAR in sports and television

Большое спасибо за Ваше вниманиеБольшое спасибо за Ваше внимание..

Vielen Dank für Ihre Aufmerksamkeit!Vielen Dank für Ihre Aufmerksamkeit!

Thank you very much for paying attention.Thank you very much for paying attention.

DiscussionDiscussion

Questions?Questions?

ExcursusExcursus CAR, SHEEP and Co. as motion pictures.CAR, SHEEP and Co. as motion pictures. A close look at DWARF: The Source Code of A close look at DWARF: The Source Code of

a Service.a Service. What does it cost? A sample HMD and a What does it cost? A sample HMD and a

sample Tracking System.sample Tracking System.