ARTEC (ARTEC (ResearchResearch GroupGroup ))ARTEC (ARTEC (ResearchResearch GroupGroup ))ARTEC (ARTEC (ResearchResearch GroupGroup ))ARTEC (ARTEC (ResearchResearch GroupGroup ))Advanced Research & Technological Expansion Advanced Research & Technological Expansion

in Computer Graphicsin Computer Graphics

Dr. Pedro Morillo TenaInstituto de Robótica (IRTIC)

Universitat de València (UVEG)

Dr. Pedro Morillo TenaInstituto de Robótica (IRTIC)

Universitat de València (UVEG)

SummarySummary

1. Introduction1. Introduction2. Presentation of the research group: ARTEC 3. Human interaction metaphors4. 2D/3D Tracking5. Video -based interaction (AR)

A. Marker / Markerless approachA. Marker / Markerless approach

6. Sensing of real-word elements7. Conclusions

Research groups at IRTC (UVEG)Research groups at IRTC (UVEG)

••ARTECARTEC-- Advanced Research & Technological Expansion in ••ARTECARTEC-- Advanced Research & Technological Expansion in Computer Graphics

•AUTISMO- Autism and Learning Difficulties Group

••LISSITLISSIT-- Intelligent Transportation Systems (ITS)

•LSYM- Modeling and Simulation Laboratory•LSYM- Modeling and Simulation Laboratory

•GSIC- Group of Information and Communication Systems

•SITRA- Image Processing and Computer Vision

ARTECARTEC was funded in 1992 and has been oriented to the Res earch and was funded in 1992 and has been oriented to the Res earch and Development of 3D Real Time Systems including Virtu al/Augmented Reality Development of 3D Real Time Systems including Virtu al/Augmented Reality technologies and Immersive Visualization Systems. C urrently, the group is technologies and Immersive Visualization Systems. C urrently, the group is

IRTIC and the activities in VR an ARIRTIC and the activities in VR an AR

technologies and Immersive Visualization Systems. C urrently, the group is technologies and Immersive Visualization Systems. C urrently, the group is composed of roughly 35 fullcomposed of roughly 35 full--time researchers in th e following research areas:time researchers in the following research areas:

•••••••• Basic Research in 3D Interactive Graphics, Multimodal Inte raction, V/A R•••••••• Virtual Architecture and Historic Reconstruction•••••••• Driving Simulation•••••••• High-Quality Terrain Visualization•••••••• AR Systems for training and maintenance•••••••• Entertainment and Learning Systems (Edutainment)•••••••• Civil Simulation for Training and Educational purposes•••••••• Civil Simulation for Training and Educational purposes•••••••• VR Systems for Industrial environments•••••••• Custom devices for Immersive Visualization Systems (VISIONARC)•••••••• X-DOF Motion Platforms•••••••• Multiplatform Developments (Windows, OSX, and Linux)•••••••• From Cluster computing support to mobile devices….

Main human interaction metaphorsMain human interaction metaphors

Virtual RealityVirtual Reality

“Virtual Reality” involves the creation of a comput er“Virtual Reality” involves the creation of a comput er--generated generated world that a person can interact with in such a way that he or she world that a person can interact with in such a way that he or she

believes that the virtual world is realbelieves that the virtual world is real

BMW Virtual Reality Center (Munich, Alemania)

Augmented RealityAugmented Reality

The “Augmented Reality” generates a composite view for the user that is the The “Augmented Reality” generates a composite view for the user that is the combination of the real scene viewed by the user an d a virtual scene generated combination of the real scene viewed by the user an d a virtual scene generated

by the computer that augments the scene with additi onal informationby the computer that augments the scene with additi onal information

Project FTB: RA formación BMW (Bertolín)/Vodafone (Bonatel)

Basic input mechanisms for VR/AR SystemsBasic input mechanisms for VR/AR Systems

• Mechanic Tracking • Electromagnetic• Ultrasounds• Inertial systems• Gyroscopes• Localization (GPS ,etc)

• Mechanic Tracking • Electromagnetic• Ultrasounds• Inertial systems• Gyroscopes• Localization (GPS ,etc)• Optic/Infrared systems• Marking/Markerless computer vision systems• Sensing real elements (cabins, cockpits, levers, etc.)

• Optic/Infrared systems• Marking/Markerless computer vision systems• Sensing real elements (cabins, cockpits, levers, etc.)

ENSISAM: Simulation environment for maritime rescue(FOREM-Andalucía –DECOM)

Electromagnetic TrackingElectromagnetic Tracking

INMER: Social integration for people with autism

Electromagnetic TrackingElectromagnetic Tracking

Mirror Metaphor: Real-Time motion capture in a CAVE environment

3D Tracking using infrareds & accelerometers3D Tracking using infrareds & accelerometers

3D Tracking using infrareds & accelerometers3D Tracking using infrareds & accelerometersBIOMIMO: Motion analysis for ergonomic purposes using Augmented Reality techniques

3D Tracking using infrareds & accelerometers3D Tracking using infrareds & accelerometersBIOMIMO: Motion analysis for ergonomic purposes using Augmented Reality techniques

Virtual Training Systems: PREVISOR, assembly of elements and driving of machinery in construction site environments

3D Tracking using infrareds & accelerometers3D Tracking using infrareds & accelerometers

3D Tracking using infrareds & accelerometers3D Tracking using infrareds & accelerometersVirtual Training Systems: PREVISOR, assembly of elements and driving of machinery in construction site environments

3D Tracking using infrareds & accelerometers3D Tracking using infrareds & accelerometersVirtual Training Systems: ENSISAM II, collaborative training in processes for maritime rescue

3D Tracking using infrareds & accelerometers3D Tracking using infrareds & accelerometersVirtual Training Systems: ENSISAM II, collaborative training in processes for maritime rescue

3D Tracking using infrareds on screens3D Tracking using infrareds on screensVirtual Training Systems: processes for a safe manipulation of electrical panelboards

3D Tracking using infrareds on screens3D Tracking using infrareds on screensVirtual Training Systems: processes for a safe manipulation of electrical panelboards

3D Tracking using infrareds on screens3D Tracking using infrareds on screensVirtual Training Systems: Multitouch surface platform for general purposes

Video -based interaction. Using AR marksVideo -based interaction. Using AR marksBIOMIMO II: Evaluation of virtual prototypes and control systems

Control Control ModelModel

Data InputData InputMoCapMoCap

CriticalCritical HardwareHardware

Simulation of the IDEALSimulation of the IDEALstatestate((AugmentedAugmented RealityReality ))

Visualization of the Visualization of the REAL state REAL state

((Virtual RealityVirtual Reality ))

Video -based interaction. Using AR marksVideo -based interaction. Using AR marksSUPERVISAR, ANGEL & SICURASUPERVISAR, ANGEL & SICURA--TUN: TUN: Training in action protocols using RA technologies

FASYS:FASYS: Absolutely Safe and Healthy Factory. Security protocols using Augmented Reality

Video -based interaction. Using AR marksVideo -based interaction. Using AR marks

Testing the FASYS system (TRW-Pamplona)

Video -based interaction. Markerless systemsVideo -based interaction. Markerless systemsPictogramROOMPictogramROOM:: Using AR to enhance the communication skills in people with autism

Video -based interaction. Markerless systemsVideo -based interaction. Markerless systemsFASYSFASYS:: Absolutely Safe and Healthy Factory. Security protocols using Virtual Reality

Orange ARFS: Orange ARFS: PROMOTION OF CULTURAL AND HERITAGE TOURIMS: Augmenting the Sagrada Familia monument using mobile devices. Monitoring the evolution of the constructions works for the Sagrada Familia monument

Video -based interaction. Markerless systemsVideo -based interaction. Markerless systems

3D Model

Recognition Process

Combined Visualization:Real World +

3D Model

Video -based interaction. Markerless systemsVideo -based interaction. Markerless systemsOrange ARFS: Orange ARFS: PROMOTION OF CULTURAL AND HERITAGE TOURIMS: Augmenting the Sagrada Familia monument using mobile devices. Monitoring the evolution of the constructions works for the Sagrada Familia monument

Video -based interaction. Markerless systemsVideo -based interaction. Markerless systems3DOrangeWorkd:3DOrangeWorkd: 3D navigation using simulated accelerometers by real-time video analysis

Services offered trhough Intennet

Video -based interaction. Markerless systemsVideo -based interaction. Markerless systemsOrange IntelligentEye: Urban navigation with contextual help using real-time image recognition

Services offered trhough Intennet

INTELLIGENT EYE

Internet

- Search server- Database- Classifier

Mobile Client Mobile Client

(WebPages, videos, forms, etc.)

On-line information of the business establishment

Server

Driving Simulation Systems: SIRCA, TUTOR, PREVISIM & SIAUTOSIRCA, TUTOR, PREVISIM & SIAUTO

Interaction by sensing real-world objects Interaction by sensing real-world objects

Interaction by sensing real-world objects Interaction by sensing real-world objects PREVISIMPREVISIM:: A collaborative simulator for traffic risk prevention

Interaction by sensing real-world objects Interaction by sensing real-world objects Simulation systems oriented to increased the skills of crane operators: Stewart, Transtainer,..

Interaction by sensing real-world objects Interaction by sensing real-world objects ESTRIBORESTRIBOR: : Simulation system oriented to the operation of rescue boats Simulation system oriented to the operation of rescue boats

ARTEC (ARTEC (ResearchResearch GroupGroup ))ARTEC (ARTEC (ResearchResearch GroupGroup ))ARTEC (ARTEC (ResearchResearch GroupGroup ))ARTEC (ARTEC (ResearchResearch GroupGroup ))Advanced Research & Technological Expansion Advanced Research & Technological Expansion

in Computer Graphicsin Computer Graphics

Dr. Pedro Morillo TenaInstituto de Robótica (IRTIC)

Universitat de València (UVEG)

Dr. Pedro Morillo TenaInstituto de Robótica (IRTIC)

Universitat de València (UVEG)