Augmented Reality maintenance modelling - Mer Augmented Reality: definition Augmented reality (AR) is

  • View
    3

  • Download
    0

Embed Size (px)

Text of Augmented Reality maintenance modelling - Mer Augmented Reality: definition Augmented reality (AR)...

  • Augmented Reality maintenance modelling

    mailto:vhavard@cesi.fr mailto:dorian.boulch@viacesi.fr mailto:dbaudry@cesi.fr

  • Mer Innovate final event in Technopole at Portsmouth, 18 March 2015

    Context and problematic

    • Wind turbine are off-shore • ~1 hour for accessing a wind farm.

    • Accessibility depends on weather conditions.

    => Little time to perform maintenance.

    • Expert can’t always be physically on site.

    • Expert may not be allowed to be part of the crew.

    2

    Need to

    improve communication

  • Mer Innovate final event in Technopole at Portsmouth, 18 March 2015

    Summary

    Definition of augmented reality

    Principles

    Display technologies

    Existing frameworks

    Augmented reality maintenance modelling

    Collaboration configuration

    Works done

    Next steps

    3

  • Mer Innovate final event in Technopole at Portsmouth, 18 March 2015

    Augmented Reality: definition

    Augmented reality (AR) is defined by (Azuma 1997) as: 1. Combines real and virtual 2. Interactive in real time 3. Registered in 3-D

    Augmentation: virtual object added to the real world.

    Real

    environment

    Virtual

    environment

    Augmented

    reality

    Augmented

    virtuality

    Reality-virtuality continuum

    Mixed reality

    4

  • Mer Innovate final event in Technopole at Portsmouth, 18 March 2015

    Augmented Reality: blower maintenance demonstrator

    Demo done with

    • Unity

    • Vuforia SDK

    5

  • Mer Innovate final event in Technopole at Portsmouth, 18 March 2015

    Augmented Reality : how it works?

    Principle for augmented reality vision-based tracking (Azuma 1997)

    • A camera gets the scene Iscene • Based on the scene, camera pose is computed.

    • When cam pose is known, the virtual object is added in the scene.

    Scene image

    Iscene

    Features extraction

    Iscene

    Features matching between Iref et Iscene

    Camera Pose computation

    Adding augmentations

  • Mer Innovate final event in Technopole at Portsmouth, 18 March 2015

    Augmented reality: display technologies

    9

    Display Technologies

    Optical see- through

    Optical Glance

    Video see through

    Hand held display

    Projective worn

    Projective fixed

    Contact lens

  • Mer Innovate final event in Technopole at Portsmouth, 18 March 2015

    Augmented reality: display technologies

    10

    Ideal configuration from user perspective

  • Mer Innovate final event in Technopole at Portsmouth, 18 March 2015

    Augmented reality: display technologies

    Tablet

    Intuitive interface

    Good resolution

    Not hand free

    11

  • Mer Innovate final event in Technopole at Portsmouth, 18 March 2015

    Augmented reality: display technologies

    12

    Optical Glance Google glass-like

    Hand free usage

    Secure

    Interface

    Small display

    Only glance

  • Mer Innovate final event in Technopole at Portsmouth, 18 March 2015

    Augmented reality: display technologies

    Optical see through

    Hand free usage

    Registered

    Security issue

    Small display

    13

  • Mer Innovate final event in Technopole at Portsmouth, 18 March 2015

    Augmented reality: display technologies

    Projective

    Natural vision

    Limited virtual objects

    Indoor usage

    14

  • Mer Innovate final event in Technopole at Portsmouth, 18 March 2015

    Augmented reality: display technologies

    15

    Display Technologies

    Optical see- through

    Optical Glance

    Video see through

    Hand held display

    Projective worn

    Projective fixed

    Contact lens

    Choice depends on the application

  • Mer Innovate final event in Technopole at Portsmouth, 18 March 2015 16

    How to choose the right display technology?

    Augmented reality: display technologies

    Device Type Characteristics

    Contac lens Optical see-

    through Optical Glance

    Video see through

    Hand held display

    Projective worn Projective fixed

    Mobile + + + + + Limited Limited

    Outdoor use + + + + + Limited Limited

    User interface Gesture

    or Tactile Gesture

    or Tactile Gesture

    or Tactile Gesture

    or Tactile Tactile

    Gesture or Tactile

    Gesture or Tactile

    Free hand usage + + + + + +

    Field Of View + Limited Limited Limited + + +

    Occlusion of real world by AR object

    Yes Yes Limited Yes Limited No No

    Unrestricted Augmentation model

    Yes Yes Yes Yes Yes Limited Limited

    Eyestrain Not determined Yes Yes Yes No No No

    Power usage Limited Limited Limited Limited Limited Limited Unlimited

    Available on the white paper by the end of March

  • Mer Innovate final event in Technopole at Portsmouth, 18 March 2015

    Augmented Reality: the most relevant frameworks

    17

    AR Framework Characteristics

    Metaio Vuforia Wikitude OpenCV

    Marker tracking Yes Yes Yes Dev

    Image Target tracking

    Yes Yes Yes Dev

    3D tracking Yes Limited box and cylinder

    No Dev

    Face Tracking Yes No No Dev

    Unity 3D Yes Yes Yes Need

    Integration

    iOS Yes Yes Yes Yes

    Android Yes Yes Yes Yes

    Windows Phone 8 No No No Yes

    PC Yes Yes Yes Yes

    Accelerometer and Gyroscope sensors

    Yes Yes Yes Dev

    Developer community

    Yes Yes Yes Yes

    Official Webinar Yes No No No

    Free Yes but

    Watermark Yes but

    Watermark Yes but

    Watermark Yes

    Open Source No No No Yes

  • Mer Innovate final event in Technopole at Portsmouth, 18 March 2015

    Expert – operator maintenance problematic

    How the expert can quickly create Augmented Reality content for the operator, without development skills?

    How can we accelerate AR Player development on current device and future ones?

    18

    Need to

    improve communication

  • Mer Innovate final event in Technopole at Portsmouth, 18 March 2015

    Augmented Reality content production modelling

    19

    AR Editor SolidWorks

    Authoring tool

    Plugin

    1

    3D model

    AR player Smart Glasses

    or tablet

    Standard

    XML

    file

    Export

    Ressources

    2 3

    Expert Opérateur

  • Mer Innovate final event in Technopole at Portsmouth, 18 March 2015

    Augmented Reality: maintenance modelling

    • An AR maintenance is represented by a list of Action performed on Entity of the system (For example: Unscrew the screw 1).

    • An Action is a simple Animation played on one of the Entity of the system to maintain. • The Action Unscrew means an animation starting from position, orientation

    (x,y,z), (rx, ry, rz) = (0,0,0) (0,0,0) => (0,0,-50) (0,0, -720°) in 800 msec.

    • An Action may be composed of Illustration = an Animation on an ExternalEntity.

    • An ExternalEntity is a virtual object that is not part of the system. • For example display a screwdriver to show which tool use for disassembling an element.

    20

  • Mer Innovate final event in Technopole at Portsmouth, 18 March 2015

    Augmented Reality: maintenance modelling

    • We can define template Action like Unscrew, Screw,… that are reusable for several maintenance operations.

    21

  • Mer Innovate final event in Technopole at Portsmouth, 18 March 2015

    Augmented reality: maintenance configurations

    • Guided maintenance 1. Expert prepares the maintenance tasks in AR.

    2. Before leaving, operator loads the task he must perform from the DB.

    3. Once on site, he performs the maintenance actions.

    22

    1

    2

    Operator

    Expert

    Remote

    On site

    3

  • Mer Innovate final event in Technopole at Portsmouth, 18 March 2015

    Augmented reality: maintenance configurations

    • Assisted maintenance (or supervised maintenance) 1. Operator shares his view with the expert and explain his issue.

    2. Expert augments the operator view in order to explain and assist him.

    23

    1

    2

    Operator Expert

    Remote On site

  • Mer Innovate final event in Technopole at Portsmouth, 18 March 2015

    Works done

    • State of the art of augmented reality.

    24

    • State of the art of Augmented Reality devices and frameworks • We chose Metaio as framework and Moverio BT-200 as a smart glasses.

  • Mer Innovate final event in Technopole at Portsmouth, 18 March 2015

    Works done

    • 2 demonstrato