Augmented and Mixed Reality - oamk.fi “AR4BC” Augmented Reality for Building and Construction ... Augmented Reality + Augmented Virtuality ... CATIA - STEP - Reduced B-rep - Design

  • View
    216

  • Download
    4

Embed Size (px)

Text of Augmented and Mixed Reality - oamk.fi “AR4BC” Augmented Reality for Building and Construction...

  • 1

    Augmented and Mixed Reality

    Applications at VTT

    Charles Woodward

    April 16, 2010

    VTT TECHNICAL RESEARCH CENTRE OF FINLAND

    Mobile Augmented Reality

    one of the technologies to

    most likely to alter industries, fields of research,and the way we live.

    - MITs annual review 10 Emerging Technologies 2007

    - Gartners Top Ten Disruptive Technologies 2008-2012

    - First consumer applications on camera phones 2009

    - Commercial breakthrough predicted 2010

  • 2

    VTT TECHNICAL RESEARCH CENTRE OF FINLAND

    INTRODUCTION

    VTT TECHNICAL RESEARCH CENTRE OF FINLAND

    VTTINDUSTRY

    UNIVER

    SITIESBasic

    research

    Appliedresearch

    Developmentwork

    VTT Technical Research Centre of Finland

    Largest contract researchinstute in Northern Europe

    Producing research,development, testing andinformation servicesServing public sector andcompanies as well asinternational organizations

    Figures2700 employees245 M turnoverFunding1/3 government,1/3 industry, 1/3 jointly funded

  • 3

    VTT TECHNICAL RESEARCH CENTRE OF FINLAND

    Virtual and Augmented Reality at VTT

    General VR/AR expertise around VTT14 research teams in VR/AR roadmap workApplications from games to digital media,construction to production, ships, cars, military, Technology incl. core VR/AR solutions, sensors,GSM/GPS positioning, multimodality, 3D audio,motion tracking, ubi & ambient intelligence,Special strengths: Multi-disciplinary research,strong industry relations and experience

    Augmented Reality teamResearch personnel 9+ peopleAR core technology & applications

    AR at VTT situation todayAR driven projects > 2 M annually+ VTT coordinated 10 M EU project ManuVAROverall, AR is now bigger than VR

    VTT TECHNICAL RESEARCH CENTRE OF FINLAND

    Augmented Reality (AR)Mixes virtual objects with view of real worldC.f. Virtual Reality (completely virtual worlds)Properties: real-time, interactive, 3DTracking + rendering + display

    Introduction / AR

    Reality Augmented Reality Augmented Virtuality Virtual Reality

    Mixed Reality

    Related technologiesAugmented VirtualityMediated RealityDiminished Reality

    -> Mixed Reality (MR)

  • 4

    VTT TECHNICAL RESEARCH CENTRE OF FINLAND

    ARCatalogueViewing with data glasses / web cameraStereo 3D products appear from catalogueBased on marker detection from camera view

    Applications, e.g.B2B marketing (by Lappset Group )3D viewing of engineering drawings (by Pyry Inc.)

    Also, camera phonesConsumer solutions

    Example Application

    VTT TECHNICAL RESEARCH CENTRE OF FINLAND

    3D animated chirdrens TV seriesBy Futurecode (Finland) and Blue Arc (China)Launched April 2nd, 2010, MTV3 / SubTV Junior

    Augmented magazine advertisementTV-magazines Katso and TVSeiska, March 20103D animations appear augmented from magazineInteraction with augmented Dibidog

    Dibidogs

  • 5

    VTT TECHNICAL RESEARCH CENTRE OF FINLAND

    Mobile phone implementationIntroduced at Sanghai World Expo, May 1st, 2010Nokia Maemo (N900) and Symbian S60 platforms

    Augmented childrens storybookKustannus Paasilinna, August 2010Various new interactions and narrativesMarkerless tracking, photorealistic rendering etc.

    Dibidogs, Next Steps

    VTT TECHNICAL RESEARCH CENTRE OF FINLAND

    ALVAR Subroutine Library

    A Library for Virtual and Augmented RealityAR engine developed by VTTFree trial download at www.vtt.fi/multimedia/alvar.htmlUsers e.g. Columbia Univ. (Goblin XNA), MIT (6th Sense)

    Detecting and tracking 2D markersAccurate marker pose estimation, with erro concealingMarker fields can be defined manually, or autocalibratedHiding of markers as sepcial feature

    UtilitiesCamera calibration; Correcting lense undistortions; Opticalflow tracking; Several basic filters and Kalman library;Finding exterior orientation of point-sets

    Coming upMarkerless tracking; Rendering module;Mobile phone implementation

    http://www.vtt.fi/multimedia/alvar.html

  • 6

    VTT TECHNICAL RESEARCH CENTRE OF FINLAND

    ENTERTAINMENT AND GAMES

    VTT TECHNICAL RESEARCH CENTRE OF FINLAND

    Entertainment Application - CamBall

    Augmented networked table tennisPlayed with real rackets, color markerVideo image of opponent shown at other endof tableVirtual ball, table and net

    FeaturesBall speed, racket size can be adjustedSolo game against a wall, or against "oneself"Game viewing by Internet audienceRecording of game

    BenefitsSocially and physically activatingNatural human-computer user interactionFits in small interiors, at home or officeSuits also for kids and elderly peopleNo special hardware required

  • 7

    VTT TECHNICAL RESEARCH CENTRE OF FINLAND

    Mobile Entertainment - SymBall

    Virtual table tennis game for camera phonesCamera driven game control by feature/motion detectionData connection between players via Bluetooth or GPRSFirst networked camera driven game in the world

    ImplementationSymbian OS/Series 60 (& J2ME)Features: sounds, ball speed, etc.3D stereo game using r/b goggles!

    Future plan: Augmented versionUsing turnable camera, face detectionShow real opponent on phone display

    VTT TECHNICAL RESEARCH CENTRE OF FINLAND

    Internet

    Experimental Work TelePetAugmented Networked Games for Dogs

    To entertain dogs over Internet during masters absenceGame control based on gesture detection

    Example gameMaster throws virtual ball, dog has to catchRewards, e.g. biscuit box operated over internetInitial animal tests - positive results

  • 8

    VTT TECHNICAL RESEARCH CENTRE OF FINLAND

    Vappunaamari.comVirtual masks for May Day party

    Face detection/augmenting on camera phonesProvided as public service, > 1500 downloadsFeatures: send photo to friends, user created masks,compressing augmented video,

    VTT TECHNICAL RESEARCH CENTRE OF FINLAND

    INTERIOR DESIGN

  • 9

    VTT TECHNICAL RESEARCH CENTRE OF FINLAND

    Augmented Interior Design Alternative Solutions

    Basic solutionVirtual furniture object tied to eachmarkerInterior designed by moving markerson the floorDisplay on PC screen or data glasses

    BenefitsFurniture is light to move aroundEasy to try out different alternativesNo modelling of real environmentrequired

    Things to improve Web camera isnt very mobileData glasses / PC .. not tooconvenient

    VTT TECHNICAL RESEARCH CENTRE OF FINLAND

    Augmenting in Digital Camera Images

    Application ARInteriorsSingle marker placed on floor, definingscale and perspectiveDigital camera images of the room areuploaded to computerVirtual furniture is selected and placedwith menu/mouseFeatures: lighting, shadows, hiding ofmarker etc.

    BenefitsHardware (camera, PC) found inevery homeBetter image quality, resolution,accuracy etc.Photorealistic 3D rendering

  • 10

    VTT TECHNICAL RESEARCH CENTRE OF FINLAND

    Augmenting on TabletPC

    Application ARMobileTablet PC (Sony Vaio), cameraintegratedOptional data glassesUser interface to fit small screen,stylus inputNo configuration files requiredArbitrary marker orientationsMarker fields calculated automaticallyMoving, scaling and rotating objectsAdjusting of lights, with soft shadowsErasing of markersStoring of video clip & projectKalman filter & other stabilisationFeature based tracking when markersdisappear

    VTT TECHNICAL RESEARCH CENTRE OF FINLAND

    Find appealingfurniture in ad

    Download virtualfurniture viamatrix code

    Instantly viewfurniture at homeon mobile phone

    Verify by high-endPC solution (ifrequired)

    Get to shop,purchase product

    Augmenting on Mobile Phones

    Application ARPhonePlacing virtual furniture on phones camera imageModels are loaded using matrix code in product catalogue or advertUser places the furniture in camera view by 3D rotation and zoomFurniture stays in place when moving the camera phoneBased on feature tracking - no markers requiredAlso, augmenting of real furniture

  • 11

    VTT TECHNICAL RESEARCH CENTRE OF FINLAND

    ARStudio

    Unified solution for general AR applicationsFormats: real time / stored video still imagesSingle/multimarker tracking -> markerless methodsMatching augmented content with video qualityPhotorealism, shadows, materials, lighting,

    VTT TECHNICAL RESEARCH CENTRE OF FINLAND

    VividAR

    AR interior design solution by VividWorksBased on VTTs technologyLaunched February 2010Consumer web solution later this spring

  • 12

    VTT TECHNICAL RESEARCH CENTRE OF FINLAND

    ARCHITECTURE AND CONSTRUCTION

    VTT TECHNICAL RESEARCH CENTRE OF FINLAND

    Case Study: VTTs Digitalo officesDigitalo = DigiBuildingConstruction started Dec 2003Building ready Oct 2005

    Various AR/VR methods usedRadiosity visualization3D LandscapesCAVE Virtual RealityAugmented Scale ModelAugmented Web CameraMobile Augmented Reality

    Projects involvedCommissions by Senate PropertiesProject ARARAT 2004-2006

    Augmented Reality in Architecture

  • 13

    VTT TECHNICAL RESEARCH CENTRE OF FINLAND

    Digitalo augmented on conference tableViewing through video glasses (stereo)Move around model, communicate with people

    FeaturesVirtual menu for user interactionVirtual reality tour utilising feature based trackingPredefined walk-through routes, with hyperlinksRealism: textures, reflections, shadows etc.Add, move and remove parts (AR-catalogue)

    Augmented Scale Model

    VTT TECHNICAL RESEARCH CENTRE OF FINLAND

    Digitalo augmented in live web camera viewPresentation for future users of buildingFollow-up and comparison with original plansView with or without virtual buildingRea