Visualising a New Christchurch

Mark Billinghurst HIT Lab NZ

University of Canterbury

Christchurch Earthquakes

http://www.rebuildchristchurch.co.nz/content/christchurch-building-demolition-map

Christchurch Before and After

  Professional solutions available   Autodesk REVIST, ESRI ArcGIS, Grass,etc

Emerging Novel User Interfaces   Immersive visualization   Multi-touch screens   Gesture interaction   Etc..

Limitations   Interface

 Complex to use  Unintuitive interaction   2D display for 3D content

  Context of Use   Remote visualization   Separation from the real world  Unable to show life-sized content

Augmented Reality  Combines Real and Virtual Images

-  Both can be seen at the same time   Interactive in real-time

- Content can be interacted with  Registered in 3D

- Virtual objects appear fixed in space

  Put AR pictures here

Augmented Reality Examples

AR and Architecture

AR and Archiecture   Enhanced Maps

  AR overlay on printed material

  Outdoor AR   In place visualization

  Mobile AR  Urban Visualization

AR Markerless Tracking

  OPIRA Library  Uses natural features for tracking   Fast, robust performance

Interactive AR Maps   Markerless tracking   3D model overlay   Gesture input

Enhanced City Plans

  CERA – CCDU Plan  Using tablet to track off printed maps  Overlay 3D city models onto real maps

User Experience

  Touch interaction   Pointing selection   Animated objects

Outdoor AR

  Highly accurate outdoor AR tracking system  GPS, Inertial, RTK system  HMD

  First prototype   Laptop based   2-3 cm accuracy

Image Registration

AR Surveying Application

PCI 3D Graphics Board

Hard Drive

Serial

Ports

CPU

PC104 Sound Card

PC104 PCMCIA

GPS Antenna

Tracker Controller

DC to DC Converter

Battery

Wearable Computer

GPS RTK correction

Radio

Example self-built working solution with PCI-based 3D graphics

Columbia Touring Machine

Mobile AR - Hardware

2008 - Location Aware Phones

Nokia Navigator Motorola Droid

Real World Information Overlay   Tag real world locations

 GPS + Compass input  Overlay graphics on live video

  Applications   Travel guide, Advertising, etc

  Eg: Layar (www.layar.com)   iPhone, Android based, Public API released

  Other companies  Wikitude, Metaio, AcrossAir, Tochnidot, etc

Layar – www.layar.com

3DOn

  Onsite Visualization Single Building  GPS + Compass input  Overlay graphics data on live video/Photos

HIT Lab NZ Outdoor AR Platform

  Cross platform   Android, iPhone

  3D onsite visualization   Intuitive user interface

  Positions content in space  Camera, GPS, compass

  Client/Server software architecture   Targeting museum guide/outdoor site applications

CityViewAR

  Using AR to visualize Christchurch city buildings   3D models of buildings, 2D images, text, panoramas   AR View, Map view, List view   Available on Android market

User Experience

  While walking in the real world people can see text, 2D images and 3D content on their own phones

Interface Design (1/2) Browsing Interface Content Front face

Title Screen

AR View

Map View

List View

Detail View

Image Gallery

Panorama Instruction & Information

List View

List of all assets

Map View

Icons for buildings, viewpoints, panoramas

Building History Data

Photographic Images

Panorama Images

360 degree photo bubbles

Augmented Reality View

Survey Results   Use of AR improved the user experience

  But no difference between AR and non-AR usefulness

  Favourite features   AR and Panorama views most popular  Users also enjoyed having rich data available

  Main Problems (50% reported no problems)  UI design not intuitive (24%)   System not responsive (16%)

Time Usage

  Percentage time spent in different views   AR used more then 50% of time when available

Result - Distance Traveled   As a measure of active exploration   M=96.28 meters (SD=83.85, 0~315 meters)

Mann-Whitney U-test U = 167.00, p = 0.18 With AR Without AR

Areas for Improvement   Outdoor AR tracking

  Reduce tracking errors

  User interface design  Make options more obvious

  Make system more responsive   Some lag in loading content

  Touch input   Responding to unintended touches

AR Urban Design Tool (2012)   CERA CCDU Application

  Enhanced CityViewAR application

  Add VR view mode   3D model viewing/interaction

  Add concept models of buildings  Multiple buildings at single site

  Client/Server architecture   Support for user generated feedback

VR View

Differing Views

  List View, Content View, Map View

Interaction   Touch interaction

  Information filtering   Building information   Model panning, zoom, rotation

iceFest (2012)

  Outdoor AR for Antarctic Experience   Visit Antarctica in real world   Using handheld tablet

Map Views

Content Views

AR View

  View of ice superimposed over real world

Panorama

  Immersive 360 panorama

Feedback

  A = Satisfying, B = Ease to use   C = Useful for learning, D = Better learning than internet

Looking to the Future

Next Steps   Subsurface Visualization

  See underground infrastructure

  User evaluation   Architects, Urban Designers, Public

  Develop authoring tool  Web-based content management system

  Improve technology   Tracking, 3D rendering

Subsurface Visualization

  Using AR to view underground infrastructure

Client/Server Architecture

Android application

Web application java and php server

Database server Postgres

Web Interface

Add models

Web based Outdoor AR Server   Web interface

  Showing POIs as Icons on Google Map

  PHP based REST API  XML based scene

data retrieval API   Scene creation and

modification API   Android client side

REST API interface

BASIC VIEW

PERSONAL VIEW

Augmented Reality 2.0 Infrastructure

Leveraging Web 2.0   Content retrieval using HTTP   XML encoded meta information

  KML placemarks + extensions

  Queries   Based on location (from GPS, image recognition)   Based on situation (barcode markers)

  Syndication   Community servers for end-user content   Tagging

  AR client subscribes to data feeds

Conclusions •  AR allows for information overlay in place •  CityViewAR provides pre earthquake viewing •  Opportunities for on-going development •  Important research problems need to be solved

– Wide area tracking – User experience – Social networking – Etc..

Collaborators   Jason Mills (PVI)

  Panoramas, 3D models

  Historic Places Trust   Pre-1950 building information

  Christchurch City Council   Statements of Significance

  CEISMIC  Digital content

More Information

•  Mark Billinghurst – mark.billinghurst@hitlabnz.org

•  Website – www.hitlabnz.org