The Glass Class:

AR on Glass

July 7th – 11th 2014

Mark Billinghurst, Gun Lee

HIT Lab NZ

University of Canterbury

http://www.arforglass.org

Demo

- CityViewAR for Glass

Outdoor AR Framework

Outdoor AR Visualization

OpenGL ES based scene rendering

GPS, inertial, compass

Component based Framework

Easily customizable UI

Supports various styles of apps

Android (+ Web Tools)

Free download on our website

http://www.hitlabnz.org/mobileAR

Outdoor AR Browsers

Wikitude Layar Junaio

SW Dev. Tools & Frameworks

High-level content description

AR Browsers

Hard to customize app logic and UI

Low-level functional modules

Programming libraries

Lack of higher-level abstraction of AR scene

and content (BYO)

Outdoor AR Framework High-level abstraction and low-level

programming support in a seamless way.

Low-level functional modules

- Flexible and highly customizable

Ready-to-use high level components

- Focus on building the interface, behavior and

content of the application rather than system

integration

The high-level and low-level tools share the

common scene (or POI) data structure

- Both skilled and novice developers can work

together under the same framework.

Apps built with OAR Framework

[CityViewAR 2011] [CCDU 3D 2012]

[GeoBoids 3D 2012] [AntarcticAR 2012]

Outdoor AR Framework

Covers full tier of ready-to-use

components, yet highly customizable

Mobile Device (Client)

Mobile OS

Mobile AR Application

Scene

data

Media

data

Mobile SW Library

Server

Server OS

Application Service

Scene

data

Media

data

Server SW Library Data

Communication

(Optional)

Server Software and Tools

Web Client

(Browser)

Server Mobile Client

(App)

HTML Web-based

Authoring Tool

RESTful API

Scene

Database

AR View

Data

Manager

Local

cache

Javascript

Server Software and Tools

Web-based Authoring Tool

Outdoor AR Library

Functional Component Layer

Service Component Layer

Map

Component

Scene Data Manager

AR

Component

List

Component

Tracking Data Manager

Scene Data

Structures

Tracking

Sensors

3D Graphics

Rendering UI Views

Tools &

Utilities

3D

Sound

Map View AR View List View Views

Controllers

Models

Outdoor AR Library

Customization by extending components

Customized AR scenes

- Customize scene data structure

Customized user interface

- Customize UI layout

- Customize UI behavior (e.g. onSceneSelected)

- Motion and touch gesture based interaction

Step-by-step Guide

Setup the development environment

Create an Android app

Using the Outdoor AR Library in your app

Adding outdoor AR scenes with 3D models

Customizing UI

Run on a Google Glass

Setup the dev environment

Prerequisite

Android Development Environment

- http://developer.android.com/sdk/index.html

- Eclipse, Android SDK

Google Play Services Library (for maps)

Download the Outdoor AR Library

http://www.hitlabnz.org/mobileAR

- Download, Tutorials, Forum

Setup the dev environment

Import the library and sample projects

OutdoorARLibrary

SampleOAComponent

Dependency

Google (not Android) API 4.1.2 (API Level 16)

Google Play Services Library

- extras/google/google_play_services/libproject

Android Support Library 4

- extras/android/compatibility/v4/

android-support-v4.jar

Create an Android app

Create a new Android Application project

Blank Activity

- “An activity is a single, focused thing that the user

can do.” – Android SDK API Reference

Anatomy of an Android App project

src

res & gen

assets

AndroidManifest.xml

Using the OA Library in your app

Add the Outdoor AR Library to your

project’s build path

Add permissions in the manifest

Add a new AR view (activity)

Create a custom class inheriting

OAARComponentBase

Add activity to the manifest

- Full screen landscape orientation

Wire a button to start the AR view

Adding scenes with 3D models

Preparing 3D model

OBJ/MTL with JPEG or PNG textures

OpenGL Coordinate frame

- X-right, Y-up, Z-out from screen

- North = -Z

Placing the 3D model in the real world

Latitude, Longitude

- http://itouchmap.com/latlong.html

Optionally, elevation from the ground plane

SketchUp – 3D model on Google Earth

Adding scenes with 3D models

Adding 3D models to the project

assets/OutdoorAR/3Dmodels/name/model.obj

Adding scenes to the data manager

Override onSetupScenes()

Handling the scene selection event

Override onSceneSelected()

Using a mock location

Override onCreate()

Sensor manager’s enableMockLocation()

Loading and managing scenes

Add scenes by code – difficult to manage

Load from assets or local file storage

OADataManagerAssets

OADataManagerLocal

SQLite or XML file

SQLite Database Browser

Default path

- (assets/)OutdoorAR/scenes.db

- (assets/)OutdoorAR/scenes.xml

Custom scene class

OAScene

Lat/Lon, 3D model, Name, Description,

Category

Custom scene classes extend OAScene

Override OADataManager’s loadScene()

to load in custom properties of your scene

Cast OAScene to your custom class in

onSceneSelected() or onTouchedScene()

Different scene classes can live together

- if(scene instanceof MyCustomScene)

Customizing UI

Add layout resource

Override setupUILayout()

LayoutInflater.from().inflate()

- read in the layout res file

addContentView()

- overlay UI layout on top of AR view

findViewById()

- get reference to the UI elements

onSceneSelected() vs. onTouchedScene()

Map and List views

OAMapComponentBase, OAListComponentBase

Similar class structure with OAARComponentBase

- Load scenes from a data manager

- setupXYZ() methods

- onSceneSelected() callback

Map components needs API key

Google Maps Android API v2

- OAGoogleMapsV2ComponentBase

https://developers.google.com/maps/documentation/an

droid/start

Run your app on Google Glass

Startup with AR

Set AR view as a launcher activity in the app

manifest file

Aim to select

Use Timer to trigger touch events at the

center of the screen - getWindowManager().getDefaultDisplay().getMetrics(dm);

- getSceneView().triggerTouch(x, y)

Override onResume() and onPause() to

start/stop the timer

Other tips for Google Glass

Touch gestures

onGenericMotionEvent()

Swipe forward/backward/up

Tap, double tap, tap and hold, etc.

Head motion

Add orientation listener to sensor manager

- getSensorManager().addSensorListener()

Avoid conflicts with AR scene viewing motions

- Time-outs

Other tips for Google Glass

GPS not on Glass

Pair with Phone then use remote provider

Stream location from a phone through

Bluetooth

Mock location

Turning off camera background

Application dependent

Override setupOptions()

enableCameraBackground()

http://developer.android.com/guide/topics/connectivity/bluetooth.html

CityViewAR for Glass

Demo Revisited

More Information

Website

http://www.hitlabnz.org/mobileAR

http://arforglass.org

Gun Lee

gun.lee@hitlabnz.org

Mark Billinghurst

mark.billinghurst@hitlabnz.org