The Glass Class: Android & GDK

July 7th – 11th 2014

Mark Billinghurst, Gun Lee

HIT Lab NZ

University of Canterbury

An Introduction to

Glassware Development

GDK

Gun Lee

* Images in the slides are from variety of sources,

including http://developer.android.com and http://developers.google.com/glass

Glassware Development

Mirror API

Server programming, online/web application

Static cards / timeline management

GDK

Android programming, Java (+ C/C++)

Live cards & Immersions

https://developers.google.com/glass/

GDK

Glass Development Kit

Android 4.4.2 + Glass specific APIs

Use standard Android Development Tools

GDK

GDK add-on features

Timeline and cards

Menu and UI

Touch pad and gesture

Media (sound, camera and voice input)

Dev. Env. Setup

JDK (1.6 or above, using 1.7 for the tutorial)

http://www.oracle.com/technetwork/java/javase/downl

oads/index.html

ADT Bundle (Eclipse + Android SDK)

http://developer.android.com/sdk/index.html

With Android SDK Manager (select Window>Android

SDK Manager from Eclipse menu) install:

- Tools > Android Build-tools (latest version)

- Android 4.4.2 (API19) SDK Platform, Google APIs, Glass

Development Kit Preview

- Extras > Google USB Driver (only for Windows Platform)

Hello Android!

Android App Components

Activity

A single screen with a user interface.

Most widely used main component.

Service

Performs operations in the background

without a user interface.

Content provider

Manages a shared set of app data.

Broadcast receiver

Responds to system-wide broadcast announcements.

Activities & Back Stack

An app usually consists of multiple activities that

are loosely bound to each other.

An app launches with the "main" activity

An activity can start another activity.

Navigation between activities managed in Back Stack

Intent and Activity

An Intent is a messaging object used to request

an action from another app component.

Start an Activity or Service, or broadcast.

Target component can be either

explicit(class name) or implicit (intent filter)

Can carry extra information (key-value pairs)

http://developer.android.com/guide/components/intents-filters.html

The Activity

Lifecycle

Create an Android App Project

In Eclipse

File > New > (Other > Android>)

Android Application Project

Fill in the Application name, Project name, and

Java package namespace to use

Choose SDK API 19: Android 4.4.2 for all SDK

settings

Minimum Required SDK might need to be lower

version (API 15) to match your device

Use default values for the rest

Anatomy of an Android App Project

Android App Project

AndroidManifest.xml

- Package name, version, SDK ver., app settings &

permissions, app components (activities)

src (source folder)

- Java classes organized in namespaces

res (resource folder)

- layout, drawable, menu, values, raw, xml

gen (generated Java files)

- res identifier classes

assets (other raw assets)

Live Demo

- Creating a new Android App Project

- Anatomy of an Android App Project

User Interface Layout Views

Draw on screen + Handle events = UI elements

(labels, input controls, ...)

Group views with layouts

LinearLayout, RelativeLayout, ...

Layout parameters

XML+GUI (res/layout) or Java code

http://developer.android.com/guide/topics/ui/overview.html

Handling UI Events

XML android:onClick attribute designates

the name of a public method in your

Activity to handle the click event

public void yourHanlderMethod(View v)

<Button

…

android:text=“Send"

android:onClick=“sendMessage" />

public void sendMessage(View v) {

// Do something on button click

}

res/layout/layout.xml MyActivity.java

http://developer.android.com/guide/topics/ui/controls/button.html

Toast

Provides simple feedback about an

operation in a small popup.

Brief notification message

Customizable view

Toast.makeText(this,

“Message sent.", Toast.LENGTH_SHORT).show();

// Toast with custom view

Toast toast = new Toast(getApplicationContext());

toast.setGravity(Gravity.CENTER_VERTICAL, 0, 0);

toast.setDuration(Toast.LENGTH_LONG);

toast.setView(viewWithLayout);

toast.show();

http://developer.android.com/guide/topics/ui/notifiers/toasts.html

Live Demo

- User Interface Layout

- Handling UI Events

Handling views in runtime

Programmatically referencing to views (UI

elements) in an Activity

View findViewById(int resId)

Listeners = Java interface Button button = (Button)findViewById(R.id.button_reset);

button.setOnClickListener(

new View.OnClickListener() {

public void onClick(View v) {

// Do something in response to button click

}

}

); http://developer.android.com/guide/topics/ui/controls/button.html

http://developer.android.com/guide/topics/ui/ui-events.html

Menu

Fill in the options menu items at creation

onCreateOptionsMenu(Menu menu)

Default implementation loads xml resource in

res/menu/main.xml

Override callback method in the Activity

onOptionsItemSelected(MenuItem item)

item.getItemId() gives resource id of the item

http://developer.android.com/guide/topics/ui/menus.html

Live Demo

- Managing views in runtime

- Menu

More Activities

Launch another Activity

new Intent(context, MyActivity.class)

startActivity(Intent intent)

startActivityForResult(intent, requestCode)

- onActivityResult(int requestCode,

int resultCode, Intent data)

Close an activity

finish()

setResult(int resultCode, Intent data)

http://developer.android.com/guide/components/activities.html

Debugging with Logs

Logging in the code

Log.d(String tag, String message)

Log.e(), Log.i(), Log.w(), Log.wtf()

Viewing logs

LogCat

Window>Show View>Other>Android>LogCat

Can filter with tags, verbosity, keywords

http://developer.android.com/reference/android/util/Log.html

Live Demo

- Launching and closing Activities

- Debugging with Log messages

More Information User Interface

http://developer.android.com/guide/topics/ui

Animation and Graphics

http://developer.android.com/guide/topics/graphics

Data Storage

http://developer.android.com/training/basics/data-storage

http://developer.android.com/guide/topics/data

Media and Camera

http://developer.android.com/guide/topics/media

Location and Sensors

http://developer.android.com/guide/topics/sensors

Connectivity and Networking

http://developer.android.com/guide/topics/connectivity

http://developer.android.com/reference/java/net/package-summary.html

Hello Glass!

- Setting up for development

Dev. Env. Setup for Glass

Google USB Driver for Windows

Virtual Device Definition for GUI Layout

(Processing for this afternoon ...)

Google USB Driver on Windows Control Panel > System and Security > Device Manager

Google USB Driver on Windows

Choose the path where the inf file is in

SDK\extras\google\usb_driver

Virtual Device Definition for Glass Window > Android Virtual Device Manager >

Device Definitions > New Device

640x360px

3in (hdpi)

Landscape

Using Device Definition in GUI Editor

Device: User > Google Glass

Theme.DeviceDefault.NoActionBar.FullScreen

Hello Glass! - Immersions: Android Apps on Glass

Live Cards vs. Immersions

Live cards display frequently updated

information to the left of the Glass clock.

Integrate rich content into the timeline

Simple text/images to full-blown 3D graphics

Immersions let you build a user

experience outside of the timeline.

Build interactive experiences

Extra control, less user input constraints

Live Cards

Immersions

Live Cards vs. Immersions

Develop with GDK

Android 4.4.2 (API 19) SDK and GDK Preview

from the Android SDK Manager.

Project settings:

Minimum and Target SDK Versions: 19

Compile with: GDK Preview

Theme: None (allows the Glass theme to be applied.)

GDK samples

File > New Project > Android Sample Project

On Glass, turn on USB debugging

Settings > Device Info > Turn on debug

Hello World - Immersion

App/Activity without theme

Allows the Glass theme to be applied.

Add voice trigger for launching

Touch input and Menu

Voice recognition for text input

Glass Theme

Remove app theme in AndroidManifest.xml

Voice Trigger for Launching

Add intent filter your main Acivitiy in

AndroidManifest.xml

Add xml/voice_trigger.xml to res folder

Can use additional follow up voice recognition

prompt if needed

<?xml version="1.0" encoding="utf-8"?>

<trigger keyword="hello world" />

<uses-permission android:name="com.google.android.glass.permission.DEVELOPMENT" />

…

<intent-filter>

<action android:name="com.google.android.glass.action.VOICE_TRIGGER" />

</intent-filter>

<meta-data android:name="com.google.android.glass.VoiceTrigger“

android:resource="@xml/voice_trigger" />

https://developers.google.com/glass/develop/gdk/input/voice

Official Voice Triggers on MyGlass

listen to

take a note

post an update

show a compass

start a run

start a bike ride

find a recipe

record a recipe

check me in

start a stopwatch

start a timer

start a round of golf

translate this

learn a song

tune an instrument

play a game

start a workout

https://developers.google.com/glass/develop/gdk/input/voice

Live Demo

- Apply Glass Theme (remove Theme)

- Add Voice Trigger for Launching

Touch Input as Key Input

Touch input translated as DPAD key input

Tap => KEYCODE_DPAD_CENTER

Swipe down => KEYCODE_BACK

Camera button => KEYCODE_CAMERA

https://developers.google.com/glass/develop/gdk/input/touch

@Override

public boolean onKeyDown(int keycode, KeyEvent event) {

if (keycode == KeyEvent.KEYCODE_DPAD_CENTER) {

// user tapped touchpad, do something

return true;

}

…

return false;

}

Touch Input

onGenericMotionEvent(MotionEvent e)

https://developers.google.com/glass/develop/gdk/input/touch

@Override

public boolean onGenericMotionEvent(MotionEvent event) {

switch (event.getAction()) {

case MotionEvent.ACTION_DOWN:

break;

case MotionEvent.ACTION_MOVE:

break;

case MotionEvent.ACTION_UP:

break;

}

return super.onGenericMotionEvent(event);

}

Touch gestures

GDK provides GestureDetector for Glass

com.google.android.glass.touchpad.GestureDetector

- NOT android.view.GestureDetector

BaseListener, FingerListener, ScrollListener,

TwoFingerScrollListener

Pass MotionEvent from onGenericMotionEvent()

gestureDetector.onMotionEvent(event);

https://developers.google.com/glass/develop/gdk/input/touch

https://developers.google.com/glass/develop/gdk/reference/com/google/android/glass/touchpad/GestureDetector

Live Demo

- Handling Key input

- Touch input and Detecting gestures

Menu

Open options menu on tap

openOptionsMenu()

Add 50x50 pixel icons in the menu

resource XML

android:icon="@drawable/icon"

- https://developers.google.com/glass/tools-

downloads/menu_icons.zip

Show/hide/update menu items if needed

onPrepareOptionsMenu()

https://developers.google.com/glass/develop/gdk/ui/immersion-menus

Live Demo

- Menu

Voice Input

Start activity for result with system action

Customize prompt with intent extra

Recognized strings returned in intent data

of onActivityResult()

https://developers.google.com/glass/develop/gdk/input/voice

intentData.getStringArrayListExtra(RecognizerIntent.EXTRA_RESULTS);

http://developer.android.com/reference/android/speech/RecognizerIntent.html

intent = new Intent(RecognizerIntent.ACTION_RECOGNIZE_SPEECH);

startActivityForResult(intent, 0);

intent.putExtra(RecognizerIntent.EXTRA_PROMPT,

"What is your name?");

Live Demo

- Voice Input

Hello World - Immersion ++

Play Sounds & Text-to-speech

Take a picture with camera

Card based info page

Playing Sounds & TTS

Glass system sounds

Text-to-speech

Create/destroy TTS in onCreate/onDestroy()

https://developers.google.com/glass/develop/gdk/reference/com/google/android/glass/media/Sounds

AudioManager am =

(AudioManager) getSystemService(Context.AUDIO_SERVICE);

am.playSoundEffect(Sounds.ERROR);

// DISALLOWED, DISMISSED, ERROR, SELECTED, SUCCESS, TAP

TextToSpeech tts = new TextToSpeech(context, ttsOnInitListener);

…

tts.speak(“Hello world!”, TextToSpeech.QUEUE_FLUSH, null);

tts.shutdown();

http://developer.android.com/reference/android/speech/tts/TextToSpeech.html

Playing Custom Sounds

Put sound files in res/raw

Load sounds to SoundPool object to play

soundPool = new SoundPool(MAX_STREAM,

AudioManager.STREAM_MUSIC, 0);

int soundOneID = soundPool.load(context, R.raw.sound1, 1);

int soundTwoID = soundPool.load(context, R.raw.sound2, 1);

…

soundPool. play(int soundID, float leftVolume, float rightVolume,

int priority, int loop, float rate)

http://developer.android.com/reference/android/media/SoundPool.html

Live Demo

- Playing Sounds & TTS

Camera

Calling the Glass built-in camera activity

with startActivityForResult() and Action

Intent, returned with file path to

image/video through Intent extra data.

Low level access to camera with the

Android Camera API.

http://developer.android.com/reference/androi

d/hardware/Camera.html

https://developers.google.com/glass/develop/gdk/media-camera/camera

Camera with Action Intent

https://developers.google.com/glass/develop/gdk/media-camera/camera

private void takePicture() {

Intent intent = new Intent(MediaStore.ACTION_IMAGE_CAPTURE);

startActivityForResult(intent, TAKE_PICTURE);

}

@Override

protected void onActivityResult(int requestCode, int resultCode, Intent data) {

if (requestCode == TAKE_PICTURE && resultCode == RESULT_OK) {

String picturePath =

data.getStringExtra(Intents.EXTRA_PICTURE_FILE_PATH);

// smaller picture available with EXTRA_THUMBNAIL_FILE_PATH

processPictureWhenReady(picturePath); // file might not be ready for a while

}

super.onActivityResult(requestCode, resultCode, data);

}

Live Demo

- Taking a picture

Using Cards as Views

Create a Card object then call getView()

Card card = new Card(this);

card.setText("The HIT Lab NZ");

card.setFootnote("http://www.hitlabnz.org");

card.setImageLayout(ImageLayout.LEFT);

card.addImage(R.drawable.hitlabnz);

setContentView(card.getView());

https://developers.google.com/glass/develop/gdk/ui/theme-widgets

Live Demo

- Using cards as views

Scrolling Cards in Activity

Set a CardScrollView as the content view

Use a custom class extending the

CardScrollAdapter class to populate the

CardScrollView

https://developers.google.com/glass/develop/gdk/ui/theme-widgets

https://developers.google.com/glass/develop/gdk/reference/com/google/android/glass/widget/package-summary

CardScrollView cardScrollView = new CardScrollView(this);

cardScrollView.setAdapter(new InfoCardScrollAdapter());

cardScrollView.activate();

setContentView(cardScrollView);

Scrolling Cards in Activity In your custom CardScrollAdapter class

Create a list of cards

Implement abstract methods in your custom

CardScrollAdapter class - int getCount() => return the number of cards (items)

- Object getItem(int position) => return the card at the position

- View getView(int position, View convertView, ViewGroup

parentView) => return the view of the card at the position

- int getPosition(Object item) => find and return the position of

the given item (card) in the list. (return -1 for error)

https://developers.google.com/glass/develop/gdk/ui/theme-widgets

https://developers.google.com/glass/develop/gdk/reference/com/google/android/glass/widget/package-summary

Live Demo

- Scrolling cards

Next time:

Live cards - Native Glassware sitting on timeline

More Information

Website

https://developers.google.com/glass

http://developer.android.com

http://arforglass.org

http://www.hitlabnz.org

Gun Lee

gun.lee@hitlabnz.org

Mark Billinghurst

mark.billinghurst@hitlabnz.org