Cyborg Design:

IA Summit Bennett King

@skunkwUrX

Multimodal Interactions, Information, and

Environments for Wearable Computing

Opinions expressed in this presentation are my own and do not

necessarily reflect those of my company

source: wearable technology database | Vandrico Inc

2

There is a lot of hype about wearable devices with even predictions of a $30B market over the

next 2 years. The reality…most wearable devices will fail

2

Wearable Computing Spaces:

3

The focus of this presentation is Information wearables: connected devices that are worn for

extended periods of time to enhance the user’s capability. Specifically, smartglasses &

smartwatches. These have the best chance of sustained use. 4

5

fitness/health

smartwatches

smart others

smartglasses

source: Strategy Analytics

2013 2013 2017

Wearable Device Shipment

by Category

But why should we care?

Information wearables represent the first major change in computing in the last 30 years,

breaking away from devices that require you to conform to them. Information wearables

conform to our needs at any given time and enhance our capabilities in the real world.

6

photo: Steve Mann | eyetap.blogspot.com

”

“

Steve Mann

Chief Scientist, Meta

Information wearables become an “extension of ourselves”. We interact with them in more

natural ways and the content provided is relative to our given tasks.

7

8

Current forces behind wearable devices

9

photos: samsung.com; sony.com; getpebble.com; moto360.com. apple logo TM of Apple Inc.

Qualcomm Toq

Galaxy Gear

Sony Smartwatches

Pebble

Moto 360

(Mythical) iWatch

10

photo courtesy: google.com/glass; laforgeoptical.com; buhel.com photo courtesy: spaceglasses.com; lumus-optical.com

Meta

Lumus

ODG

Google Glass

LaForge

Buhel

11

photo: Laforge Optical photo: Google

Even with more palatable designs, acceptance and sustained use are formidable issues.

Less than half of people under 30 wear watches. If you don’t already wear glasses, what would

make you put them on day after day? How do we affect changes in behavior?

12

photo: tomsguide.com

Another major problem facing wearable devices is that for every well-designed device, there

are 10 that are…let’s say not as well designed.

13

Blogs, market reports, tech reviews, editorials claim success lies with “additional use cases”,

“meeting user needs”, “creating meaningful & engaging experiences”. These terms should not

be unfamiliar to our industry and are key to modifying existing habits and creating the desire

for continued use.

14

What’s missing from the picture is you. If we cannot create meaningful experiences to go along

with form and function then wearables will join the long list of failed tech.

15

changes...

16

17

+

If mobile device

content is about

truncation, wearable

device content is

about encapsulation.

• Smartwatches are limiting – space, colors, content, readability.

• Smartwatches should allow consumption and interaction at a glance.

• You’ll need to use every trick in the book to enhance the experience – visual affordances,

LEDs, haptics.

18

• Monocular devices limit you to roughly a 15º FOV.

• Usually fixed location.

• Varying levels of opacity and background noise.

19

• Binocular devices are currently maxed at about 40º FOV (but are improving).

• Stereoscopic display allows for 3D design – the x y and z can be used in your information,

interaction, and interface design.

• Varying levels of opacity and background noise.

20

The differentiator for binocular glasses is the possibility of 3D augmented reality interfaces –

augmenting content out of cyberspace and into the physical space. Standing alone, AR and

smartglasses are underwhelming but together they can enhance adoption and retention.

21

touch voice gesture

22

micro

gestures

eye gaze

& dwell

enhanced

gestures

body

movement

23

natural

language

brain-computer

interfaces

emotion

24

”

“

Bill Buxton

Principal Researcher, Microsoft

Flikr: Hamed Mousami

25

Photo: wired.com

26

When designing in a new area, research and prototyping are more valuable than ever. But

research requires building new platforms. Luckily our work corresponds with the makers

movement: “If you can think it, you can build it”.

• ID: FabLabs, makers clubs, mfg.com

• SW: Google wear

• HW: Arduino kits, Dragon boards, sensor

platforms

26

Build (whatever you can) and learn. This is our first AR research rig made out of reflective

plexiglass, metal rods, a Pico projector and a used marching band drum harness. You can

quickly move to 3D printed prototypes and iterate further.

27

28

29

30

31

* I cannot take credit for AEIOU but it’s been around so long that I am not sure who to credit. Feel free to say it was you.

photos: moto360.motorola.com

Wearable interfaces should work with your activity. The interfaces and apps we are used to on

our phones and tablets do not apply while riding a bike or running. Subtle, contextual,

glanceable…create designs that understand activity.

32

We have spent a considerable time understanding how user’s interact with their wearable

device in different environments…and it’s rarely constant. You devices and designs should

support multiple modes of interaction and understand their context of use.

33

Understand the object and its intended use. While talking to your watch is cool in the movies, it

is less so in the real world. Does this seem right to you?

34

35

> Display

> Creating media

> Communication

> Consumption

> Text Entry

> Content creation

> Communication

> Display

> Consumption

> Biometrics

> Natural Input

> Display

> Consumption

While some wearables are stand-alone, there is greater benefit if they work together. Each

device has its strengths for interaction, information, and input. Multiple sensors on multiple

devices can work together for better accuracy and experience (not to mention benefits in

processing and battery life).

36

37

One of the keys to designing for immersion is understanding instance. Fitness wearables have

had some success by understanding a specific instance. If we want to create devices that

people will adopt an continue to use we need them to juggle multiple instances. What is the

experience while shopping with family, while out with friends, while at home alone relaxing…

37

38

Many of the wearable demos you’ll see are overkill. While you are walking down the street you

probably don’t need a constant overlay for navigation with live updates.

38

Instead, augmented reality allows us to overlay objects fixed to specific locations. So navigation

now appears where and when you need it, attached to the world around you.

39

40

The Glance mode represents the always-on interface that is the backbone of engagement

(think notifications). It should be discreet and not detract from the user’s real world activity. It

should inform without monopolizing attention.

41

photo: howto.com

The Assist mode (best with but not limited to smartglasses) is the home of apps. It can

partially occupy the screen, follow and update based on tasks, and autohide or turn off based

on need. The input should also be multimodal to allow voice and hands-free interaction.

42

The Augment mode is the home of AR. Markers, information and interfaces augmented over

the real world. AR will be one of the greatest challenges for Information Architects going

forward – the vast amount of information available online presented exactly when and where it

is needed without distraction or annoyance to the user.

43

The final mode, Immerse, is used for interfaces that are intended to monopolize our attention –

reading a book, gaming, looking at photo galleries…

44

45

Always on and always listening; this is the minimum standard for wearable devices. If the user is

interacting with the world around them, they expect the wearable device to react in real time.

While there are technical challenges to this, the multiple sensors allow for us to provide the

illusion of always on. Our first goal should be to remove the smartphone glance.

46

photo: flikr: Shawn Willis

The camera is one of the best examples. During our research this feature consistently stood

out for expected immediacy – look and shoot.

47

48

Our devices should not create peculiar social interactions like the Glass Gloss. Remember that

we will be interacting with others with these things on. We must find ways to use the

technology without straying from social norms.

49

Example: We’ve all forgotten someone’s name. We can replace the awkward interaction caused

by us wandering off into the ether to find them online with an AR overlay that appears in our

line of sight. In a wearable system, you have cameras, connectivity, CV and AR running in

conjunction to enhance our forgetful self.

50

photo credit: www.nbcbayarea.com

Perceived personal privacy has also become one of the biggest issues we need to design for.

The lifelogging capabilities of wearable devices have caused a backlash based on worries of

surveillance. How can we make these actions noticeable and acceptable to those around us?

51

52

Wearables are just the tip of the iceberg. Today’s

technology is advancing exponentially faster since the

birth of the smartphone.

BCI units are already on the market and should be commercialized in devices within the next

five years. There are already several projects looking to add the input to smartglasses. With the

availability of Open BCI (EEG capture platform) the technology is now available to makers

everywhere. Your next interaction – Think and make it happen.

53 53 53 53 53 53

54

Why wear technology when you can just make it part

of you?

55

photo: brown.edu photo: givenimaging.com photo: illinois.edu

Wireless Optogenetics:

using injectable lights to

stimulate targeted neural

pathways in the brain

SmartPill: an ingestible

capsule that wirelessly

measures pressure, pH

and temperature

Implantable BCI: a fully

implantable and

rechargeable wireless

brain sensor

sight

At the beginning of the talk I said “If it can be done it will be done”. While current wearables

and implantables are being designed as assistive devices, they could easily start to enhance us

well beyond current human capabilities (my coworkers already clamor for a brain chip). This

moves us to a gray ethical area. With projects advancing so rapidly, we are often the only voice

for humans in tech discussions. I wonder if ethics, philosophy and even neurosciences will

become part of our future skillset.

Actual meme

at my office

56 56 56 56 56 56 56

bennettmking@gmail.com

@skunkwUrX

*