Superhuman Computing: Designing Custom Software and Hardware Interfaces to Support Our Natural Abilities

Show art

Superhuman Computing: Designing Custom Software and Hardware Interfaces to Support Our Natural Abilities

Shaun Kane @shaunkaneDepartment of Computer ScienceUniversity of Colorado BoulderShaun Kane @shaunkaneThank you all SO MUCH for being here1DisclaimerNo actual superpowers in (todays) talk

Instead, how to use computing to be more independent, more interdependent, more creative2Shaun Kane @shaunkaneNo flying robots either2Disclaimer, part 2My first plenary talk

Just be inspirational.3Shaun Kane @shaunkaneStudents say no one wants to hear about your researchImpostor syndrome. You might think oh, once I get my degree it will go away. Once I get a faculty job it will go away. Then you get invited to do something like this, and you see youre on the same list of speakers as Freeman Hrabowski 3This talkHow human-centered computing and design can support empowerment

Designing for diversity

How to do your best work and to apply computing for good4Shaun Kane @shaunkaneMy academic timeline2003 BS in Computer Science, UMass Amherst

2005 MS in Computer Science, UMass Amherst

2011 PhD in Information Science, Washington2011 Assistant Professor, UMBC Information Systems

2014 Assistant Professor, Colorado Computer Science

5Shaun Kane @shaunkaneOur researchDeveloping innovative interfaces to support people with disabilities

Understand and extend peoples abilities

Combine innovative human-centered design with real world problems6

Shaun Kane @shaunkaneDisability vs ability. Disability is what someone else can do and you cant. Who cares what someone else can do, what can I do?Problems: have to be exciting, have to be difficult, and have to be impactful6Big ideaAccessibility can be about what we have to do, and rules, and other boring stuff.

But it can also be about working with motivated people to design and deploy cool new technology, and make peoples lives better.

7Shaun Kane @shaunkanePeople with disabilities are test pilots / daredevilsTheyre already using future user interfaces

They were into wearable computing before it was cool

Less affected by technology inertia willing to take on new tech if it really helps

8Shaun Kane @shaunkane89

Shaun Kane @shaunkanehttp://cheezburger.com/60248248329Designing for diversityMost of my work involves design for accessibility for people with disabilities

How can we design technology to empower everyone?10Shaun Kane @shaunkaneOur researchIntensive fieldwork with community organizations (rehabilitation centers, community organizations)

Rapid prototyping and design

Evaluation in the lab and in the field (sometimes having to invent new methods)11

Shaun Kane @shaunkane

12

Shaun Kane @shaunkaneNew opportunitiesUbiquitous mobile computing (at least in developed world)

DIY and maker culture brings computing to the physical world

Computing can change the world

13

Shaun Kane @shaunkaneIf you want to change the world, build technology. Took inspiration from DilmaAs Chad said, change the worldhttp://i.dailymail.co.uk/i/pix/2013/03/15/article-2293785-18AF1C59000005DC-332_634x317.jpghttp://i.huffpost.com/gen/2045390/thumbs/n-WOLVERINE-HAND-large570.jpg13ProjectsDesigning eyes-free touch screens

Designing interactive power wheelchairs

Understanding DIY assistive technology sharing14Shaun Kane @shaunkaneProjectsDesigning eyes-free touch screens


Understanding DIY assistive technology sharing15Shaun Kane @shaunkaneEyes-Free Touch Screens [2008-]How can we design touch screen user interfaces that can be operated without sight?

Can we achieve this with off-the-shelf devices?16

Collaborators: Jacob O. Wobbrock, Richard E. Ladner, Jeffrey Bigham, Meredith Ringel Morris, Annuska Perkins, Daniel Wigdor, Brian FreyShaun Kane @shaunkaneThe first project is called Slide Rule. It began with the question: how can we take an existing touch screen based mobile device, which is not accessible, and redesign the fundamental interactions that it supports

And to give you some context about this project, although we did see accessibility on the iPhone in the video. This project, Slide Rule, predates that work. At the time this work began, most commercially available touch screen devices provided no accessibility support, and those that did were quite limited.16Motivation91% of US adults have a mobile phone, and 56% have a smartphone (Pew 2013)

Almost 30% of AT abandoned by users (Philips and Zhao 1993)

Users increasingly reject custom accessible technology for mainstream technology (Kane et al. 2009, Shinohara and Wobbrock 2011)

Challenge: How can we make mainstream devices accessible through human-computer interaction design?

17Shaun Kane @shaunkane17How to do itAppropriate outputSpeech and audio

Appropriate inputHow do users touch the device?Screen layoutUsable, reliable gesturesSpeech input18

Shaun Kane @shaunkaneAnd in particular how can we change an existing device

And Ill primarily focus on input.18Projects on eyes-free touchDesigning an accessible gesture set for smartphones [Slide Rule, Kane et al., ASSETS 08]

Adapting accessible gestures to large displays while preserving spatial understanding [Access Overlays, Kane et al., UIST 09]

Comparing blind and sighted gesture performance [Kane et al., CHI 11]

Using accessible gestures in augmented reality applications [Access Lens, Kane et al., CHI 13]19Shaun Kane @shaunkane20

Accessible Touch Screen Map[Access Overlays, Kane et al., UIST 09]Shaun Kane @shaunkaneAnd to give you a brief example of these limitations. Ill show you a video of 20

Edge projectionSearching a touch screen using your hands is time consuming How can we provide guides to support exploration?Can we design gestures to make navigation easier?21Shaun Kane @shaunkane21


Shaun Kane @shaunkane24

Shaun Kane @shaunkaneThese are demo videos using sighted users, and with screen on for illustrative purposes. Illustrative of tasks in the study Ill describe.24Touch-and-Speak25Voice commands for browsing map locationsSearch command provides guided speech feedback to goal

Shaun Kane @shaunkaneBrowsing also2526

Shaun Kane @shaunkaneResearch findingsBlind and sighted users preferred different types of gestures, and performed gestures differently [CHI 11]

Touch interfaces can be faster than physical interfaces for some tasks [ASSETS 08]

Design of gesture set matters: influences performance, spatial understanding [UIST 09]27Shaun Kane @shaunkane27TakeawaysModern computing devices often have unused input/output capabilities

Potential for redesigning to support accessibility

Can support accessibility in a much more integrated way

28Shaun Kane @shaunkaneLegacyAcademic impact, 200+ citations(technology for blind, new mobile devices, interactive tabletops)

Gesture-based screen readers now common on mobile phones

Hundreds+ of accessible apps available

29

VoiceOver for iPhone(Apple 2009)

Android Eyes-Free Shell(Google 2009)Shaun Kane @shaunkane29ProjectsDesigning eyes-free touch screens


Understanding DIY assistive technology sharing30Shaun Kane @shaunkaneDesigning user interfaces for power wheelchairs [2013-]Power wheelchairs can be cumbersome

How can we make using a power wheelchair an asset?

Use wheelchair as a computing platform31Collaborators: Patrck A. Carrington,Amy Hurst

Shaun Kane @shaunkaneFormative investigationPhD student Patrick Carrington interned at spinal cord rehabilitation clinic for 3 months

Assisted patients, physicians, therapists

Conducted design activities with patients and physicians32

Shaun Kane @shaunkanePhysical prototypes33

Shaun Kane @shaunkaneBig idea: Chairable technologyMobile and wearable devices designed to accommodate a power wheelchair form factor [Carrington et al., CHI 14]34

Shaun Kane @shaunkaneThe Gest-Rest (ASSETS 2014)An interactive physical prototype

Low cost (estimated components cost < 50 USD)

Extremely flexible inputSupports variable strength and mobility35Shaun Kane @shaunkane36



Shaun Kane @shaunkaneSupporting diverse gestures

39Shaun Kane @shaunkane40

Shaun Kane @shaunkaneTakeawaysHow can we think of individuals abilities (and their assistive technology) as assets?

Real opportunities to combine rapid prototyping, low cost hardware, and gesture recognition41Shaun Kane @shaunkaneCould we design a wheelchair thats so cool that everyone wants one?41ProjectsDesigning eyes-free touch screens


Understanding DIY assistive technology sharing42Shaun Kane @shaunkaneFrom adaptive technology to collaborative work enablingthefuture.org43

thingiverse.comCollaborators: Erin Buehler, Amy Hurst, Abdullah Ali, Kevin Chang, Stacy Branham, Megan Hofmann

Shaun Kane @shaunkaneE-nable has printed more than 1000 hands; 3000 volunteers43Research approachHow are people sharing assistive technology designs online?

What objects? Who makes them? Who are they for?

Conducted a systematic keyword search of thingiverse.com; (363 unique designs)

Surveyed 70 designers44Shaun Kane @shaunkane45



Shaun Kane @shaunkaneFidget rings for adhd, pill box, hands4748


Shaun Kane @shaunkaneFidget rings for adhd, pill box, hands49High-level findingsDesigners made AT for themselves (13) or a friend/family member (24)

Most designers had STEM background

Many designs (40.5%) required some assembly50Shaun Kane @shaunkaneTakeawaysModels for designing, producing, and distributing assistive technology are changing

Great opportunities for collaboration in solving individuals accessibility problems

Limitations in tools available; connections between designers and users51Shaun Kane @shaunkaneBig pictureAccessible technology is a fertile environment for research and design

Early adopters, interesting needs, many technical challenges

Technology designed for extreme use cases can help everyone52Shaun Kane @shaunkaneAccessibility is for everyone53

Shaun Kane @shaunkanehttp://farm4.static.flickr.com/3240/2866391581_99487d347a.jpghttp://i.dailymail.co.uk/i/pix/2012/12/13/article-0-0C014CC3000005DC-908_634x372.jpg

53Accessibility is for everyone54

(Bonfire, Kane et al., UIST 09)(Easy Make Oven, Carrington et al., iConference 15)Shaun Kane @shaunkaneThis is a really rich area for research, understanding how these specialized approaches translate to general technology54Here is the inspirational part of the talk55Shaun Kane @shaunkaneI dont know if this is actually inspirational, but a few things that Ive figured outA sense of what its like to work with meId be happy if you all came to the University of Colorado. Colorados a big state. But in case you havent 55What problems should I work on?56Exciting(will I want to work on this every day?)Challenging(if not, why hasnt someone done it before?)Impactful(is this worth spending my life on?)Shaun Kane @shaunkaneYou can bring yourself into your work(on your own terms)57

Shaun Kane @shaunkaneSerious example, then a more fun onehttps://hampedia.org/wiki/File:Lemelson_OpenHouse.JPG5758

[CSUN 15]Shaun Kane @shaunkaneInteresting problems in making comics accessible58Build it now59Fail fast to succeed sooner. David Kelley, IDEO

Shaun Kane @shaunkaneNot about ignoring the user! Well, you can wait until the end of my talkhttp://static.ibnlive.in.com/pix/slideshow/11-2011/nokia-hits-the/nokia-6030-071111.jpg59Think about how the things that you create impact diversityIn technology, design decisions can strongly affect whether someone can participate

Cant be an afterthought

Consider this in the work that you do (and spread the word)60



Michele Williams, UMBCmichele-williams.comPatrick Carrington, UMBCpatrickcarrington.com

Stacy Branham, UMBCstacybranham.comShaun Kane @shaunkane

63

Visit sunny Boulder, COShaun Kane @shaunkaneThanks!64Shaun Kane [email protected]/shaunkane

Shaun Kane @shaunkane65Shaun Kane @shaunkaneExtra slides66Shaun Kane @shaunkaneGestures as accessible inputMost people can perform some type of body movement (hands, arms, head)

We can develop accessible gesture-based interfaces that match the users ability67Shaun Kane @shaunkane6768
