Upload
diannepatricia
View
58
Download
0
Embed Size (px)
Citation preview
Cognitive Assistant for The Real-world Accessibility
Chieko Asakawa IBM Fellow, IBM ResearchIBM Distinguished Service Professor, Carnegie Mellon University
© 2011 IBM Corporation2
Cognitive Ability of Visually Impaired People
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
0.8
1.0
1.2
1.4
1.6
1.8
2.0
2.2
2.4
2.6
2.8
3.0
3.2
3.4
3.6
音声速度(倍速)
再生
率 (%)
A
B
C
D
E
F
G
Reca
ll Rat
e (%
)
Speech speed
X X X X X X X X X X X X X XX
3 ©2016IBMCorporation
4 ©2016IBMCorporation
Information accessibility Mobility
Braille Digitalization
Paper Braille TypewriterComparison:
Braille and Printed Japanese Dictionary
Digital Braille System
Home Page Reader
6 © 2016 IBM Corporation
nThe first practical voice browsers in the world
nBecame a product in 1997 and translated into 11 languages
IBM Home Page Reader (1999) Japanese, Italian, French, German, Spanish, English (U.S.,
U.K.)
“Home Page Reader is my small window to the world. I can read, write, and access information.I can do everything to participate in society.”
— comment from a user, 1997
©2016IBMCorporation
Smart phone demo
©2016IBMCorporation
Exponential Increase of Accessible Information (Conceptual Chart)
©2016IBMCorporation
Paper BrailleDigital Braille
Voice Web access
1980 20001990 2010
Amou
nt o
f ac
cess
ible
info
rmat
ion
9
https://www.youtube.com/watch?v=cTHa_5Y5IA4
10
https://theses.lib.vt.edu/theses/available/etd-05072012-161514/unrestricted/DAngio_PC_D_2012_1.pdf
http://www.ted.com/talks/dennis_hong_making_a_car_for_blind_drivers
Blind Driver Challenge at the Rolex 24 At Daytona with Audio Description
Uber Self-driving Car Pilot
11 ©2016IBMCorporation
Walk by myself Finding discount signsFinding restaurantsShopping
The Second Challenge: Real World Accessibility
©2016IBMCorporation12
Cognitive Assistant
13
Augment missing or weakened abilities by the power of cognitive computing.
©2016IBMCorporation
“甦るヒーローライブラリー第3集 光速エスパー”, TCエンタテインメントより
Chika
14 ©2016IBMCorporation
©2016IBMCorporation
https://www.ted.com/talks/chieko_asakawa_how_new_technology_helps_blind_people_explore_the_world
Foundation of Cognitive Assistant
©2016IBMCorporation16
LocalizationRadio-wave
VisionDead reckoning
RecognitionObjectsPeople
Environment
KnowledgePersonal
Social mediaEncyclopedic
InteractionSpeechGesture
Sonification
Localization
Quasi-zenith satellitesfor Japan
2018
http://qzss.go.jp/technical/technology/tech01_orbit.html
Accurate indoor localizationBLE Beacons + Dead reckoning
WIFI
Recognition of Environment
Vision-based EnvironmentRecognition
Infrared Depth Sensor on smartphonesGoogle Tango
https://youtu.be/Qe10ExwzCqk
Recognition of Things
Watson Vision API
coffeeespresso
wineglass
glasses potterydish
Oil filterChicken soup
Pen sharpener Spice bottle
Cristmas socks Cheetos
Beer bottle Diet coke
Photos for learning taken by blind subjects
Incorrect resultsBefore leaning
Correct resultsAfter learning
Personal Object Recognition
Honorable mention at CHI2017
©2016IBMCorporation
Knowledge – Importance of Open data
Detailed pedestrian maps
Crowd prediction
RestaurantShops
Items
Temporal changes
21
Accessibility information
Interaction
©2016IBMCorporation
Approaching to human accuracy
http://blogs.microsoft.com/next/2016/10/18/historic-achievement-microsoft-researchers-reach-human-parity-conversational-speech-recognition/#sm.0019gj85a12umd89ys32kew39p8d6
Autonomous Bus with Voice Dialog
https://www-03.ibm.com/press/jp/ja/pressrelease/50250.wss
デモ
©2016IBMCorporation
Cognitive Assisting Robot
Manuela Veloso is leading the CoBot project. On November 18, 2014, the CoBot robots jointly reached 1,000km of autonomous localization and navigation.
©2016IBMCorporation
Sep 2016 - August 2019 Awarded Amount: $1millionInvestigator:• Kris Kitani, Computer Vision, Robotics Institute, CMU• Manuela Veloso, Robotics, Robotics Institute, CMUAdvisor:• Chieko Asakawa, IBM Fellow, Robotics Institute, CMU
National Robotics InitiativeA Cognitive Navigation Assistant for the Blind
25
Implementation of a navigation assistant that uses a collection of sensing modalities and algorithms to guide a blind person through the knowledge landscape of an unfamiliar environment.
World Robot Summit
©2016IBMCorporation
Industry robots Services robots Disaster response robots
26
Japanese government is planning a robot competition event in 2020, the same year, Olympics and Paralympics will be held in Tokyo.
http://www.nedo.go.jp/robot/about.html http://www.miraikan.jst.go.jp/exhibition/future/robot/robotworld.html#slide1
https://www-robotics.jpl.nasa.gov/tasks/taskImage.cfm?TaskID=236&tdaID=700043&Image=501
18th century, the mathematician Euler authored a leading textbook on mathematics after he became blind.
Captions for TV programs were invented for deaf people, but are now widely used in sports bars and to help immigrants, etc.
Based on Self-standing wheelchair.
2011, voice dialogtechnologies for smartphones
TelephoneKeyboard
Voice recognition
Voice dialogTV Captions
Voice synthesis and OCR
Modern Mathematics
Autonomic vehicles
Segway
“Blind driving challenges”
Early use of keyboards was a to help people with hand-and-motor disabilities in writing
First consumer product was a reading assistant for the blind in the 1970s.
1876, Alexander Graham Bell invented the phone after teaching hearing impaired people.
Innovations Flourish From Accessibility NeedsSmart
Machines
27
17th century, Dom Perignon. Champagne
©2016IBMCorporation
1960sTalking Typewriter
1960s Talking Typewriter
1975 1403 Braille Printer
1984 Talking 3270 Terminal
1990 VoiceType™
2008 Social Accessibility
1998 ViaVoice®
2000 Accessibility Center
1997 Home Page Reader
1988 ScreenReader/DOS
2004 aDesigner
2007 Eclipse Accessibility Tools Framework
2009 Rich Internet Accessibility Technologies
1914 First IBMer with disabilities
IBM History of Accessibility
28 ©2016IBMCorporation
2012 Brazil Accessible Vocational Training Project
2011 Senior Workforce Project 1984Talking 3270 Terminal
©2016IBMCorporation
2010 20302020
Amou
nt o
f Ac
cess
ible
real
wor
ld in
form
atio
n(c
once
ptua
l num
ber)
Future: Real-world Accessibility – Exponential Growth
Cognitive Assistant Lab @ CMU
Smartphone accessibility
Standardization?Core technology development
Mobile Cognitive ?
Apple Siri
Microsoft Seeing AI project
Google Tango Phone
Tokyo NihonbashiDemo
Watson Vision API
Adoption?Early days
2016Dec.
Tokyo OlympicsParalympics