Click here to load reader
Upload
institute-of-systems-science-national-university-of-singapore
View
370
Download
2
Embed Size (px)
Citation preview
#ISSlearn
#ISSlearn
VOICE COMPUTING
THE NEXT DIGITAL DISRUPTION
11 Aug 2017 / Prasanna Veerapandi & Cjin Pheow Lee
© 2017 National University of Singapore. All Rights Reserved
#ISSlearn 2
Evolution of Human Computer Interaction (HCI)
#ISSlearn 3
HCI Evolution Timeline
1890
s
1930
s
1960
s
1980
s
1990
s
201
5
PUNCH CARD
KEYBOARD PEN
TOUCHMOUSE
VOICE
#ISSlearn 4
1890s - Punch Card
#ISSlearn 5
1930s - Keyboard
#ISSlearn 6
1960s - Mouse
#ISSlearn 7
1980s - Pen
#ISSlearn 8
1990s - Touch
#ISSlearn 9
2015 - Voice
#ISSlearn 10
2017 and beyond
#ISSlearn#ISSlearn
Introducing Alexa
Alexa is an intelligent personal assistantdeveloped by Amazon, made popular by the
Amazon Echo and the Amazon Echo Dot devices developed by Amazon Lab126. It is capable of
voice interaction, music playback, making to-do lists, setting alarms, streaming podcasts, playing audiobooks, and providing weather, traffic, and
other real time information, such as news. Alexa can also control smart devices using itself as a home
automation system.
11
#ISSlearn 12
Alexa Skills
#ISSlearn13
#ISSlearn
#ISSlearn15
#ISSlearn 16
8 Alexa Skills by StackUp Community and Counting
#ISSlearn
#ISSlearn
Amazon Alexa Skill Promo
https://developer.amazon.com/alexa-skills-kit/alexa-developer-skill-promotion
Terms and Conditions apply.
Publish a Skill by 31st August 2017Get an Echo Dot & Dev Swag
17
#ISSlearn
Building Blocks of an Alexa Custom Skill
18
#ISSlearn#ISSlearn
Intents
Intents represent what users can ask your skill to do. Your skill might help plan a trip, get a status, tell a joke, or attack a monster — these are intents.
A skill for planning a trip might have five intents, such as PlanATripIntent, BookTheTripIntent, StopIntent, CancelIntent, and HelpIntent.
19
#ISSlearn#ISSlearn
Sample Utterances
An utterance is what a person says to Alexa that invokes an intent. Utterances are made
up of keyword commands, natural speech sounds like filler words, and slots for
information that varies. One of the most important aspects of designing a voice
experience is defining the range of what people might say.
20
#ISSlearn 21
Sample utterances corresponding to PlanATripIntent
“I’d like to go on a trip”“Let’s begin planning a trip”“Plan a trip”“I need a vacation”
#ISSlearn#ISSlearn
Slots
Slots allow people to specify variable parts of an utterance, for example, city or date. Slots are commonly used in task and information-focused skills. There are two classes of slot types: 1. Amazon’s Built-in Slot Type, like AMAZON.DATE
and AMAZON.NUMBER2. User’s Custom Slot Type
22
#ISSlearn#ISSlearn
Invocation Name
A name that identifies the skill. The user includes this name when invoking and initiating a conversation with your skill.Examples of invocation names:
• plan my trip• command ninja• stackup greeter• s. g. weather
23
#ISSlearn#ISSlearn
Invocation Name Guidelines
• No infringement of Intellectual Property (IP)• No one-word name, unless you own the Brand, like
starbucks or uber or even lion• Must not contain the wake words “alexa”, “amazon”,
“echo”, “computer”, or the words “skill” or “app”• Lower-case alphabetic characters, spaces between
words, possessive apostrophes, periods used in abbreviations
• Specific to the functionality of the skill, like soccer trivia, sleep sounds, vocabulary builder
• Fit smoothly with at least one of the Alexa skill launch phrases (for example, “launch”, “ask”, “tell”, “load”, “begin”) to allow customers to naturally invoke the skill
24
#ISSlearn 25
What Users Say
#ISSlearn
#ISSlearn#ISSlearn
Skill Service
A cloud-based service that accepts these intents as structured requests and then
acts upon them. This service must be accessible over the Internet. You provide an endpoint for your service when configuring
the skill.
27
#ISSlearn#ISSlearn
Skill Interface
A Voice User Interface (VUI) configuration that brings all of the above together so that Alexa can route requests to the service for
your skill. You create this configuration in the developer portal.
28
#ISSlearn 29
A Skill’s Typical Request Lifecycle
#ISSlearn 30
User Interaction Flow
#ISSlearn
How to Create a Custom Skill
31
Using the Alexa Skills Kit (ASK)
#ISSlearn#ISSlearn
Requirements
1.Computer ready for Node.js development2.Knowledge of JavaScript or Python
programming3.Developer’s account at
https://developer.amazon.com4.An account with Amazon Web Services5.An idea for a skill
32
#ISSlearn
Alexa Skills Kit
ASK is a collection of self-service APIs, tools, documentation, and code samples that makes it fast and easy for you to add skills to Alexa.
33
#ISSlearn
Alexa Voice Service
The Alexa Voice Service (AVS) allows you to integrate Alexa’s built-in voice capabilities into your connected products.
34
#ISSlearn 3535
CODING EXERCISE
#ISSlearn#ISSlearn
Creating Custom Skill
● Get started with Alexa:
https://developer.amazon.com/edw/home.html
● Choose “Alexa Skills Kit”
● Skill Information
○ Skill Type: Choose Custom Interaction Model
○ Name: Learning Day Greeter
○ Invocation Name: learning day greeter
○ Answer “No” to all other fields for now
36
#ISSlearn#ISSlearn
Creating Custom Skill
●Interaction Model○ Launch Skills Builder BETA (GUI)○ Define using JSON and Text
■ Intent Schema: The schema of user intents in JSON format
■ Custom Slot Types (Optional)■ Sample Utterances: These are what people say to
interact with your skill. Type in all the ways that people can invoke the intents.
37
#ISSlearn#ISSlearn
Intent Schema
{"intents": [{"intent": "WelcomeGreetingIntent"
},{"intent": "FarewellIntent","slots": [{"name": "Guest","type": "AMAZON.US_FIRST_NAME"
}]
}]
}
Also to include Help, Cancel, Stop and Unhandled intents.
38
#ISSlearn#ISSlearn
Sample Utterances
WelcomeGreetingIntent helloWelcomeGreetingIntent hiWelcomeGreetingIntent I'm hereWelcomeGreetingIntent knock knockWelcomeGreetingIntent anyone's homeFarewellIntent say goodbye to {Guest}FarewellIntent {Guest} is leaving
39
#ISSlearn#ISSlearn
Creating Custom Skill
●Configuration○ Endpoint
■ AWS Lambda ARN (Amazon Resource Name)● Choose “North America” as region● Key in Lambda ARN in text box (after creating the
Lambda function)
■ HTTPS (Self-hosted secured service)
○ Account Linking: No○ Permissions (Don’t select for now)
40
#ISSlearn#ISSlearn
Creating Custom Skill
• Initialize a new Node.js app using: npm init• Install the node package “alexa-sdk”• Patch node_modules/alexa-sdk/lib/alexa.js
line 131• Create starting script “index.js” and code the
handlers for the various intents• Zip the project directory into index.zip:
• zip -r ../index.zip ./*
41
#ISSlearn#ISSlearn
Creating Custom Skill
Upload to AWS Lambda• Go to https://console.aws.amazon.com/lambda/home• Click “Create a Lambda function”• Under “Select blueprint”, select Runtime “Node.js 6.10” and
blueprint “Blank Function”• Under “Configure triggers”, select “Alexa Skills Kit”• Under “Configure function”, specify a name for the function and
select “Node.js 6.10” as Runtime.• Under Lambda function code, select Code entry type as
“Upload a .ZIP file”. Click “Package function” to upload “index.zip”
• Under “Lambda function handler and role”, leave Handler as “index.handler”, Role as “Create a custom role”, IAM Role as “Create a new IAM Role. Then click Allow at the bottom right.
• Click “Create function” and note the ARN.Back on Alexa skill configuration, specify this AWS Lambda ARN under the North America region textbox.
42
#ISSlearn#ISSlearn
Testing the Custom Skill
• Test• Under Service Simulator, enter an utterance and
click “Ask StackUp Greeter”. See the Lambda Request JSON and the Lambda Response JSON.
• Utterances to try:• “Hi”• “Knock knock”• “Chris is leaving”• “Bid farewell to Susan”• “Help”
43
#ISSlearn 44
Alexa Apps
Amazon AlexaReverb for
Amazon Alexa
http://alexa.amazon.com/
#ISSlearn 45
ADVANCED TOPICS
45
#ISSlearn 46
Providing a Visual Component for Your Skill with Home Cards
• Interaction between the user and a physical Alexa device can include Home Cards
• Cards are displayed on the Amazon Alexa App, the companion app running on mobile devices or on the web
• Cards can contain graphics and verbose text responses
• Cards enhance the voice interaction
#ISSlearn#ISSlearn
Including a Card in Your Skill’s Response
Instead of:this.emit(':ask',…) or this.emit(':tell',…)
Use:this.emit(':askWithCard',…) or
this.emit(':tellWithCard',…)
47
#ISSlearn#ISSlearn
Responding with a Card example
var cardTitle = 'Welcome';var cardContent = 'Welcome to the Learning
Day 2017!';var cardImages = {
smallImageUrl: 'https://pic_720x480.png',largeImageUrl: 'https://pic1200x800.jpg'
};this.emit(':tellWithCard', speechOutput,
cardTitle, cardContent, cardImages);
48
#ISSlearn#ISSlearn
Advanced Features
• Custom Slots - define your own slot type and sample values
• Account Linking - connect the identity of the end user with a user in another system via the Alexa App
• Dialog model - multi-turn conversation between your skill and the user to collect all the information needed to fulfill each intent
• State-machine based intent handling - define different event handlers based on the current state of the skill
• Persisting Skill Attributes through DynamoDB -remembering the user across different interaction sessions
49
#ISSlearn#ISSlearn
Submitting an Alexa Skill for Certification
Submission Checklist❏ Policy guidelines❏ Security requirements❏ Functional tests❏ Voice interface and user experience tests
50
#ISSlearn#ISSlearn
Policy Guidelines
• Trademarks, IP, Brands• No child-directed skills (under age 13)• No compensation or rewards - no vote buying• No advertising or promotional messaging• No inappropriate or illegal content
51
#ISSlearn#ISSlearn
Security Requirements
Skills hosted as Lambda functions• Verify that requests are intended for your service
- check Application IDSkills hosted as web services on your own endpoint• Possess a valid & trusted certificate (SSL)• Verify incoming requests were sent by the Alexa
service• Verify that requests are intended for your service
- check Application IDSkills with Account Linking
52
#ISSlearn#ISSlearn
Functional Tests
The functional tests verify that:• The skill’s basic functionality matches the
information displayed on the skill’s detail card in the Amazon Alexa app
• The skill’s core functionality works and provides useful home cards to the Amazon Alexa app
53
#ISSlearn#ISSlearn
Voice Interface and User Experience TestsVoice interface and user experience testing focuses on:
• Testing the user experience to ensure that the skill is aligned with several key features of Alexa that help create a great experience for customers.
• Reviewing the intent schema, the set of sample utterances, and the list of values for any custom slot types you have defined to ensure that they are correct, complete, and adhere to voice design best practices.
54
#ISSlearn#ISSlearn
Tips & Avoid Common Certification Pitfalls
Tips• Short and concise utterances and prompts• Use of reprompt texts (after a moment of 8s silence)• Implement the Amazon.HelpIntent to assist user• Implement the Amazon.StopIntent/CancelIntent to allow user to
stop session anytime • Use Cards that are sent to the Alexa App as a record of the
user’s session with the skill with text and images
Avoid Common Certification Pitfalls
• Verify skill’s Invocation Name meets the requirements• Implement all required intents• Properly close the session• Trusted SSL Cert if self hosting skill service• Tests using service emulator and on Reverb app
55
#ISSlearn
#ISSlearn
Amazon Alexa Skill Promo
https://developer.amazon.com/alexa-skills-kit/alexa-developer-skill-promotion
Terms and Conditions apply.
Publish a Skill by 31st August 2017Get an Echo Dot & Dev Swag
56
#ISSlearn 57
THANK YOU ☺
[email protected] - [email protected]
57© 2017 National University of Singapore. All Rights Reserved
#ISSlearn 58
RESOURCES
58
#ISSlearn
#ISSlearn#ISSlearn
Resources you can tap on
Official Amazon Alexa Resources
• https://developer.amazon.com/alexa-skills-kit• https://github.com/alexa/alexa-skills-kit-sdk-
for-nodejs• https://www.youtube.com/channel/UCbx0SPp
WT6yB7_yY_ik7pmgHCIVoice Community (let’s build one together)
• https://github.com/HCIVoice• https://hcivoicecommunity.slack.com/
Blogs
• https://www.voicebot.ai/
60