•Making Great SiriKit Experiences - Apple Inc.€¦ · •Making Great SiriKit Experiences ......

#WWDC17

José Angel Castillo Sanchez, SiriKit Engineering Rohit Dasari, SiriKit Engineering

•Making Great SiriKit Experiences • Session 228

App Frameworks

•Contact resolution •Security •Custom vocabulary •UI testing

•Contact Resolution

UnicornChat

extension INPerson : INSpeakable { open var siriMatches: [INPerson]? { get } }

Contact Resolution INPerson.siriMatches

extension INPerson : INSpeakable { open var siriMatches: [INPerson]? { get } }

“Text John hello on UnicornChat”

“Text John hello on UnicornChat”Siri

Application Extension

resolveRecipients

INPerson Name: John Baily

INPerson Name: John Appleseed

“John Baily”“John Baily”

resolveRecipients

“John Baily”Siri

“John Baily”

resolveRecipients

“John Baily”

resolveRecipients

“John Baily”

Application Extension INPerson Name: John Baily

resolveRecipients

But what if…

“John Baily”

“John Baily”Siri

“John Baily”

resolveRecipients

“John Baily”

resolveRecipients

“John Baily”

resolveRecipients

mobile

“mobile”

“mobile”Siri

“mobile”

resolveRecipients

“mobile”

resolveRecipients

mobile

“mobile”

resolveRecipients

mobile

func resolveRecipients(forSendMessage intent: INSendMessageIntent, with completion: @escaping ([INPersonResolutionResult]) -> Void) { guard let recipients = intent.recipients, !recipients.isEmpty else { completion([.needsValue()]) return } let results = recipients.map { recipient in database.resolvePerson(for: recipient.user) } completion(results) }

•Demo

“Read my messages on UnicornChat”

open class INMessage : NSObject, NSCopying, NSSecureCoding {

open var identifier: String { get }

@available(iOS 11.0, *) open var conversationIdentifier: String? { get }

open var content: String? { get } open var dateSent: Date? { get }

@NSCopying open var sender: INPerson? { get }

open var recipients: [INPerson]? { get } }

open class INMessage : NSObject, NSCopying, NSSecureCoding {

open var identifier: String { get }

@available(iOS 11.0, *) open var conversationIdentifier: String? { get }

open var content: String? { get } open var dateSent: Date? { get }

@NSCopying open var sender: INPerson? { get }

open var recipients: [INPerson]? { get } }

open class INSendMessageIntent : INIntent {

open var recipients: [INPerson]? { get }

open var content: String? { get } open var groupName: String? { get } open var serviceName: String? { get }

@available(iOS 11.0, *) open var conversationIdentifier: String? { get } @NSCopying open var sender: INPerson? { get } }

open class INSendMessageIntent : INIntent {

open var recipients: [INPerson]? { get }

open var content: String? { get } open var groupName: String? { get } open var serviceName: String? { get }

@available(iOS 11.0, *) open var conversationIdentifier: String? { get } @NSCopying open var sender: INPerson? { get } }

Contact Resolution Conversation identifier

“Read my messages on UnicornChat”Siri

SearchForMessagesIntent handle

INSearchForMessagesIntent

INMessageconversationIdentifier

Application Extension INMessageconversationIdentifier

“Read my messages on UnicornChat”Siri

SearchMessageIntent handle

Application Extension INMessageconversationIdentifier

“Read my messages on UnicornChat” Would you like to reply?Siri

SearchMessageIntent handle

“Yes, hello”

“Yes, hello”Siri

resolveRecipients

INSendMessageIntentconversationIdentifier

resolveRecipients

INSendMessageIntentconversationIdentifier

Application Extension INSendMessageIntentconversationIdentifier

resolveRecipients

Application Extension INSendMessageIntentconversationIdentifier

resolveRecipients

Related Session

Enabling Your App for CarPlay WWDC 2017 Video

public enum INStartVideoCallIntentResponseCode : Int { case unspecified case ready case continueInApp case failure case failureRequiringAppLaunch case failureAppConfigurationRequired case failureCallingServiceNotAvailable

@available(iOS 11.0, *) case failureInvalidNumber }

public enum INStartVideoCallIntentResponseCode : Int { case unspecified case ready case continueInApp case failure case failureRequiringAppLaunch case failureAppConfigurationRequired case failureCallingServiceNotAvailable

@available(iOS 11.0, *) case failureInvalidNumber }

•INPerson.siriMatches

•INPerson.siriMatches•Multiple handles

•INPerson.siriMatches•Multiple handles•Conversation identifier

•INPerson.siriMatches•Multiple handles•Conversation identifier•Response codes

Rohit Dasari, SiriKit Engineering

•Security •Device unlock, authentication, and payments

Security Requiring device unlock

class INSetCarLockStatusIntent : INIntent { var carName: INSpeakableString? { get }

var locked: Bool? { get }}

•Demo

Security LocalAuthentication.framework

Multiple forms of authentication

Multiple forms of authentication• TouchID

Multiple forms of authentication• TouchID• Passcode

Call in your handle method

Siri waits for user input

Keychain and Authentication with Touch ID WWDC 2014

•Custom Vocabulary •User-specific and app global

Custom Vocabulary User-specific

Help Siri understand words unique to users

Help Siri understand words unique to users• Contact, photo album and keyword, car, workout, bank, account, note, list item

Use INVocabulary API

Introducing SiriKit WWDC 2016

Extending your apps with SiriKit WWDC 2016

open class INVocabulary : NSObject { open func setVocabularyStrings(_ vocabulary: NSOrderedSet, of type: INVocabularyStringType)

Custom Vocabulary INVocabulary

“SharedAccount”

“NestEgg”

SharedAccount Object

NestEgg Object

Application

“SharedAccount”

“NestEgg”

NestEgg Object

“SharedAccount”

“NestEgg”

“SharedAccount”

“NestEgg”

Application

NestEgg Object

Application

NestEgg Object

“SharedAccount”

“NestEgg”Siri

Application

NestEgg Object

“SharedAccount”

“NestEgg”Siri

“Show my nest egg balance”

Application

NestEgg Object

“SharedAccount”

“NestEgg”Siri

Application

NestEgg Object

“SharedAccount”

“NestEgg”Siri

Application

NestEgg Object

“SharedAccount”

“NestEgg”Siri

Application

NestEgg Object

“Show my nest egg balance”Siri

“SharedAccount”

“NestEgg”

“Show my NestEgg balance”

Application

NestEgg Object

Siri“SharedAccount”

“NestEgg”NestEgg

Application

NestEgg Object

“Show my NestEgg balance”Siri

“SharedAccount”

“NestEgg”

Application

NestEgg Object

“Show my NestEgg balance”Siri

“SharedAccount”

“NestEgg”

Fuzzy match Object for "NestEgg"

@available(iOS 11.0, *) open func setVocabulary(_ vocabulary: NSOrderedSet, of type: INVocabularyStringType) // vocabulary items can be of any type that conforms to the INSpeakable protocol }

public protocol INSpeakable : NSObjectProtocol { public var spokenPhrase: String { get }

// "NestEgg", "Spyn"

public var pronunciationHint: String? { get } // "nest egg", "spin"

public var vocabularyIdentifier: String? { get } // "nestegg", "object-id:456ABCDEF1237890"

public var alternativeSpeakableMatches: [INSpeakable]? { get } }

SharedAccount <INSpeakable>

NestEgg <INSpeakable>

NestEgg Object

Application

NestEgg Object

Application

NestEgg Object

Application

NestEgg <INSpeakable>Siri

Application

NestEgg Object

Application

NestEgg Object

Application

NestEgg Object

SiriSharedAccount <INSpeakable>

Application

NestEgg Object

Application

NestEgg Object

Application

NestEgg Object

NestEgg <INSpeakable>NestEgg

Application

NestEgg Object

Application

NestEgg Object Look up object by vocabularyIdentifier

Custom Vocabulary App global

Help Siri understand words unique to your app

Common to all of your users

Common to all of your users• Ride name, workout names

Use AppIntentVocabulary.plist

INSpeakable

vocabularyIdentifier

INSpeakable

spokenPhrase

INSpeakable

spokenPhrase

pronunciationHint

•UI Testing

UI Testing

Inside your project in Xcode

UI Testing

Siri in simulator lets you test on multiple devices

UI Testing

Automate your tests to check and maintain quality

UI Testing

Language independence

UI Testing

Language independence

UI Testing in Xcode WWDC 2015

Continuous Integration and Code Coverage in Xcode 7 WWDC 2015

let siriService = XCUIDevice.shared.siriService NEW

let siriService = XCUIDevice.shared.siriServicesiriService.activate(voiceRecognitionText: "Unlock my car")

•Demo

Summary

Contact resolution

Summary

Contact resolution

Security

Summary

Contact resolution

Security

Custom vocabulary

Summary

Contact resolution

Security

Custom vocabulary

UI testing

More Informationhttps://developer.apple.com/wwdc17/228

Related Sessions

What's New in ApplePay and Wallet Executive Ballroom Thursday 5:10PM

Enabling Your App for CarPlay WWDC 2017 Video

Introducing SiriKit WWDC 2016

Extending your apps with SiriKit WWDC 2016

Continuous Integration and Code Coverage in Xcode 7 WWDC 2015

UI Testing in Xcode WWDC 2015

Keychain and Authentication with Touch ID WWDC 2014

SiriKit Lab Technology Lab C Fri 9:00AM–12:00PM

•Making Great SiriKit Experiences - Apple Inc.€¦ · •Making Great SiriKit Experiences ......

Documents

Making Shift Happen - Experiences of Google Apps in ScHARR

SiriKit iOS10

Making Experiences Better with Personalization and Testing

Nurses´ experiences of ethical decision making in nursing

Brochure - Queen Sirikit Centre for Breast Cancer

KKBOX WWDC17 SiriKit and CoreSpotlight - Seraph

School based experiences as contributors to career decision-making

Making Outcome Mapping Work: Evolving Experiences from Around

Making Sense: Experiences with Multi-Sensor Fusion in

· Pqrk Statio Station . Lake Rajada Benjakiti Park Queen Sirikit National Convention Centre C) Sirikit Stock Exchange Station of Thailand (SET) Rama 4 Road Time

Treatment decision making experiences of migrant cancer

Making experiences in india: The events and activations industry

Making website experiences

Queen Sirikit National Institute of Child Healthpthaigastro.org/Document/xebcry55fj1oybjntlfqg2a2A... · Queen Sirikit National Institute of Child Health ... SBP, secondary peritonitis

Bloggers’ Experiences on Women’s Participation in decision making

A Case study: The Queen Sirikit national convention center

MAKING UNFORGETTABLE EXPERIENCES - EACTS · MAKING UNFORGETTABLE EXPERIENCES. The City of Milan 2 • A compact historic center easily explored on foot through cobbled streets lined

•What’s New in SiriKit - Apple Inc. · PDF fileSirisha Yerroju, SiriKit Engineer Tin Tran, SiriKit Engineer •What’s New in SiriKit ... Resume my daily run Workouts Pay the

Sirikit in 1980

EY Making Experiences in India