Version History · Web viewThe delegate method recognitionResult will provide the recognized word including non-final hypothesis. func recognitionResult(withText text: String?, isFinal:

Installation and Usage of Orbita Voice SDK

- Technical Document

Version 1.1

Prepared by Orbita

October 26, 2018

Confidential – Orbita Voice SDK

Version History

Ver. No.

Ver. Date Revised By

Description Filename

1.0 19-Jun-2018 Initial Version

Technical Document on installation and usage of Orbita Voice SDK– Orbita

Orbita Voice SDK Document

1.1 October 26, 2018 Orbita Technical Document on installation and usage of Orbita Voice SDK– Orbita

Orbita_Voice_SDK_Document_V1.1

1Confidential – Orbita Voice SDK

Table of ContentsVersion History.........................................................................................................................1

1 Introduction.......................................................................................................................3

2 Prerequisites.....................................................................................................................3

3 Get started with Orbita Voice SDK Installation.................................................................3

3.1 Creating a new project in Xcode................................................................................3

3.2 Import the SDK files to the project.............................................................................6

3.3 Pod Installation..........................................................................................................8

3.4 Build procedure..........................................................................................................9

4 Using the SDK................................................................................................................10

4.1 Login to Orbita services...........................................................................................10

4.2 Using SpeechAudioProcessor Class for Audio Processing.....................................11

4.2.1 Start recognition task........................................................................................11

4.2.2 Post Utterance to Orbita Voice server..............................................................12

4.3 Play Audio Content..................................................................................................14

4.3.1 To play the audio content from Orbita voice service........................................14

4.3.2 To synthesize the text from Orbita voice service to speech (TTS)...................14


1 IntroductionThe Orbita Voice SDK will enable the integration of Orbita Voice services across iOS apps. Development of services such as audio processing, speech to text conversion from the scratch can be time consuming. For this reason, Orbita has developed the SDK that can integrate all the voice related services that a business requires, for easy and quick implementation.

2 Prerequisites- iOS 10.0 or above- Xcode 9.0 or above

3 Get started with Orbita Voice SDK Installation

3.1 Creating a new project in Xcode

1. Open Xcode.2. Select “Project” from the File menu – File - New


3. Select “Single View App” (Default selection) and click “next”

4. Fill in the Product name and click “next”

5. Select a directory where the project must be created and select “Create”


6. A new Project will be created with “Product name” that was given in the previous step.


3.2 Import the SDK files to the project

1. Unzip the SDK.zip file.2. Copy the unzipped folder named “SDK” and paste it in the project’s directory (as

shown in the screenshot below).

3. Move the ‘googleapis.podspec’ file and ‘google’ folder from the SDK folder to project directory.

From “SDK” folder

To “Project name” folder


4. In Xcode, right click on the project directory and select “Add files to <Project name>”.

5. Select the folder named “SDK” which is added to Xcode and click on “Add”


3.3 Pod Installation

1. Go to Terminal and change the directory to project directory

“cd /Users/username/Desktop/<Project name>”

2. Type “pod init” and press enter – this will create a Podfile in the project directory3. Open Podfile in a text editor and paste the below lines after “use_frameworks!”

pod 'Alamofire'

pod 'googleapis', :path => ‘.'

4. Save the Podfile.5. Go to terminal and type “pod install” and press enter. This will install all the

dependencies that were added, in the previous steps, into the Podfile.6. The files that are highlighted in the below screenshot will be created.


7. Close the project in Xcode and open ‘<Project name>.xcworkspace’ file from the project directory

3.4 Build procedure

1. Go to Product – Run from the file menu. Build would run into errors in google STT. Please follow the below link to resolve it.

https://github.com/GoogleCloudPlatform/ios-docs-samples/blob/master/speech/Swift/Speech-gRPC-Streaming/BUILDFIXES

In the above-mentioned web page, ignore the first step and proceed with the second step.

2. Run the project from product – Run from file menu till you see no error in the build3. Replace the Google API key in SpeechRecognitionService.swift




4 Using the SDK

4.1 Login to Orbita services

let servicePayload = [“username":"","password":""]

WebServices.sharedManager.LoginRequest(servicePayload as NSDictionary, success: { (response, requestName, status) in}) { (error, requestName,status) in}

The SDK will decode the java web token and save the authorization token in the userdefaults.

The response will be the decoded JWT.

Sample Response

{ "_id": "58e72a68343f94a80803e3ba", "roles": [ "admin" ], "firstName": "Alpesh", "lastName": "Patel", "avatarSrc": "img/avatar0.jpg", "attributes": { "oauthSettings": { "alexa": { "userId": "M2G739YPXVIGU8", "access": { "expiresAt": "2018-05-15T18:36:55+05:30", "tokenSecret": "Atza|IwEBIAWQ8n0JijjapDCAJ6ArugipV59D3NHROLV5vyLBXdJQrTFqhk09-101K1Z7Xy_OdGZFH0w9_Wt5DExWhUq2_M6crnMARDilsUeaMydVdytZnWQ4sQBbk4vk7yDgtnduSE_IBAEKV6hGn3ulgoKihs83leKlRybf17lj3ABrs6XFhZ9NrlYAhurZvrDWBVsPzgyXqcLzJXcPUKILXal_0Dxg3zV9HmtrZJcTPEc1UtrA8eFlEyDv0xPXHxbkokJSjfEMa4V0KXuvs-eEn64sLpThlhlp79qAeVqH9JGuEH38jPrJPx7IiKmR8M1YifghP2-i6vdHj1c3DroY7H0r4ImoG52Thmz0AKg9qU3H0cjc3nw70YZHgTWw6ugjP-IYgGiFc55kBKEnYUeq6Q0svoas1AxmigmwJoj_263k6uII68K6FPA4-K6vqG0h0xRh2JMwgwUPEVsdk4jcouaECqNURsUDafGSiuB_mrTZqk4ZzuOPRkYxmqJTW4GWKD7Y2BuaVvUVXA_JNkDzaUneFDoGXrEZMVx83ZH_eVNkXE4fntcO2r1wCAz1RNvQC-DbzpIYo2PsFHEtfX-idXbpwfFL9WniweKp001NP3gj_5fc8Q", "token": "Atzr|IwEBIIp5Jk9CqAXJUzdUGO9joTffDL0_1FreAoOSzBC_3GJPCU7C_eci82IbD9GMB


18Quhj8ZYLK57etBm7n5_ezuf8vzihNPFJYIiJ8OXjeKGVilgM93CMXPxuPfXrgm0YXywb4K605uOOhTN2QmaXxwZdj0yVa0hSH_CyiVRhmI9CHuP9NCIs0RBDj5dH_Gm0Hjh1XrVwRq_b2BM2UiesR8PHYGAjXEgSrQY3nc0K_C5wvu9NNOyqQrUSmAOPkmoLXqacc73wybzDNU1GHBScmd-PVs9DJ0trPJP38-Z0eSwIAs4wlwHd0fiqpb5q4ZfDVxPxKj8hSo2Sgb1EdLzWhvCcGYiWjhRUfilo12OjHGHDAC_suuHST3xLeuGXyRIL9QtSx3NVviFlCnf2nQhYxBr9tpvMVNmZ-plC9uTL5m4beAhKHUnlAKFYBKg110qLh4HR76JeLiMAuXSohIMy45ydkRl62X5k5va4RI1xUy_G1F_etCS0d_FIqD0KYj6LA-eWuxOLdQV09e0I_rJOS534dUAzh6vKC7byBXORo2CsdYAz80GirjM-I1WrZKPo4fX6WIr0j6_kT9pUrZbgvtRKm" } } }, "email": "[email protected]", "title": "CTO", "pinSecurityExpired": "2018-05-03T12:07:46.586Z" }, "token": "", "personaType": "592e9e508c0f67381784b15a", "personaProfile": { "username": "[email protected]", "firstName": "Alpesh", "lastName": "Patel", "securityPin": "2548", "securityPinInterval": "1", "timezone": "America/Chicago" }, "iat": 1526480494, "exp": 1526494894}

4.2 Using SpeechAudioProcessor Class for Audio Processing

Assign SpeechAudioProcessor.shared.delegate = self

4.2.1 Start recognition task

We need to request authorization from user to use microphone for voice recognition.

SpeechAudioProcessor.shared.RequestAuthorization()

The AuthorizationStatus delegate method returns true for granted request and false for declined request

func AuthorizationStatus(status: Bool) {}On successful authorization status, we can start recognition task.

SpeechAudioProcessor.shared.startRecognition()


To stop recognition, we can use

SpeechAudioProcessor.shared.StopRecording()

the delegate method recognitionCancel() is called when the task has been cancelled, either by client app, the user, or the system. The delegate method recognitionResult will provide the recognized word including non-final hypothesis.

func recognitionResult(withText text: String?, isFinal: Bool) {}

isFinal = true - Called only for final recognitions of utterances. No more about the utterance will be reported.

isFinal = False - Called for all recognitions, including non-final hypothesis.

text = recognized text from speech recognizer.

4.2.2 Post Utterance to Orbita Voice server

let requestToken = UserDefaults.standard.value(forKey: "orbitaRequestToken") as! String let serviceDict = ["utterance":UtteranceString,"sessionId":sessionID,"orbitaToken":requestToken] WebServices.sharedManager.PostVoiceRecognitionService(params:serviceDict as NSDictionary, success: { (response, requestName, status) in

})

{ (error, requestName, status) in

}

sessionID - Random 16 digit stringUtteranceString - string from device voice recognition result for Orbita voice processing.

Sample Request:

{“utterance":"launch","sessionID":"wwerfnr9lsdnfskfspj"}

Response :

{ "orbitaPayload": { "payload": {


"multiagent": { "voice": { "sayText": "Hello World! <break strength=\"none\" time=\"5s\"/> How can I help? hello how are you ", "rePrompt": "how can i help?<audio src=\"https://s3.amazonaws.com/orbitahealth/clients/sandbox5/assets/editor/audio/SampleAudio_0.4mbd7504e20-5ec6-11e8-91c4-cf90c900530e.mp3\"/>" }, "chat": { "chatText": "Hello World! How can I help?", "rePrompt": "how can I help you?" }, "screen": { "shortTitle": "Hi", "longTitle": "Hi there", "body": "Hello World! How can I help?", "smallImage": "", "largeImage": "" }, "buttons": { "type": "", "name": "buttons", "choices": [] } } }, "type": "4", "name": "orbita" }, "text": "Hello World! How can I help? hello how are you ", "reprompt": "Hello World! How can I help? hello how are you ", "sessionEnd": false, "replaceWord": { "two": [ "to", "too" ], "six": [ "sex" ], "ten": [ "tan", "tin" ] },“sayTextAudio”:[],“audioContent”:[],”repromptAudio”:[]}


4.3 Play Audio Content 4.3.1 To play the audio content from Orbita voice service. var audioContent = [String:Any]()audioContent = serviceResponse.value(forKey: "sayTextAudio") as! [String:Any]var data = NSData() if(audioContent["data"] is [String]){ let dataArray = audioContent["data"] as! [String] data = NSData(base64Encoded: dataArray[0], options: NSData.Base64DecodingOptions.ignoreUnknownCharacters)! }

SpeechAudioProcessor.shared.PlayAudio(data: data as Data)

serviceResponse - Response from orbita voice service.audioContent - property name for audio file in response JSON.We will be receiving byte array of string which needs to be base 64 decoded.

Delegate method AudioFileDidFinishPlaying will be called after the audio has finished playing.

we can use stopAudio to stop audio player anytime while the audio player is playing. SpeechAudioProcessor.shared.StopAudio()

4.3.2 To synthesize the text from Orbita voice service to speech (TTS)var audioText = serviceResponse.value(forKey: "text") as! StringSpeechAudioProcessor.shared.SpeakText(string: audioText)

Delegate method SpeechSynthesizer(didFinishSpeaking text: String) will be called after the SpeechSynthesizer has finished playing the text.

We can use stopAudio to stop SpeechSynthesizer anytime while the audio player is playing.SpeechAudioProcessor.shared.StopAudio()

Documents

Version History · Web viewThe delegate method recognitionResult will provide the recognized word including non-final hypothesis. func recognitionResult(withText text: String?, isFinal: