InterConnect2017

Bring Watson to Your Telephone: Introducing the IBM WebSphere Connect Voice Gateway for Watson

Tom Banks: IBM Offering Manager and Technical EvangelistEmail: tom.banks@uk.ibm.com

1 3/22/17

Brian Pulito: IBM Senior Technical Staff MemberEmail: brian_pulito@us.ibm.comTwitter: @brianpulito

IBM Voice Gateway

Galeal Zino: Founder NetFoundryTata Communications

2 3/22/17

Please noteIBM’s statements regarding its plans, directions, and intent are subject to change or withdrawal without notice at IBM’s sole discretion.

Information regarding potential future products is intended to outline our general product direction and it should not be relied on in making a purchasing decision.

The information mentioned regarding potential future products is not a commitment, promise, or legal obligation to deliver any material, code or functionality. Information about potential future products may not be incorporated into any contract.

The development, release, and timing of any future features or functionality described for our products remains at our sole discretion.

Performance is based on measurements and projections using standard IBM benchmarks in a controlled environment. The actual throughput or performance that any user will experience will vary depending upon many factors, including considerations such as the amount of multiprogramming inthe user’s job stream, the I/O configuration, the storage configuration, and the workload processed. Therefore, no assurance can be given that an individual user will achieve results similar to those stated here.

3

Customer services todayWeb based support is going through a cognitive bot revolution• Making it cheaper and faster to handle common queries• Handling queries 24 hours a day, 365 days a year

Call center challenges• Cost of optimization• Wait time reduction• Improve customer satisfaction• Easier access to internal documents and info

Why not take the cognitive chat bots we are using online, and integrate them into telephone based support?

4 3/22/17

Watson Voice Services Today

• No connectivity to telephony (no support for SIP and RTP)

• Text To Speech and Speech To Text only support Web Sockets or REST

• No IVR integration capabilities (e.g. MRCP)

• No orchestration of services to provide a telephony voice interface

• No integration with SIP trunking or enterprise Session Border Controllers

5 3/22/17

Bringing Watson to telephone based support

Bringing Watson to call centers will:• Increase the number of calls that can be handled automatically

• Lower queue times, saves agents for complex calls

• Improve customer service experience and satisfaction• If the customer’s inquiry or problem is resolved in the first call, only 1% of

those customers are at risk to go to your competitors

• Assist call center agents during a call• Remove the need for agents to be experts on all topics• Get useful info to agents when they need it, quickly and automatically

6 3/22/17

The Value Proposition

• Call center agents take time to train, and have high turnover

• IVRs have 51% satisfaction with customers

• 7.3% of customers who use IVRs consider competitors after their experience

• Watson can provide a better customer experience while handling more

calls without needing to transfer out to a person

7 3/22/17

Introducing the new IBM Voice Gateway

The IBM Voice Gateway for Watson will be released on March 24th, 2017

What is it?

• Next generation (Cognitive) Interactive Voice Response system

• Cloud native, Microservice solution (Docker images)

• Horizontally scalable

• Deployable to on premise networks or the cloud

• Provides a SIP endpoint for connecting to, and orchestrating between, Watson services

8 3/22/17

Customer calls support

Watson Speech To Text, Text To Speech and Conversation services orchestrated by gateway to talk with

customer

Call routed to Voice Gateway via SIP Trunk or

Session Boarder Controller

Transfer out to agent when necessary

Use case 1: Cognitive Self-Service Agent

IBM Voice Gateway

9 3/22/17

Self-Service Agent Demonstration

Text To Speech

Watson Services

Conversation (Dialog, NLC)

Speech To TextSIP Trunk

SIP Orchestrator

Media RelayTwilio used

for demo

IBM Voice Gateway

WS

10 3/22/17

Self-Service Agent: Single Turn Orchestration Flow

Text To Speech

Watson Services

Conversation (Dialog, NLC)

Speech To Text

1) Caller Ask Question

2) Question streamed to STT over WS

3) Text Utterance Returned

4) Text sent as Ask

5) Ask response

6) Ask Response text sent to TTS

7) Synthesized audio returned

8) Audio response streamed to caller

IBM Voice Gateway

11 3/22/17

Conversation Example

12 3/22/17

Use case 2: Cognitive Agent Assistant

Customer calls support and is connected with an agent

Speech To Text used to convert streams to text then analytics used to provide

real-time feedback to agent

SBC forks media to the Voice Gateway

SBC

Agent uses real-time cognitive feedback to help customer

Watson Services

Session Border Controller

IBM Voice Gateway

Agent Assistant Demonstration

Watson Speech To Text

Analytics via Watson(NLC + Retrieve and Rank)

SIPREC

Customer (RTP / G.711)

Transcribed Text published over MQTT

Web Browser

Agent Desktop

MQTT Broker

PowerMedia XMS (Transcoding)

WebRTC Gateway(WebSphere Liberty)

Agent

Agent (RTP / G.711)

Customer

WebRTC

Forking SBC

NetFoundry SIP Trunk

IBM Voice Gateway

Self-Service Agents

Point of sale agent

Pre-screening agent

Credit checking agent

Support agent

Booking agent

Agent Assistants

Point of sale agent assistant

Real-Time trouble shooter

Agent coaches and trainers

Healthcare diagnostic assistant

Transcriptions

Omni-Channel Cognitive Agents

WhatsAppFacebook Messenger

SlackTwitter

Customers connect to BOTs hosted in the Watson Conversation Service

Watson Conversation

Speech To Text, Dialog and Text To Speech

services orchestrated to talk and chat with customer

IBM Voice Gateway

Use case 3: Cognitive IVR

IVRCustomer calls

support

Watson Speech To Text, Text To Speech and Conversation

services, integrated with IVR application server

Call routed to Cognitive Gateway via MRCP 2.0 from the existing IVR

provider

Transfer out to agent when necessary

The MRCP 2.0 standard allows for tight integration into existing IVR solutions and extends the life into the cognitive world.

IVR providers like Avaya, CISCO, Genesys and

Blueworx

MRCP IVR

Gateway

IBM Voice Gateway for WatsonUnder the hood

17 3/22/17

IBM Voice Gateway Microservice Composition

Composed of two co-dependent Docker containers

SIP Orchestrator (SO) Microservice

• Runs on WebSphere Liberty• Handles all SIP signaling• Watson Conversation orchestration• Controls the media relay through a

web socket connection per call• Acts as a SIP endpoint or a SIPREC

server depending on use case

Media Relay (MR)Microservice

• Runs on Node.js• Transcodes between RTP and raw

PCM (for Watson services)• Orchestrates Speech To Text (STT)

and Text To Speech (TTS)• Publishes text utterances to SO• Synthesizes text utterance from SO

IBM Voice Gateway

SIP Trunks,SBCs,

SIP Load Balancer, SIP Endpoints,

Etc.

Media Relay Container(ports 16384-32767)

SIP Orchestrator Container(ports 5060 or 5061)

Offer SDP Answer SDP

Play Text Transcoded Text

Web Socket API

Web Socket API

Node.js

WebSphere LibertySIP/SIPREC

RTP/G.711

Web Socket APIs

1919

Microservices

Reporting (Splunk, REST)

Transcriptions to MQTT Broker

Watson Conversation

Service

Watson Speech To Text Service

Watson Text To Speech Service

REST APIs

MQTT

Microservice Architecture

IBM Voice Gateway

20 3/22/17

Customizing the Voice Gateway when acting as a self-service agent

Gateway API exposed to the Watson Conversation service for:• Call control (hangup, transfer)• SIP header access for logging (Request, To and From URIs, etc.)• Speech control (switch language, STT confidence and alternates)• Music on hold and audio wav file playback to caller• Error message handling (transfers, error utterances) • Contact center integration

State variables (prefixed with vgw) in JSON context

State variables (prefixed with vgw) in JSON contextExamples include vgwHangup, vgwMusicOnHoldURL

Examples include vgwHangupReason, vgwBargeInOccurred

Watson ConversationIBM Voice Gateway

21 3/22/17

Reporting from the Voice GatewayReporting events published from the voice gateway

REST server(Splunk HEC)

Conversation Turn Events *Splunk event headersInput textOutput textIntentsEntitiesContext state variablesConversation IDTurn counterAnd more…

* JSON from Conversation response

Call Detail Record EventsSplunk event headersStart and Stop timeCall lengthSIP details (e.g. SipTo, SipFrom)Number of turnsNumber of barge insWatson transaction latenciesIntents details (#, list of all)And more…

IBM Voice Gateway

22 3/22/17

Securing the IBM Voice Gateway

The voice gateway supports the following:• Connecting to Watson over TLS • Connecting to Watson via a forwarding proxy• Mutual authentication for REST and Web Sockets• Supports for SIPS• Supports trust and key store configuration• Default trust store with certs for Watson connectivity

IBM Voice Gateway

HA using Kubernetes (k8s)

• Kubernetes used for container orchestration and scheduling• A Voice Gateway pod consist of a single SIP Orchestrator and Media Relay container

• Pod deployed in Net Host mode to expose VM IP addresses and ports• Only one Voice Gateway Pod per node (enforced with k8s configuration)

• External SIP Load Balancing of k8s pods• SIP LB statically updated with POD address details • POD must be queisced of all active calls before taking out of service

• Sample kubectrl scripts available here:• https://github.com/WASdev/sample.voice.gateway.for.watson

• IBM Spectrum Conductor for Containers tested• Recommended for an on premise Docker substrate

23

IBM Voice Gateway HA using Kubernetes

24

EtcdConfigManagement

k8sAPIServer

Kube MasterScheduler

WVG POD

MediaRelay

SIPOrchestrator

WVG POD

MediaRelay

SIPOrchestrator

WVG POD

MediaRelay

SIPOrchestrator

Session Border Controller / External SIP Load Balancer SIP signalling

RTP Media

VM / Node VM / Node VM / Node

IBM Voice Gateway

IBM Spectrum Conductor for Containers

3/22/17 25

Distributed Key -value

Install

Proxy

Logging/Reporting

CI/CD

Tenant management

Network

Authentication Authorization

PersistentVolume

GUI

Image Registry Service Discovery

Service Load BalancingTrouble Shooting

HA

Open Source + Integration + Value Add

Resource Management

LSF Community Edition

Application Catalog

Enterprise Integration

26 3/22/17

27 3/22/17

Enterprise integration requirements

• Integrate with contact centers and enterprise telephony systems• Compliance requirements (e.g. HIPAA, PHI, PCI and PII) • Integration with existing Systems of Record• Voice analytics• Voice security (e.g. hybrid connectivity, authentication, etc.)• De-identification of Watson Conversation requests

Enterprise Telephony Integration with Watson

Conversation(Dialog + NLC)

Speech To Text

Text To Speech

Watson Services

Service Orchestration Engine (SOE)

Customer Specific APIs(for PHI, PCI and PII)

Web Socket over TLS

Web Socket over TLS

REST over TLS

Sess

ion

Bord

er C

ontro

ller

Contact Center / IVR

Hybrid-Cloud Solution Pattern

Enterprise Network

IBM

Voic

e G

atew

ay

MPLS, NetFoundry,

Public Internet

29 3/22/17

Service Orchestration Engine (SOE) Sample

Conversation Ask Request

Watson ConversationIBM Voice Gateway

Custom API Services

Service Orchestration

Engine

Modified Ask Request

Conversation Ask ResponseModified Ask Response

Why most deployments will need an SOE:

• De-identification of Watson Conversation requests• Personalize the Watson Conversation responses• Use telephony features like Caller-id, DTMF, etc.• Integrate with API’s to enhance the user interaction• Exploit Voice Gateway features using state variables • Voice security integration using DTMF or biometrics

SOE Banking Sample Application shows:

1. How to call API’s to enhance WCS responses2. How DTMF can be used to process user input3. How to log the interactions to WCS4. How to approach building your own SOE5. How to enable user security6. How to personalize the interactions

https://github.com/WASdev/sample.voice.gateway/tree/master/soe

30 3/22/17

IVR and Contact Center Integration

Voice gateway integrates with existing IVRs and Contact Centers using SIP initiated call transfers:

• Transfer to voice gateway from IVR• Transfer to Contact Center from voice gateway

Session Border Controllers can handle SIP REFER messages and perform the transfer:

• SBCs catch refer and perform the transfer

Agent dashboards integrate via:• Metadata exchanged in SIP signaling• Conversation history published from an SOE

Session Border

Controller

IBM Voice Gateway

Contact Center

1

23

4

1. Call arrives at SBC2. SBC routes call to Voice gateway3. Customer request transfer and a

SIP REFER is sent to SBC.

4. SBC re-routes the call to an agent through the Contact Center.

Hybrid-Connectivity

Tata NetFoundry

31 3/22/17

Global, Turnkey Solution for Enterprises and Contact Centers to Leverage Watson Voice Cognitive Services

Voice from anywhere

Interoperate PSTN & SIP voice to WatsonSecure, reliable delivery to Watson over Internet

Watson Cognitive Services

https://www.NetFoundry.io

IBM Voice Gateway

33 3/22/17

Details on the new NetFoundry AppWANTM for Watson

The NetFoundry AppWANTM for Watson was released on March 20th, 2017

What is it?

• Connects business and contact center sites to Watson Services

• Integrates with the IBM Voice Gateway for turnkey Watson interop

• Cloud native, microservice solution

• Horizontally scalable and deployable on premises

https://www.NetFoundry.io

The IBM Voice Gateway provides Watson interoperability, saving enterprises and contact centers from needing expensive telephony interworking projects to consume Watson services.

Make it Easy for Customers to Consume Watson Services

NetFoundry connects sites to Watson with top security & reliability over software defined networks, accessed from any Internet connection, enabling customers to connect to Watson without multi-month, expensive MPLS extension projects

IBM Voice Gateway

Enterprise Telephony Integration with Watson

Conversation(Dialog + NLC)

Speech To Text

Text To Speech

Watson Services

Watson Custom Proxy

Customer Specific APIs(for PHI, PCI and PII)

Sess

ion

Bord

er C

ontro

ller

Contact Center / IVR Enterprise Network

Hybrid-Cloud Solution Pattern

Enterprise Network

Net

Foun

dry

App

WA

NTM

Gat

eway

For

Wat

son

Net

Foun

dry

App

WA

NTM

Gat

eway

For

Wat

son

IBM

Voic

e G

atew

ay

This shows the hybrid cloud pattern. IBM Voice and NetFoundry gateways install on-premises or in the cloud as Docker containers or VMs, scaling horizontally with agility, reliability and DevOps efficiency

36 3/22/17

Self-Service Agent in Cloud Hosted Deployment Pattern

Text To Speech

Conversation (Dialog, NLC)

Speech To Text

Watson Services

IBM Data Center

IBM Voice Gateway

Enterprise Network

Net

Foun

dry

App

WA

NTM

Gat

eway

For

Wat

son

Sess

ion

Bord

er C

ontro

ller

Net

Foun

dry

App

WA

NTM

Gat

eway

For

Wat

son

RTPSIP

Contact Center

IBM Voice Gateway

Use Case #2: Cognitive Agent Assistant in Hybrid Pattern

Watson Speech To Text

Analytics via Watson

(NLC + Retrieve and Rank)

SIPREC

Customer (RTP / G.711)

Transcribed Text published over MQTT

Web Browser

Agent Desktop

MQTT Broker

Forking SBC

Agent

Agent (RTP / G.711)

NetFoundry SIP Trunk

Customer

NetFoundry SDN over Internet

Watson assists the contact center agent in serving the customer

NetFoundry encrypts all data, blocks all unauthenticated data from entering customer and cloud sites, and improves performance

Contact Center Environment

NetFoundry AppWANTM

for Watsongateway

NetFoundry AppWANTM

for Watsongateway

Security

As demonstrated by Mirai and similar attacks, security is increasingly the top challenge of a hyperconnected world

TCO

It is expensive and time consuming to extend private networks and infrastructure to the

Watson cloud

Performance

Most security solutions inhibit performance whereas real-time

Watson services require top performance

3 Key Challenges for Watson Connectivity

Security

Makes Watson customer networks “dark” to

security threats. Encrypt and secure the Watson

data across any network.

TCO

Eliminates private network and telco CPE costs. Turnkey solution

for SIP/telephony interoperability

Performance

Delivers enterprise grade reliability and

performance for Watson interworking

NetFoundry + IBM Voice Gateway Helps Solve the 3 Key Challenges for Watson Connectivity

IBM Voice Gateway

NetFoundry AppWANTM enables cloud services to leverage service-specific software defined networks over any Internet access providers. The result is businesses securely and reliably take cloud services from prototype to production, without network and infrastructure build-out delays and costs.

• “Partnering with NetFoundry gives our joint customers greater options for delivering quality of experience across the Internet,” said Sanjay Uppal, CEO, VeloCloud. “NetFoundry gives businesses improved performance and security across the Internet.”

• “We are seeing an increased demand from our customers for greater security, control, and network configurability for their IoT private networks,” stated Bryan Lubel, President of Integron. “Our partnership with NetFoundry allows us to satisfy those requirements.”

https://www.netfoundry.io

NetFoundry Overview and Testimonials

IBM Voice Gateway for Watson

Get started now!

41 3/22/17

42 3/22/17

Many companies are already working with the Voice Gateway

• Many POCs underway (including production usage)

• Many minutes processed

• Early prototypes have shown a 30% advantage over traditional IVRs to completely handle a call

• POCs underway in many different languages

43 3/22/17

How to get started

• Link to Docker images in DockerHub (developer only license)• https://hub.docker.com/r/ibmcom/voice-gateway-so/• https://hub.docker.com/r/ibmcom/voice-gateway-mr/

• Link to documentation in Knowledge Center• https://ibm.biz/VoiceGatewayDoc

• Link to samples in public GitHub Repository• https://github.com/WASdev/sample.voice.gateway

• Get questions answered at dwAnswers forum• https://developer.ibm.com/answers/topics/voicegateway/

• Marketplace link (includes video demonstration)• https://www.ibm.com/us-en/marketplace/voice-gateway

44 3/22/17

Become part of our community on Slack!

Chat on Slack

Then join the #ibmvoicegateway channel here: https://ibm-cloud-tech.slack.com/messages/ibmvoicegateway

First register with the ibm-cloud-tech team here: http://ibm.co/2mblgXr

45 3/22/17

Conclusion: Why Cognitive Voice AgentsBetter than traditional IVRs

• Understands natural language, agents are conversational• Fewer opt-outs means lower contact center cost• Can handle deeper, more complex conversations• Can handle multiple intents in a single call

Deep Analytics lead to improved KPIs• All aspects of system contribute to analytic processing. For example:

• Barge-in correlation points to areas to focus training and improvements• Opt-outs correlated to points in conversation lead to improved flows

InterConnect2017

46 3/22/17

47 3/22/17

Notices and disclaimersCopyright © 2017 by International Business Machines Corporation (IBM). No part of this document may be reproduced or transmitted in any form without written permission from IBM.

U.S. Government Users Restricted Rights — use, duplication or disclosure restricted by GSA ADP Schedule Contract with IBM.

Information in these presentations (including information relating to products that have not yet been announced by IBM) has been reviewed for accuracy as of the date of initial publication and could include unintentional technical or typographical errors. IBM shall have no responsibility to update this information. This document is distributed “as is” without any warranty, either express or implied. In no event shall IBM be liable for any damage arising from the use of this information, including but not limited to, loss of data, business interruption, loss of profit or loss of opportunity. IBM products and services are warranted according to the terms and conditions of the agreements under which they are provided.

IBM products are manufactured from new parts or new and used parts. In some cases, a product may not be new and may have been previously installed. Regardless, our warranty terms apply.”

Any statements regarding IBM's future direction, intent or product plans are subject to change or withdrawal without notice.

Performance data contained herein was generally obtained in a controlled, isolated environments. Customer examples are presented as illustrations of how those customers have used IBM products and

the results they may have achieved. Actual performance, cost, savings or other results in other operating environments may vary.

References in this document to IBM products, programs, or services does not imply that IBM intends to make such products, programs or services available in all countries in which IBM operates or does business.

Workshops, sessions and associated materials may have been prepared by independent session speakers, and do not necessarily reflect the views of IBM. All materials and discussions are provided for informational purposes only, and are neither intended to, nor shall constitute legal or other guidance or advice to any individual participant or their specific situation.

It is the customer’s responsibility to insure its own compliance with legal requirements and to obtain advice of competent legal counsel as to the identification and interpretation of any relevant laws and regulatory requirements that may affect the customer’s business and any actionsthe customer may need to take to comply with such laws. IBM does not

provide legal advice or represent or warrant that its services or products will ensure that the customer is in compliance with any law.

48 3/22/17

Notices and disclaimers continuedInformation concerning non-IBM products was obtained from the suppliers of those products, their published announcements or other publicly available sources. IBM has not tested those products in connection with this publication and cannot confirm the accuracy of performance, compatibility or any other claims related to non-IBM products. Questions on the capabilities of non-IBM products should be addressed to the suppliers of those products. IBM does not warrant the quality of any third-party products, or the ability of any such third-party products to interoperate with IBM’s products. IBM expressly disclaims all warranties, expressed or implied, including but not limited to, the implied warranties of merchantability and fitness for a particular, purpose.

The provision of the information contained herein is not intended to, and does not, grant any right or license under any IBM patents, copyrights, trademarks or other intellectual property right.

IBM, the IBM logo, ibm.com, Aspera®, Bluemix, Blueworks Live, CICS, Clearcase, Cognos®, DOORS®, Emptoris®, Enterprise Document Management System™, FASP®, FileNet®, Global Business Services®,Global Technology Services®, IBM ExperienceOne™, IBM SmartCloud®, IBM Social Business®, Information on Demand, ILOG, Maximo®, MQIntegrator®, MQSeries®, Netcool®, OMEGAMON, OpenPower, PureAnalytics™, PureApplication®, pureCluster™, PureCoverage®, PureData®, PureExperience®, PureFlex®, pureQuery®, pureScale®, PureSystems®, QRadar®, Rational®, Rhapsody®, Smarter Commerce®, SoDA, SPSS, Sterling Commerce®, StoredIQ, Tealeaf®, Tivoli® Trusteer®, Unica®, urban{code}®, Watson, WebSphere®, Worklight®, X-Force® and System z® Z/OS, are trademarks of International Business Machines Corporation, registered in many jurisdictions worldwide. Other product and service names might be trademarks of IBM or other companies. A current list of IBM trademarks is available on the Web at "Copyright and trademark information" at: www.ibm.com/legal/copytrade.shtml.