65
How Watson, Bluemix, Cloudant, and XPages Can Work Together In A Real World Application Frank van der Linden, elstar IT Full stack Java Developer Track 3 - Session 1 EntwicklerCamp 2018

How Watson, Bluemix, Cloudant, and XPages Can Work ... · How Watson, Bluemix, Cloudant, and XPages Can Work Together In A Real World Application Frank van der Linden, elstar IT Full

  • Upload
    others

  • View
    7

  • Download
    0

Embed Size (px)

Citation preview

Page 1: How Watson, Bluemix, Cloudant, and XPages Can Work ... · How Watson, Bluemix, Cloudant, and XPages Can Work Together In A Real World Application Frank van der Linden, elstar IT Full

How Watson, Bluemix, Cloudant, and XPages Can Work Together In A Real

World ApplicationFrank van der Linden, elstar IT

Full stack Java Developer Track 3 - Session 1

EntwicklerCamp 2018

Page 2: How Watson, Bluemix, Cloudant, and XPages Can Work ... · How Watson, Bluemix, Cloudant, and XPages Can Work Together In A Real World Application Frank van der Linden, elstar IT Full

Me• Freelance Full stack Java developer

• Owner of elstar IT

• Curious of new technology

• IBM Champion 2013, 2014, 2017 and 2018

Page 3: How Watson, Bluemix, Cloudant, and XPages Can Work ... · How Watson, Bluemix, Cloudant, and XPages Can Work Together In A Real World Application Frank van der Linden, elstar IT Full

How do we proceed• HR Assistant

• Running it on IBM BlueMix

• Store it in Cloudant

• Analyse it with IBM Watson

• Show it with Charts

• Add new features

• Lessons learned

Page 4: How Watson, Bluemix, Cloudant, and XPages Can Work ... · How Watson, Bluemix, Cloudant, and XPages Can Work Together In A Real World Application Frank van der Linden, elstar IT Full

HR Assistant

Page 5: How Watson, Bluemix, Cloudant, and XPages Can Work ... · How Watson, Bluemix, Cloudant, and XPages Can Work Together In A Real World Application Frank van der Linden, elstar IT Full

Demo time

Page 6: How Watson, Bluemix, Cloudant, and XPages Can Work ... · How Watson, Bluemix, Cloudant, and XPages Can Work Together In A Real World Application Frank van der Linden, elstar IT Full
Page 7: How Watson, Bluemix, Cloudant, and XPages Can Work ... · How Watson, Bluemix, Cloudant, and XPages Can Work Together In A Real World Application Frank van der Linden, elstar IT Full

Frédéric Dehédin• At the time of the project: Senior Application developer

at Belsoft Collaboration AG

• @fdehedin

• http://www.fdehedin.ch/

Page 8: How Watson, Bluemix, Cloudant, and XPages Can Work ... · How Watson, Bluemix, Cloudant, and XPages Can Work Together In A Real World Application Frank van der Linden, elstar IT Full

Facts• Winner of the ICS Developer Competition 2016

• Runs on the XSP runtime on IBM BlueMix

• Store all the data in Cloudant

• Analyse jobs and job applications with IBM Watson

• Backend is pure Java, no Java Notes objects

Page 9: How Watson, Bluemix, Cloudant, and XPages Can Work ... · How Watson, Bluemix, Cloudant, and XPages Can Work Together In A Real World Application Frank van der Linden, elstar IT Full

Add ons• Cloudant connector OSGi plugin

• Take out the security issue

• Storing in and retrieving from Cloudant. JSON <—> POJO

• Can be accessed from Java backend.

• Do the heavy lifting. Don't reinvent the wheel.

Page 10: How Watson, Bluemix, Cloudant, and XPages Can Work ... · How Watson, Bluemix, Cloudant, and XPages Can Work Together In A Real World Application Frank van der Linden, elstar IT Full

Add ons• Jackson Mapper OSGi plugin

• Take out the security issue

• Used with the IBM Watson services

• Conversion of JSON to POJO

• Makes @Annotations available in Java back end

Page 11: How Watson, Bluemix, Cloudant, and XPages Can Work ... · How Watson, Bluemix, Cloudant, and XPages Can Work Together In A Real World Application Frank van der Linden, elstar IT Full

Roadmap

Page 12: How Watson, Bluemix, Cloudant, and XPages Can Work ... · How Watson, Bluemix, Cloudant, and XPages Can Work Together In A Real World Application Frank van der Linden, elstar IT Full

Running it on IBM BlueMix

Page 13: How Watson, Bluemix, Cloudant, and XPages Can Work ... · How Watson, Bluemix, Cloudant, and XPages Can Work Together In A Real World Application Frank van der Linden, elstar IT Full

Overview• BlueMix is IBM PaaS, with various services

• We picked a few services

Page 14: How Watson, Bluemix, Cloudant, and XPages Can Work ... · How Watson, Bluemix, Cloudant, and XPages Can Work Together In A Real World Application Frank van der Linden, elstar IT Full

Credentials

Page 15: How Watson, Bluemix, Cloudant, and XPages Can Work ... · How Watson, Bluemix, Cloudant, and XPages Can Work Together In A Real World Application Frank van der Linden, elstar IT Full

Deploy to IBM BlueMix• Fill the BlueMix Manifest file under

Application Configuration

• Will result in a yaml file

Page 16: How Watson, Bluemix, Cloudant, and XPages Can Work ... · How Watson, Bluemix, Cloudant, and XPages Can Work Together In A Real World Application Frank van der Linden, elstar IT Full

Deploy to IBM BlueMix• Set location of deployment

• Add shared-plugin folder

• Add the OSGi plugins

Page 17: How Watson, Bluemix, Cloudant, and XPages Can Work ... · How Watson, Bluemix, Cloudant, and XPages Can Work Together In A Real World Application Frank van der Linden, elstar IT Full

Deploy to IBM BlueMix• Deploy Application

Page 18: How Watson, Bluemix, Cloudant, and XPages Can Work ... · How Watson, Bluemix, Cloudant, and XPages Can Work Together In A Real World Application Frank van der Linden, elstar IT Full

Store it in Cloudant

Page 19: How Watson, Bluemix, Cloudant, and XPages Can Work ... · How Watson, Bluemix, Cloudant, and XPages Can Work Together In A Real World Application Frank van der Linden, elstar IT Full

What is Cloudant• It is the cloud version of CouchDb

• It is document based

• CouchDb is created by Damien Katz

• Cloudant is bought by IBM

• Is like MongoDb or NSF a NoSQL datastore

Page 20: How Watson, Bluemix, Cloudant, and XPages Can Work ... · How Watson, Bluemix, Cloudant, and XPages Can Work Together In A Real World Application Frank van der Linden, elstar IT Full

What is Cloudant• Data stored as JSON

• View and Search ‘formula’ defined in javascript

• Replication

• Tasks

• Build in API

• No annoying limitations

Page 21: How Watson, Bluemix, Cloudant, and XPages Can Work ... · How Watson, Bluemix, Cloudant, and XPages Can Work Together In A Real World Application Frank van der Linden, elstar IT Full

Cloudant database

Page 22: How Watson, Bluemix, Cloudant, and XPages Can Work ... · How Watson, Bluemix, Cloudant, and XPages Can Work Together In A Real World Application Frank van der Linden, elstar IT Full

Design document• Special type of document

• Holds javascript for View and FT Search definition.

Page 23: How Watson, Bluemix, Cloudant, and XPages Can Work ... · How Watson, Bluemix, Cloudant, and XPages Can Work Together In A Real World Application Frank van der Linden, elstar IT Full

• It is JSON everywhere.

• _rev and _id are reserved parameters.

• doc{}, is the actual data from application

Document

Page 24: How Watson, Bluemix, Cloudant, and XPages Can Work ... · How Watson, Bluemix, Cloudant, and XPages Can Work Together In A Real World Application Frank van der Linden, elstar IT Full

Attachment @ Cloudant• Stored as base64 encoded String

• Can be as standalone

• Or as part of Document (we choose this way)

• Store id as attachmentId, so it can be retrieved when needed

Page 25: How Watson, Bluemix, Cloudant, and XPages Can Work ... · How Watson, Bluemix, Cloudant, and XPages Can Work Together In A Real World Application Frank van der Linden, elstar IT Full

Attachment @ Cloudant• AttachmentDAO

Page 26: How Watson, Bluemix, Cloudant, and XPages Can Work ... · How Watson, Bluemix, Cloudant, and XPages Can Work Together In A Real World Application Frank van der Linden, elstar IT Full

Attachment @ Cloudant• AttachmentController

• Convert upload to Attachment object

Page 27: How Watson, Bluemix, Cloudant, and XPages Can Work ... · How Watson, Bluemix, Cloudant, and XPages Can Work Together In A Real World Application Frank van der Linden, elstar IT Full

Attachment @ Cloudant• ControllerBase

• All controllers can use these methods

Page 28: How Watson, Bluemix, Cloudant, and XPages Can Work ... · How Watson, Bluemix, Cloudant, and XPages Can Work Together In A Real World Application Frank van der Linden, elstar IT Full

Attachment @ Cloudant• Servlet is called in application for download

• base64 data for the image tag

Page 29: How Watson, Bluemix, Cloudant, and XPages Can Work ... · How Watson, Bluemix, Cloudant, and XPages Can Work Together In A Real World Application Frank van der Linden, elstar IT Full

Analyse it with IBM Watson

Page 30: How Watson, Bluemix, Cloudant, and XPages Can Work ... · How Watson, Bluemix, Cloudant, and XPages Can Work Together In A Real World Application Frank van der Linden, elstar IT Full

Watson API explorer• Very useful to test input and see what is returned

Page 31: How Watson, Bluemix, Cloudant, and XPages Can Work ... · How Watson, Bluemix, Cloudant, and XPages Can Work Together In A Real World Application Frank van der Linden, elstar IT Full

Services we used• Tone analyser, to get the tone of the text

• Personality Insight, to determine the personality of the job applicant

• Natural Understanding Language, to get the keywords from the Job and Job application

Page 32: How Watson, Bluemix, Cloudant, and XPages Can Work ... · How Watson, Bluemix, Cloudant, and XPages Can Work Together In A Real World Application Frank van der Linden, elstar IT Full

tone analyser schema

Page 33: How Watson, Bluemix, Cloudant, and XPages Can Work ... · How Watson, Bluemix, Cloudant, and XPages Can Work Together In A Real World Application Frank van der Linden, elstar IT Full

Tone analyser• The same structure to call the Watson services

Page 34: How Watson, Bluemix, Cloudant, and XPages Can Work ... · How Watson, Bluemix, Cloudant, and XPages Can Work Together In A Real World Application Frank van der Linden, elstar IT Full

Tone analyser• ToneAnalyzerRequest contain the text to analyse

• ToneAnalyzerInjector gets text from model class

Page 35: How Watson, Bluemix, Cloudant, and XPages Can Work ... · How Watson, Bluemix, Cloudant, and XPages Can Work Together In A Real World Application Frank van der Linden, elstar IT Full

Tone analyser• Post the data to IBM Watson service (1)

• Convert the response back to Java class (2)

1

2

Page 36: How Watson, Bluemix, Cloudant, and XPages Can Work ... · How Watson, Bluemix, Cloudant, and XPages Can Work Together In A Real World Application Frank van der Linden, elstar IT Full

Tone analyser• ToneAnalyzerResponse holds the results

• Use @Annotations from Jackson to map JSON to properties

• Result is the source of the charts

Page 37: How Watson, Bluemix, Cloudant, and XPages Can Work ... · How Watson, Bluemix, Cloudant, and XPages Can Work Together In A Real World Application Frank van der Linden, elstar IT Full

Show it with charts

Page 38: How Watson, Bluemix, Cloudant, and XPages Can Work ... · How Watson, Bluemix, Cloudant, and XPages Can Work Together In A Real World Application Frank van der Linden, elstar IT Full

ChartJS• Open source HTML5 javascript based

• Responsive

• DataSets = Array of Objects

Page 39: How Watson, Bluemix, Cloudant, and XPages Can Work ... · How Watson, Bluemix, Cloudant, and XPages Can Work Together In A Real World Application Frank van der Linden, elstar IT Full

ChartJS• Factory to produce the chart data, accepts objects

• Factory set default Chart Type, like Radar or Bar

Page 40: How Watson, Bluemix, Cloudant, and XPages Can Work ... · How Watson, Bluemix, Cloudant, and XPages Can Work Together In A Real World Application Frank van der Linden, elstar IT Full

ChartJS

Page 41: How Watson, Bluemix, Cloudant, and XPages Can Work ... · How Watson, Bluemix, Cloudant, and XPages Can Work Together In A Real World Application Frank van der Linden, elstar IT Full

ChartJS• Custom control to display data

Page 42: How Watson, Bluemix, Cloudant, and XPages Can Work ... · How Watson, Bluemix, Cloudant, and XPages Can Work Together In A Real World Application Frank van der Linden, elstar IT Full

ChartJS• Custom control to display data.

Page 43: How Watson, Bluemix, Cloudant, and XPages Can Work ... · How Watson, Bluemix, Cloudant, and XPages Can Work Together In A Real World Application Frank van der Linden, elstar IT Full

Lessons learned

Page 44: How Watson, Bluemix, Cloudant, and XPages Can Work ... · How Watson, Bluemix, Cloudant, and XPages Can Work Together In A Real World Application Frank van der Linden, elstar IT Full

Lessons learned• No Single Sign On service on all datacenters

• Domino Designer and GIT are not an happy couple

• API’s are not consistent across Watson services, but it gets better

• Cloudant is a very reliable, flexible and fast NoSQL datastore

• IBM Watson services are really powerful.

Page 45: How Watson, Bluemix, Cloudant, and XPages Can Work ... · How Watson, Bluemix, Cloudant, and XPages Can Work Together In A Real World Application Frank van der Linden, elstar IT Full

Lessons learned• There is no RichText on the web, it is HTML

• Handling attachments on Cloudant was a challenge. Took some time get a robust solution.

• IBM Watson keeps changing. For example the Document Conversion

Page 46: How Watson, Bluemix, Cloudant, and XPages Can Work ... · How Watson, Bluemix, Cloudant, and XPages Can Work Together In A Real World Application Frank van der Linden, elstar IT Full

New features (not on GitHub)

Page 47: How Watson, Bluemix, Cloudant, and XPages Can Work ... · How Watson, Bluemix, Cloudant, and XPages Can Work Together In A Real World Application Frank van der Linden, elstar IT Full

Multi language• Property with labels are stored in design

• Export of this labels

• Import of newly translated labels

• Everything is stored in datastore

Page 48: How Watson, Bluemix, Cloudant, and XPages Can Work ... · How Watson, Bluemix, Cloudant, and XPages Can Work Together In A Real World Application Frank van der Linden, elstar IT Full

Multi language

Page 49: How Watson, Bluemix, Cloudant, and XPages Can Work ... · How Watson, Bluemix, Cloudant, and XPages Can Work Together In A Real World Application Frank van der Linden, elstar IT Full

Multi language

Page 50: How Watson, Bluemix, Cloudant, and XPages Can Work ... · How Watson, Bluemix, Cloudant, and XPages Can Work Together In A Real World Application Frank van der Linden, elstar IT Full

Multi language

Page 51: How Watson, Bluemix, Cloudant, and XPages Can Work ... · How Watson, Bluemix, Cloudant, and XPages Can Work Together In A Real World Application Frank van der Linden, elstar IT Full

Multi language

Page 52: How Watson, Bluemix, Cloudant, and XPages Can Work ... · How Watson, Bluemix, Cloudant, and XPages Can Work Together In A Real World Application Frank van der Linden, elstar IT Full

Runs on Domino• ACL is enforced when run on Domino

• 3 Roles can be assigned to a NAB group

• Application ‘knows’ where it runs

Page 53: How Watson, Bluemix, Cloudant, and XPages Can Work ... · How Watson, Bluemix, Cloudant, and XPages Can Work Together In A Real World Application Frank van der Linden, elstar IT Full

Runs on Domino

Page 54: How Watson, Bluemix, Cloudant, and XPages Can Work ... · How Watson, Bluemix, Cloudant, and XPages Can Work Together In A Real World Application Frank van der Linden, elstar IT Full

NSF as datastore• Make use of the OpenNTF Domino API

• Stores data as JSON in 1 field

• ODA convert field to RichText as limit of field is reached

Page 55: How Watson, Bluemix, Cloudant, and XPages Can Work ... · How Watson, Bluemix, Cloudant, and XPages Can Work Together In A Real World Application Frank van der Linden, elstar IT Full

NSF as datastore

Page 56: How Watson, Bluemix, Cloudant, and XPages Can Work ... · How Watson, Bluemix, Cloudant, and XPages Can Work Together In A Real World Application Frank van der Linden, elstar IT Full

Document Conversion API• Spring Boot application runs on IBM Cloud

• Convert document as base64 encoded string

• Convert document as uploaded file

• Code is open source

• Swagger support

Page 57: How Watson, Bluemix, Cloudant, and XPages Can Work ... · How Watson, Bluemix, Cloudant, and XPages Can Work Together In A Real World Application Frank van der Linden, elstar IT Full

Document Conversion API

Page 58: How Watson, Bluemix, Cloudant, and XPages Can Work ... · How Watson, Bluemix, Cloudant, and XPages Can Work Together In A Real World Application Frank van der Linden, elstar IT Full
Page 59: How Watson, Bluemix, Cloudant, and XPages Can Work ... · How Watson, Bluemix, Cloudant, and XPages Can Work Together In A Real World Application Frank van der Linden, elstar IT Full

Settings• Settings stored in a custom property file

• Edit in the browser

• Only available on the Domino server

• Thanks to Sven Hasselbach (http://hasselba.ch/blog/?p=1763)

Page 60: How Watson, Bluemix, Cloudant, and XPages Can Work ... · How Watson, Bluemix, Cloudant, and XPages Can Work Together In A Real World Application Frank van der Linden, elstar IT Full

Settings

Page 61: How Watson, Bluemix, Cloudant, and XPages Can Work ... · How Watson, Bluemix, Cloudant, and XPages Can Work Together In A Real World Application Frank van der Linden, elstar IT Full

Settings

Page 62: How Watson, Bluemix, Cloudant, and XPages Can Work ... · How Watson, Bluemix, Cloudant, and XPages Can Work Together In A Real World Application Frank van der Linden, elstar IT Full

Questions

Page 63: How Watson, Bluemix, Cloudant, and XPages Can Work ... · How Watson, Bluemix, Cloudant, and XPages Can Work Together In A Real World Application Frank van der Linden, elstar IT Full

Resources• HR Assistant on BlueMix

• https://hrassistant1.mybluemix.net/

• HR Assistant code on GitHub • https://github.com/flinden68/HR-Assistant

• IBM Watson services

• https://www.ibm.com/watson/developercloud/services-catalog.html

• IBM Cloudant documentation • https://docs.cloudant.com/

• HR Assistant Demo • https://youtu.be/yM5Foxn9Fvg

Page 64: How Watson, Bluemix, Cloudant, and XPages Can Work ... · How Watson, Bluemix, Cloudant, and XPages Can Work Together In A Real World Application Frank van der Linden, elstar IT Full

Resources• OSGi Cloudant connector

• https://bitbucket.org/flinden68/cloudant-connector

• OSGi Jackson JSON Wrapper • https://bitbucket.org/flinden68/jackson-json-mapper-plugin

• ChartJS

• http://www.chartjs.org/

• Watson API Explorer

• https://watson-api-explorer.mybluemix.net/

• Document Conversion API

• https://bitbucket.org/flinden68/document-conversion-api

Page 65: How Watson, Bluemix, Cloudant, and XPages Can Work ... · How Watson, Bluemix, Cloudant, and XPages Can Work Together In A Real World Application Frank van der Linden, elstar IT Full

Reach out to me• @flinden68

• http://www.elstarit.nl

[email protected]

• https://nl.linkedin.com/in/flinden68