Embed Size (px)
Citation preview
Gain insights into your business operations with BPM Analytics
Allen ChanChief Architect, Business Process Management
IBM Cloud University 2017 | October
Please Note
IBM’s statements regarding its plans, directions, and intent are subject to change or withdrawal without notice at IBM’s sole discretion.
Information regarding potential future products is intended to outline our general product direction and it should not be relied on in making a purchasing decision.
The information mentioned regarding potential future products is not a commitment, promise, or legal obligation to deliver any material, code or functionality. Information about potential future products may not be incorporated into any contract.
The development, release, and timing of any future features or functionality described for our products remains at our sole discretion.
Performance is based on measurements and projections using standard IBM benchmarks in a controlled environment. The actual throughput or performance that any user will experience will vary depending upon many factors, including considerations such as the amount of multiprogramming inthe user’s job stream, the I/O configuration, the storage configuration, and the workload processed. Therefore, no assurance can be given that an individual user will achieve results similar to those stated here.
Agenda
• Digital Process Automation
• Business Monitoring
• Elasticsearch + Kibana with IBM BPM
3 9/29/17
… to deliver the end-to-end customer journey
… built on an extensible framework for innovation and insight
… enabled by a platform for rapid delivery and iteration, led by the business
Digital Process Automation is… IBM’s platform for rapidly creating business–led process apps
that deliver new levels of customer experience and digital self service
Key attributes of the Digital Process Automation platform:
Customer focus
Speed Scale
Digital Process Automation capabilities
Define customer journeys and business operations to discover new opportunities for innovation
Design, automate and govern business operations to deliver a scalable, consistent
customer experience
Analyze data and events to augment knowledge work and take the right action at the right time
ActionableInsights
CollaborativeDiscovery
Business Automation
Digital Process
Automation
To get Actionable Insights –we need business monitoring
• Increase profitability – Reduce costs by ensuring timely completion of work– Real-time insight into operational processes for
• Improved operational efficiency• Dynamic business response
• Increase customer satisfaction– Proactive alerts to business situations– Access to relevant information to
resolve situations
• Increase team productivity– Monitor overall team productivity– Monitor individual worker productivity– Identify high performing workers to
mentor lower productivity workers
! CICS
! IMS
! WAS
Executive!Management!
Customer!Service!
Risk ManagementTeams!
Finance!and Ops!
Account Administration!
6
Drill down to specific situations
Track your KPIs and get alerted as situations
occurTrend analysis and
prediction
Analyze a range of business data with BAMTake action faster with real-time information on personalized dashboards
§ Drill down to investigate why John Smith’s new bank account request has not been processed within the specified service level agreement
§ See if the average amount of time to open a new bank account is within an acceptable or desired range – from your desktop or mobile
§ Compare new account opening KPIs across time periods to manage resources more effectively and predict future problems
7
1. Track2. Store3. Report4. Analyze5. Respond
8 9/29/17
5 Steps to manage your Operational Intelligence
• Built-in data store for business events.• BPM will send out business events with:
• Auto-Tracking • Tracking Points• Timing Intervals
• Built-in Performance Dashboard
• Team Performance Dashboard• Process Performance Dashboard
9 9/29/17
Performance Data Warehouse
• Once turned on, will track all process level events, e.g pre-and post- task execution.
• Individual variable values can also be tracked.
10 9/29/17
Auto Tracking
A database table by the
“Autotracking name” will be
created in the PDW
database.
1. Add custom Tracking Groups with a specific set of variables.
2. Specify in the BPD where to write Tracking Groups data before or after a certain activity.
11 9/29/17
Tracking Points & Tracking Groups
In this case, we will write an entry in
the Tracking Group table
TG_TRAVELREQUEST
Using Intermediate Event, one can create timing interval with tracking groups to measure elapsed time between any two events.
12 9/29/17
Tracking Points
BPM has a built-in Performance Dashboard that will allow you to visualize the information about process performance.
13 9/29/17
Performance Dashboard
1. Customize the information we want to show in the Dashboard?2. Federate information across multiple BPM systems?3. Perform data mining and analytics with business data?4. Roll Your Own analytics solutions by using 3rd part analytic software?
14 9/29/17
But, what if we want to …
BPM Analytics Tech Preview• Enable the BPM users to gain insight of the
processes with big data and analytics• Performance overhead ~ 10% or less• Expose the rich information of the processes – system
view and business view• Near real time• Archive and restore historical data to form new monitor
model and new analysis• Out of the box sample dashboards
• 3rd Party Analytics• Integrate the data which do not come to the BPM system• Publish the process data to external data and analytics
solutions provided by IBM, customer and partners
• For evaluation only, not for production use
Process Server
JMS
Business Events
BPM Cell
defconfig.xml BPMEvent
Emitter Kibana
Elasticsearch
Topology reference
Process Server
JMS
Business Events
BPM Cell
defconfig.xml BPMEvent
Emitter
Not packaged by BPM
Event Summary
AgentRaw Event
Combined Event
BPM Pre-defined
Dashboard
BPM Analytics components
Object Storage
Event Summary
Agent
Archive
Archive Restore
Restore
Optional
• Two level storages, Elasticsearch for the online storage, the Object Storage is the archive storage
• Support multiple object storage interfaces, like: OpenStack Swift and Amazon S3(to be ready)
• Raw events will be stored at the archive object storage in the bulks
• Archive storage offline by accident will not impact the analytics function
• Archive operation will avoid peak time automatically• Archived events can be selected by the time range to re-build
the indices at the ES
Topology – option 1
• Kafka can be configured as the event cache in front of all event consumers
Process Server
JMS
Business Events
BPM Cell
defconfig.xml
BPMEventEmitter Ka
fka
Not the OOTB capability
Kibana
ElasticsearchEvent
SummaryAgent
Raw Event
Combined Event
Object Storage
Event Summary
Agent
Archive
Archive Restore
Restore
Topology – option 2
• Kafka can be the message broker for any analytics solutions
Process Server
JMS
Business Events
BPM Cell
defconfig.xml
BPMEventEmitter
Kafka
Not the OOTB capability
Any Analytics Solution
Not the OOTB capability
BPMEventEmitter Design
• If the BPM enable emit the JSON format DEF event, the “XML to JSON” will be skipped automatically• ES and Kafka are event consumers, they can be enabled together or separately• Events emit to the different consumers are identical, it’s easy to build up event tunnel between:
• Kafka to Analytics – customized analytics solution• Kafka to ES – event cache in front of BPM Analytics• Kafka to Object Storage – customized archive solution
MDB
XML to JSON
Def event (xml or json)
To Elasticsearch
To Kafka
BPMEventEmitterconfig
Activity transformer
Process transformer
Activity Event
• Contains all the information from activity level events. • One activity level event generates one Activity Event document• One task has several Activity Event documents
Activity Summary
• Contains information that represents one task, created by combining information from several Activity Event documents.
• Simple flat structure – simplify search query and visualzation by Kibana
20 9/29/17
Elasticsearch Indexes
Raw Event Type (immutable)
Elasticsearch
Data model at Elasticsearch
Combined Type (mutable)
Kibana
Elasticsearch Index
Activity Event
Process Event
ActivitySummary
Process Summary
MDB
Transform
(aggregate)
AggregatipnHandler
• Aggregation is the batch processing with low latency
• Not stream processing because the aggregated event’s time range can be weeks, or even months
• Data as Combined Types is mutable:• Update the process status for
example from the running to finished
• Data as Raw Event Types is immutable• Support re-derive Combined
Types over again based on the Raw Event Types
ElasticsearchIndex
ProgressRecord
Process Event
• Contains all the information from a Process instance as it happens.
• One process event generates one Process Event document• One process instance can have several Process Event
documents
Process Summary
• A single document containing the most useful information of a process instance by combining data from several Process Event documents
• One Process Summary document represents one process instance
• Simple flat structure – simplify search query and visualization by Kibana
22 9/29/17
Elasticsearch Indexes { "cellName":"BPM1", "eventVersion": "2010-11-11", "eventKind": "bpmnx:PROCESS_TERMINATED", "occurrenceTime": "2017-02-06T13:24:54.283+08:00", "processApplicationId": "532cc340-7d17-4ce7-a1c1-6e8380fec134", "processApplicationVersion": "2064.2142be53-2334-4338-a194-4e3861992ae2", "processApplicationName": "Hiring Sample", "processApplicationSnapshotName": "v5", "startingProcessInstanceFullId":"cd974bd9-0ba2-499d-abdc-319c3be66993.2064.ca663330-
14b8-40c2-8a57-2c63a173c677.44000000435", "processFullId": "cd974bd9-0ba2-499d-abdc-319c3be66993.2064.ca663330-14b8-40c2-8a57-
2c63a173c677.44000000435", "processName":"Standard HR Open New Position", "processInstanceId":44000000435, "processState":"Terminated", "processVersion":"2064.ca663330-14b8-40c2-8a57-2c63a173c677", "processPOId":"cd974bd9-0ba2-499d-abdc-319c3be66993", "processSnapshotName": "v5"
}
{ "cellName":"BPM1", "processStartTime":"2017-02-06T13:00:00.384+08:00", "processTerminatedTime": "2017-02-06T13:24:54.283+08:00", "processApplicationId": "532cc340-7d17-4ce7-a1c1-6e8380fec134", "processApplicationVersion": "2064.2142be53-2334-4338-a194-4e3861992ae2", "processApplicationName": "Hiring Sample", "processApplicationSnapshotName": "v5", "startingProcessInstanceFullId":"cd974bd9-0ba2-499d-abdc-319c3be66993.2064.ca663330-
14b8-40c2-8a57-2c63a173c677.44000000435", "processFullId": "cd974bd9-0ba2-499d-abdc-319c3be66993.2064.ca663330-14b8-40c2-8a57-
2c63a173c677.44000000435", "processName":"Standard HR Open New Position", "processInstanceId":44000000435, "processState":"Terminated", "processVersion":"2064.ca663330-14b8-40c2-8a57-2c63a173c677", "processPOId":"cd974bd9-0ba2-499d-abdc-319c3be66993", "processSnapshotName": "v5"
}
Info calculated from several Process Event document
Process Event
Process Summary
23 9/29/17
{"cellName":"BPM1","eventVersion": "2010-11-11", "eventKind": "bpmnx:ACTIVITY_ACTIVE","occurrenceTime":2017-01-25T17:32:32.084+08:00,"performerId":"deadmin","performerName":"deadmin","activityType":"userTask","activityName":"HT1","activityMonId":"bpdid:54fb3da14f1aa466:3550961c:159b077f5d7:-7f6b","activityMonVersion":"2064.402aa322-1a07-4886-a848-c4af1f2a8c3e","processApplicationId": "532cc340-7d17-4ce7-a1c1-6e8380fec134","processApplicationVersion": "2064.2142be53-2334-4338-a194-4e3861992ae2","processApplicationName": "PA01","processApplicationSnapshotName": "v5","activityFullId": "a0884aa1-5112-4142-a79a-e87657b142fb.2064.402aa322-1a07-4886-a848-
c4af1f2a8c3e.234-3-6.16","startingProcessInstanceFullId":"cf3e2fcc-5aba-43fb-abc5-76347b9fd7b0.2064.c7e467ab-e7b3-4307-
88e6-44fbd1b74fbf.234", }
{"cellName":"BPM1","eventVersion": "2010-11-11", "eventKind": "bpmnx:ACTIVITY_RESOURCE_ASSIGNED","occurrenceTime": 2017-01-25T14:54:20.490+08:00"taskInstanceId":"280","potentialPerformerId":"All Users_T_da7e4d23-78cb-4483-98ed-b9c238308a03.5e6546c5-
5ae4-429c-bc57-cdbc3bb015f5","potentialPerformerName":"All Users","potentialPerformerDocumentation":"All Users","activityType":"userTask","activityName":"HT1","activityMonId":"bpdid:54fb3da14f1aa466:3550961c:159b077f5d7:-7f6b","activityMonVersion":"2064.402aa322-1a07-4886-a848-c4af1f2a8c3e","processApplicationId": "532cc340-7d17-4ce7-a1c1-6e8380fec134","processApplicationVersion": "2064.2142be53-2334-4338-a194-4e3861992ae2","processApplicationName": "PA01","processApplicationSnapshotName": "v5","activityFullId": "a0884aa1-5112-4142-a79a-e87657b142fb.2064.402aa322-1a07-4886-a848-
c4af1f2a8c3e.234-3-6.16","startingProcessInstanceFullId":"cf3e2fcc-5aba-43fb-abc5-76347b9fd7b0.2064.c7e467ab-
e7b3-4307-88e6-44fbd1b74fbf.234", }
{"cellName":"BPM1", "taskInstanceId":"280","potentialPerformerId":"All Users_T_da7e4d23-78cb-4483-98ed-b9c238308a03.5e6546c5-5ae4-429c-bc57-cdbc3bb015f5","potentialPerformerName":"All Users","potentialPerformerDocumentation":"All Users","performerId":"deadmin","performerName":"deadmin","taskStartTime":2017-01-25T14:54:20.490+08:00,"taskClaimedTime":2017-01-25T17:32:32.084+08:00,"taskCompletedTime": 2017-01-25T17:34:32.084+08:00,"taskState":"Completed","activityType":"userTask","activityName":"HT1","activityMonId":"bpdid:54fb3da14f1aa466:3550961c:159b077f5d7:-7f6b","activityMonVersion":"2064.402aa322-1a07-4886-a848-c4af1f2a8c3e","processApplicationId": "532cc340-7d17-4ce7-a1c1-6e8380fec134","processApplicationVersion": "2064.2142be53-2334-4338-a194-4e3861992ae2","processApplicationName": "PA01","processApplicationSnapshotName": "v5","activityFullId": "a0884aa1-5112-4142-a79a-e87657b142fb.2064.402aa322-1a07-4886-a848-c4af1f2a8c3e.234-3-6.16","startingProcessInstanceFullId":"cf3e2fcc-5aba-43fb-abc5-76347b9fd7b0.2064.c7e467ab-e7b3-4307-88e6-44fbd1b74fbf.234",
}
Activity Event
Activity Summary
Info calculated from several Activity Event documents
Elasticsearch Indexes
BusinessData
• Contains business data and KPI
24 9/29/17
Elasticsearch Indexes
{"cellName":"BPM1", "activityFullId": "a0884aa1-5112-4142-a79a-e87657b142fb.2064.402aa322-1a07-4886-a848-c4af1f2a8c3e.234.16","startingProcessInstanceFullId":"cf3e2fcc-5aba-43fb-abc5-76347b9fd7b0.2064.c7e467ab-e7b3-4307-88e6-44fbd1b74fbf.234", "processApplicationName": "Hiring Sample","processApplicationSnapshotName": "v5","processName":"Standard HR Open New Position","processInstanceId":234,"taskId":280"activityName":"Submit position request ","wleGroupId": "19b73424-1bdc-4ee4-aa5a-f4ed8784d4a8","wleName": "Submit position request (PRE)","wleGroupVersion": "2064.be7c5d58-e3b6-4550-a253-09be00e5434e","wleGroupName": "aEmpRequisition121381434563922","wleTime": 2017-01-17T15:31:06.852+08:00,"wleVersion": "2064.be7c5d58-e3b6-4550-a253-09be00e5434e","wleId": "-f4ed8784d4a8bpdid431b0753c33842e23d5457c0141a2fd3448-75fb (PRE)","kpiData":{
"Rework.integer":0}"trackedField":{
"GMApproval.boolean":false,"NumberRequested.integer": "2","Location.keyword": "Chicago","EmploymentStatus.keyword": "Full-time","Department.keyword": "Finance","HiringManager.keyword": "Jack Smith"
}}
Business Data sample
Out of box Dashboard – Process Performance
25
Identify total process count
Process statistics based on BPM system and state
Average process running duration
Process instance started per 3 hours – help to find busy time
Process instance completed per 3 hours – help to find busy time
Process count for each process
Out of box Dashboard – Task Performance
26
Drill down for specific task: review new position request
Identify total task count
Task count by BPM system and state
Task count for different tasks
Task started and completed per 3 hours – help to find busy time
26,124 out of 128,926 tasks are “review new position request”
Average task running duration increases from 35.07m to 63.86m
Average Task Running Duration
Out of box Dashboard – Team Performance
27
Task assigned to each team and team members
Average task duration for each team member
Task completed per 3 hours by each team
Sample Custom Dashboard – KPI
28
Average Task Time, Average Execute Time and Average Wait Time for each task
Average task time change trend per 3 hours
Average task time has a sudden increase here. It may indicate abnormal.
Sample Dashboard – Business at a Glance
29
• Business Data dashboard is different for each process and can only be customized by yourself.
• Here we show a sample business dashboard for Hiring Sample
How many people are hired at each location and what type
How many people are hired in each department and their job type
Tile map shows hiring people at each location. Deeper color indicates more people are hired there
Approval rate for each manager’s requests
Create “Business at a Glance” Dashboard for Hiring Sample
• On discover tab, search ”Location:*”
• Save the save search as “HiringBusinessData”
• On visualization tab, select “Pie Chart”
30 9/29/17
Create Your Dashboard
Create “Business at a Glance” Dashboard for Hiring Sample
• Select “HiringBusinessData” save search
• Choose “Split Slices”• Select “Terms” as
Aggregation• Select “Location” as Field• Save pie as “Hiring
Request by Location”
31 9/29/17
Create Your Dashboard
Create “Business at a Glance” Dashboard for Hiring Sample
• On dashboard tab, add a new dashboard
• Select “Hiring Requests by Location”
• Save the dashboard “Business at a Glance”
• Create other visualization and add to the dashboard
32 9/29/17
Create Your Dashboard
Install BPM Analytics Technology Preview
• Install BPM 8.6
• Download add-ons from: https://github.com/ibmbpm/bpm_analytics
• Contents:• EventSummaryAgent.tar
• Monitors the new events added in the configured elasticsearch index and then generate summary events
• BPMDashboardKibana.zip• Contains five Kibana dashboards
• IBM Business Process Manager Analytics.pdf• Install instructions
CF 2017.09 (8.6)
IBM Cloud University 2017 | October
Please Note
IBM’s statements regarding its plans, directions, and intent are subject to change or withdrawal without notice at IBM’s sole discretion.
Information regarding potential future products is intended to outline our general product direction and it should not be relied on in making a purchasing decision.
The information mentioned regarding potential future products is not a commitment, promise, or legal obligation to deliver any material, code or functionality. Information about potential future products may not be incorporated into any contract.
The development, release, and timing of any future features or functionality described for our products remains at our sole discretion.
Performance is based on measurements and projections using standard IBM benchmarks in a controlled environment. The actual throughput or performance that any user will experience will vary depending upon many factors, including considerations such as the amount of multiprogramming inthe user’s job stream, the I/O configuration, the storage configuration, and the workload processed. Therefore, no assurance can be given that an individual user will achieve results similar to those stated here.
IBM Cloud University 2017 | October
Notices and disclaimersCopyright © 2017 by International Business Machines Corporation (IBM). No part of this document may be reproduced or transmitted in any form without written permission from IBM.
U.S. Government Users Restricted Rights — use, duplication or disclosure restricted by GSA ADP Schedule Contract with IBM.
Information in these presentations (including information relating to products that have not yet been announced by IBM) has been reviewed for accuracy as of the date of initial publication and could include unintentional technical or typographical errors. IBM shall have no responsibility to update this information. This document is distributed “as is” without any warranty, either express or implied. In no event shall IBM be liable for any damage arising from the use of this information, including but not limited to, loss of data, business interruption, loss of profit or loss of opportunity. IBM products and services are warranted according to the terms and conditions of the agreements under which they are provided.
IBM products are manufactured from new parts or new and used parts. In some cases, a product may not be new and may have been previously installed. Regardless, our warranty terms apply.”
Any statements regarding IBM's future direction, intent or product plans are subject to change or withdrawal without notice.Performance data contained herein was generally obtained in a controlled, isolated environments. Customer examples are presented
as illustrations of how those customers have used IBM products and the results they may have achieved. Actual performance, cost, savings or other results in other operating environments may vary.
References in this document to IBM products, programs, or services does not imply that IBM intends to make such products, programs or services available in all countries in which IBM operates or does business.
Workshops, sessions and associated materials may have been prepared by independent session speakers, and do not necessarily reflect the views of IBM. All materials and discussions are provided for informational purposes only, and are neither intended to, nor shall constitute legal or other guidance or advice to any individual participant or their specific situation.
It is the customer’s responsibility to insure its own compliance with legal requirements and to obtain advice of competent legal counsel as to the identification and interpretation of any relevant laws and regulatory requirements that may affect the customer’s business and any actionsthe customer may need to take to comply with such laws. IBM does not provide legal advice or represent or warrant that its services or products will ensure that the customer is in compliance with any law.
IBM Cloud University 2017 | October
Notices and disclaimers continuedInformation concerning non-IBM products was obtained from the suppliers of those products, their published announcements or other publicly available sources. IBM has not tested those products in connection with this publication and cannot confirm the accuracy of performance, compatibility or any other claims related to non-IBM products. Questions on the capabilities of non-IBM products should be addressed to the suppliers of those products. IBM does not warrant the quality of any third-party products, or the ability of any such third-party products to interoperate with IBM’s products. IBM expressly disclaims all warranties, expressed or implied, including but not limited to, the implied warranties of merchantability and fitness for a particular, purpose.
The provision of the information contained herein is not intended to, and does not, grant any right or license under any IBM patents, copyrights, trademarks or other intellectual property right.
IBM, the IBM logo, ibm.com, Aspera®, Bluemix, Blueworks Live, CICS, Clearcase, Cognos®, DOORS®, Emptoris®, Enterprise Document Management System™, FASP®, FileNet®, Global Business Services®,Global Technology Services®, IBM ExperienceOne™, IBM SmartCloud®, IBM Social Business®, Information on Demand, ILOG, Maximo®, MQIntegrator®, MQSeries®, Netcool®, OMEGAMON, OpenPower, PureAnalytics™, PureApplication®, pureCluster™, PureCoverage®, PureData®, PureExperience®, PureFlex®, pureQuery®, pureScale®, PureSystems®, QRadar®, Rational®, Rhapsody®, Smarter Commerce®, SoDA, SPSS, Sterling Commerce®, StoredIQ, Tealeaf®, Tivoli®Trusteer®, Unica®, urban{code}®, Watson, WebSphere®, Worklight®, X-Force® and System z® Z/OS, are trademarks of International Business Machines Corporation, registered in many jurisdictions worldwide. Other product and service names might be trademarks of IBM or other companies. A current list of IBM trademarks is available on the Web at "Copyright and trademark information" at: www.ibm.com/legal/copytrade.shtml.
