Daniël  Palstra,  Senior  Consultant  Yenlo    

How  to  Monitor  Business  Processes  with  WSO2  ESB  

and  WSO2  BAM  

18  March  2014  

Sinthuja  Ragendran,  So7ware  Engineer  WSO2    

2  

About  the  Presenters  

Sinthuja  Ragendran    So0ware  Engineer  at  WSO2  Sinthuja  joined  WSO2  in  December  2011.  She  is  a  soEware  engineer  in  the  Data  technologies  team  where  she  mainly  focuses  on  the  Business  AcHvity  monitor.  She  holds  a  B.Sc.  Engineering  Graduate  from  the  Department  of  Electronics  and  TelecommunicaHon  Engineering,  University  of  Moratuwa  (Sri  Lanka)  and  is  also  a  Sun  CerHfied  Java  programmer  and  has  completed  the  BCS  (BriHsh  Computer  Society)  ExaminaHons.  

 

 

Daniël  Palstra  Senior  consultant  at  Yenlo  Daniël  Palstra  is  WSO2  Expert  at  Yenlo  and  has  a  bachelor  degree  in  InformaHon  Technology  and  over  6  years  experience  working  for  systems  integrators  with  global  exposure.  In  his  current  role  he  has  a  strong  focus  on  building  enterprise  integraHon  layers  based  on  WSO2  products.  Daniël  is  a  presenter  at  numerous  internaHonal  workshops.  

Agenda  

3  

In  this  webinar,  Daniël  Palstra  and  Sinthuja  Ragendran  will  demonstrates  how  organizaHons  can  plug-­‐in  WSO2  BAM  into  WSO2  ESB  and  immediately  gain  insight  into  the  performance  of  business  processes.  

๏   Case  Study  

๏   Student  RegistraHon  Process  

๏   SoluHon  with  WSO2  ESB  and  WSO2  BAM  

๏   Demo  

Case  Study  

4  

๏  University  in  the  The  Netherlands  

๏  Students  apply  for  studies  

๏  Before  start  of  the  semester  students  need  to  be  registered  in  mulHple  systems  

๏  Administrators  need  to  have  insight  in  succeeded  and  failed  system  registraHons  

Student  registraHon  proces  

1.  Student  applies  for  a  study  

2.  Student  is  approved  to  follow  study  

3.  Student  needs  to  be  registered  in  the  Base  AdministraHon  

4.  Student  needs  access  to  Electronic  Learning  Environment  

5.  Student  needs  be  registered  in  system  X  

6.  Student  needs  to  be  registered  in  another  system  

5  

6  

๏  Conclusion:  One  student  registraHon  must  end  up  in  mulHple  systems.  

SoluHon!  

7  

 

 

 

 

 

๏  Create  reusable  messages  and  components  

๏  Decouple  systems  from  each  other  

๏  Deliver  messages  reliable  

Event Driven

Architecture!

8  

Produce  student    

registraHon  events

 

Event Driven Architecture!

Consume  student    

registraHon  events

 

9  

 

 

 

 

 

๏  Easily  add  systems  without  changing  exisHng!  

๏  Messages  are  reliable  delivered  to  consumers  by  Topics  

๏  Topics  are  durable  for  consumers  loosing  and  resuming  connecHons.  

10  

 

 

                                                                         But  wait………  

 

11  

Where  did  all  the  student  messages  go  to?  

What  was  the    message  payload?  

What  went  wrong?  

Which  systems  did  not    process  the  student  registraHon?    

Which  systems  did  receive    the  student  registraHon?  

Where  do  I  need  to    start  my  invesHgaHon?  

Why?  

……Common  SOA  and  EDA  administraHon  quesHons  

New  challenge!  

12  

 

๏  Monitor  messages  flowing  through  message  producers,  consumers  and  topics.  

SoluHon:  BAM!  

13  

 

Event Driven

Architecture+Monitoring

Student  RegistraHon  system  

14  

 

๏ Are  Proxy  Services  and  API’s  on  the  ESB  

๏ Use  the  BAM  Mediator  for  custom  monitoring  

๏  Gather  specific  student  informaHon  

๏  Gather  specific  message  informaHon  

๏ Require  a  BAM  Profile  

๏  Describes  what  data  to  publish  to  BAM  

๏ Are  monitored  automaHcally  by  BAM  Data  Agents  

 

 

Producers  

Topic  

15  

 

๏ Monitored  by  using  BAM    

๏  Input  Event  Adaptors  

๏  Event  Builders  

๏  Event  Formakers  

๏  Output  Event  Adaptors  

๏ CorrelaHon  is  done  by  the  acHvity  id    

 

Consumers  

16  

 

๏ Are  Proxy  Services  on  the  ESB  

๏ Can  be  any  thirdparty  JMS/  AMQP  clients  

๏ Publish  to  BAM  via  

๏  BAM  streams  in  ESB  

๏  BAM  REST  API  

๏  BAM  ThriE  client  

Monitor  producers,  topics  and  subscribers  

17  

 

๏ WSO2  Enterprise  Service  Bus  ๏  MulHple  “Data  Agents”  

๏  BAM  Mediator  

๏  BAM  Profiles  

๏  BAM  Streams  

๏ WSO2  Business  AcHvity  Monitor  ๏  AcHvity  Monitoring  

๏  ThriE  Event  Receiver  

๏  REST  API  

Why  WSO2  Business  AcHvity  Monitor?    ๏ Fully-­‐open  source  soluHon  for    

๏  aggregaHng    ๏  analyzing  data    ๏  presentaHon  informaHon    

about  business  acHviHes    

๏ Cloud-­‐enabled,  lightweight,  developer-­‐friendly  and  easy-­‐to-­‐deploy  

 ๏ High  performance  data  capture  framework    ๏ ElasHcally-­‐scalable  data  analysis  powered  by  Apache  Hadoop  

 ๏ Pre-­‐built  Data  Agents  for  WSO2  products     18  

๏ Collect  events/data  from  all  components  in  the  system    ๏ Have  a  unique  ID  for  enHre  acHvity  –  AcHvity  ID    ๏ Correlate  the  events  together  with  the  unique  AcHvity  ID    ๏ Visualize  the  path/status  of  the  AcHvity  

What  is  AcHvity  Monitoring?  

19  

WSO2 Business Activity Monitor

Activity Event

Activity Event Activity Event Activity Event

Activity Event

WSO2  BAM  in  Student  RegistraHon  

20  

How  to  send  events  to  WSO2  BAM?  

Where  the  events  stored  in  WSO2  BAM?  

๏  Compatible with CEP/BAM ๏  Asynchronous & non-blocking ๏  Thrift for high performance message throughput

๏  Receives data and stores it in Cassandra ๏  Scalable, big data repository ๏  Asynchronous & non-blocking ๏  Combination of Cassandra, Thrift and the non-blocking nature results in extremely fast writes ๏  Shared with WSO2 CEP for real time analysis ๏  Supports Thrift & REST API

Data Agents WSO2 BAM (Data Receiver)

Data Agents WSO2 BAM (Data Receiver)

Data stores (Cassandra)

21  

๏  Powered by Apache Hadoop with querying/managing through Apache Hive

๏  Runs on local Hadoop node or delegates to Hadoop cluster ๏ Scalable analytics ๏ Cluster can range from a couple of nodes to 1000s

๏  Scripts are based on an easy-to-learn, SQL-like query language

๏  Activity Dashboard in WSO2 BAM make use of Indexes to correlate the activities and query directly from Cassandra database.

How  to  correlate  acHviHes?  

Analyzer Engine

Hadoop Cluster Data Store (Cassandra, RDBMS, etc)

22  

๏  Activity Dashboard for correlating activities

๏  Gadget Portal for the dashboard

๏  Gadget Generation Wizard – Tool for generating custom gadgets for the dashboard

๏  Plug in your own report server/ dashboard server

How  to  visualize  the  AcHviHes?  

Data Store (Cassandra, RDBMS, etc)

Dashboard Server

Report Server

23  

DEMO  

24  

25  

About  WSO2  ๏  Global  enterprise,  founded  in  2005  by  

acknowledged  leaders  in  XML,  web  services    technologies,  standards    and  open  source  

๏  Provides  only  open  source  plarorm-­‐as-­‐a-­‐service  for  private,  public  and  hybrid  cloud  deployments  

๏  All  WSO2  products  are  100%  open  source  and  released  under  the  Apache  License  Version  2.0.  

๏  Is  an  AcHve  Member  of  OASIS,  Cloud  Security  Alliance,  OSGi  Alliance,  AMQP  Working  Group,  OpenID  FoundaHon  and  W3C.  

๏  Driven  by  InnovaHon  

๏  Launched  first  open  source  API  Management  soluHon  in  2012  

๏  Launched  App  Factory  in  2Q  2013  

๏  Launched  Enterprise  Store  and  first  open  source  Mobile  soluHon  in  4Q  2013  

What  WSO2  delivers  26  

27  

About  Yenlo  ๏  Global  enterprise,  founded  in  2007  

with  an  internaHonal  focus  on  delivering  integraHon  soluHons  based  on  Java  open  source    

๏  #1  in  the  field  of  IntegraHon  SoluHons  

๏  #1  in  Managed  Services  for  middleware  environments  

๏  #1  Global  Strategic  Alliance  partner  of  WSO2  

 

 

 

๏  WSO2  Product  Support  

๏  WSO2  Development  

๏  WSO2  QuickStarts  

๏  WSO2  Training  &  CerHficaHons  

๏  WSO2  24/7  Managed  Services  

๏  WSO2  Events  

What  Yenlo  delivers  28  

Enterprise  Architecture    

SoEware  Development   Managed  Services  

WSO2  Product  Support   WSO2  Development  Support   WSO2  QuickStart  

WSO2  Training  &  CerHficaHons   WSO2  Managed  Services   WSO2  Events  

29  

Workshops  &  Trainings  2014  Free  Workshop  powered  by  WSO2,  SUSE  &  Yenlo:    Join  the  Connected  Business  RevoluBon  

Trainings:  WSO2  Enterprise  Service  Bus,  WSO2  API  Manager,  WSO2  Business  AcBvity  Monitor,  WSO2  IdenBty  Server,  WSO2  Governance  Registry,  WSO2  Business  Rules  Server,  WSO2  Business  Process  Server.      

     

     

     

     

     

   

 

See  all  workshop  and  training  details  on  www.yenlo.com  

San  Francisco/  New  York    Date  

   Developer  Training  

   Expert    Training  

13  October  2014   ESB  

14  October  2014   API   ESB  

15  October  2014   BAM   ESB  

16  October  2014   IS   API  

17  October  2014   GOV   API  

20  October  2014   BRS   BAM  

21  October  2014   BPS   BAM  

22  October  2014   GOV   IS  

23  October  2014   GOV   IS  

Date   LocaNon  

25  March  2014   Amsterdam  

2  April  2014   Paris  

9  April  2014   Dublin  

23  April  2014   Madrid  

14  Mei  2014   London  

21  Mei  2014   Warsaw  

4  June  2014   Stockholm  

11  June  2014   Berlin  

16  July  2014   Stukgart  

3  September  2014   Prague  

10  September  2014   Manchester  

17  September  2014   Vienna  

24  September  2014   Brussels  

Amsterdam  /  London    Date  

   Developer  Training  

   Expert    Training  

2  June  2014   ESB  3  June  2014   API   ESB  4  June  2014   BAM   ESB  5  June  2014   IS   API  6  June  2014   GOV   API  

9  June  2014   BRS   BAM  10  June  2014   BPS   BAM  11  June  2014   GOV   IS  12  June  2014   GOV   IS  

17  November  2014   ESB  18  November  2014   API   ESB  19  November  2014   BAM   ESB  20  November  2014   IS   API  21  November  2014   GOV   API  

24  November  2014   BRS   BAM  25  November  2014   BPS   BAM  26  November  2014   GOV   IS  27  November  2014   GOV   IS  

30  

http://www.slideshare.net/YenloBV

Download  the  webinar  presentaHon  on  slideshare:  

Contact  us  !  

DEMO  

32  

Custom  BAM  Stream  (1)  

33  

Custom  BAM  Stream  (2)  

34  

BAM  Mediator  

35  

Custom  Hive  Scripts  (1)  

36  

Custom  Hive  Scripts  (2)  

37  

Custom  Dashboard  

38  

AcHviHes  

39  

AcHvity  Events  

40  

Content  AcHvity  

41  

 

๏  Monitor  the  Network  bandwidth  

๏  Monitoring  Online  transacHons  

๏  Monitor  the  Health  of  Servers    

๏  Monitor  the  Service  StaHsHcs  

๏  Monitor  the  MediaHon  StaHsHcs  

๏  Mobile  app  Monitoring  

๏  Geo  Tracking    

and  lot  more...    

Some  other  WSO2  BAM  usecases  

42