White Paper3

8/6/2019 White Paper3

1/15

2007. HCL Technologies Ltd. No information may be reproduced or retransmitted withoutexplicit permission from the company.

Real-world Enterprise SOA implementations

Using JCAPS

White Paper

Version Date Change Description Prepared / Revised By

1.0 08/09/2007 First Draft Ranga Srinivasan

SOA & Middleware

HCL Technologies

Confidential Information

Information contained within this white paper is confidential and proprietary to HCL and should not bedisclosed to anyone other than the recipients.


2/15

Table of Contents

Real-world Enterprise SOA implementations Using JCAPS

Page 2 of 15

Company Confidential HCL Technologies Ltd

Table of Contents

1. Introduction.....................................................................................................31.1. Target Audience...........................................................................................41.2. JCAPS Highlights ..........................................................................................41.3. SOA and JCAPS............................................................................................4

2. Application Integration through eGate and eWays ...........................................52.1. eWays - Adapters.........................................................................................5

3. SOA through Business Process Management eInsight ...................................64. Version Control ................................................................................................75. Deployments, Management and Monitoring .....................................................76. Adopt Best Practices ........................................................................................7

6.1. Set Up Integration Competency Center (ICC)...................................................86.2. Development Methodology Standards .............................................................86.3. Set Up Integration Testing Methodology..........................................................96.4. Messaging Standards and Common Information Model......................................96.5. Set up Generic Integration Services (Logging, Auditing and Error Handling) ........106.6. SOA Naming Standards.................................................................................10

7. Case Studies.....................................................................................................11

7.1. Case Study 1 ...............................................................................................117.2. Case Study 2 ...............................................................................................13

8. Conclusion........................................................................................................15


3/15

Introduction


Page 3 of 15


1.IntroductionBusiness enterprises have grown in size and markets they serve and their application systemshave also become more complex. The need for application programming interfaces betweenthese systems to share the data stored in these heterogeneous systems has escalated.Enterprise Application Integration (EAI) provides the means of integrating these disparate

applications within the enterprise. Service Oriented Architecture (SOA) enables reuse ofservices. Emerging EAI tools help build integration as services and maximize reuse and thereturn on investment (ROI).

Some of the important driving factors for middleware and integration are:

Growing adoption of packaged applications Base of business critical legacy systems Multiple platforms, protocols and technologies Internet enabling of applications through portals, self-service websites and so onThe solution to these requirements is using an EAI product which shields business applicationsfrom the integration complexities by adding a layer that allows them to communicateseamlessly.

As shown in figure 1, EAI system acts as an Information backbone for the organization. Once inplace, the EAI backbone becomes reusable and strategic asset, readily meet the future needs,typically to construct complex business processes. EAI also reduces the cost of change inadding new applications, supporting new processes all without impacting the users ordevelopers.

Figure 1: A typical EAI Sys System, where EAI acts as a backbone for all the application

integration

SunJava Composite Application Suite (Sun JCAPS) provides an open source platform whichenables enterprises to maximize their IT assets through easy integration of their IT systemssuch as ERPs, CRMs and custom applications, automating the business processes and easypartner integration.

The scope of this paper is to highlight HCLs real world SOA implementation using Sun JCAPS ina financial industry and beverages industry.


4/15

Introduction


Page 4 of 15


1.1.Target AudienceThis document is intended for Sun JCAPS technical analysts/architects and developers.

1.2.JCAPS HighlightsThe various types of integration provided by Sun JCAPS are:

Applicat ion Integrat ion

This provides a framework of collection of technologies (Sun JCAPS eGate, JMS and eWays) thattogether provide near real-time integration through application programming interfaces e.g.integration to SAP ERP system through BAPI eway.

Data Integrat ion

Sun JCAPS eTL provides an integrated access to variety of relational and non-relationaldatabase sources and also to mainframe and flat file data. In this approach enterprises stillmaintain separate applications but opt to use EAI tools to move data from one data source toanother.

Process Integrat ion

Business Process integration (Sun JCAPS eInsight) works at the higher abstract Business Flowlevel within the organization. The pre-condition for this is that, the applications within theenterprise must be already communicating with each other. Process Integration interconnectsthe Business Processes within the organization thereby actually managing the flow ofinformation between the applications. With the help of the Business Process Integration toolsthe business managers can define, monitor and change business processes through a graphicalmodeling tool.

Sun JCAPS eBAM provides business activity monitoring. Required key Business events arecollected from the message bus (eGate or eInsight business processes). These business eventscan be aggregated or summarized based on specific key performance indicators (KPIs) andpresented on a dashboard.

Business-to-Business Integrat ion

B2B Integration works across the businesses to implement the real e-business by integratingcustomers, suppliers and partners. Sun JCAPS eXchange enables partner integration in the SunJCAPS Suite.

1.3.SOA and JCAPSSun JCAPS enables services to be defined and reused across the tiers. Sun JCAPS conforms tomost of the SOA web services standards. SOA being loosely coupled Architecture offers betterscalability and eliminates proprietary technologies by using product independent industrystandards/specifications. Enterprises can deploy new applications faster by using exposed

services rather than building them from scratch.


5/15

Application Integration through eGate and eWays


Page 5 of 15


2.Application Integration through eGate and eWaysJCAPS provides rich integration through eWay adapters and necessary data transformation. Thedata transformation can be achieved through JAVA/XSLT. Integration applications are deployedat runtime on the J2EE applications servers (Sun SeeBeyond Integration server and Sun JavaSystem Application Server and BEA Web Logic Application Server). Deploying applications ontoBEA Weblogic application server can not be done through eDesigner.

Application services are built using eDesigner graphical interface. eDesigner is the common IDEfor all the product suites in JCAPS. eDesigner provides easy to use drag and drop features tobuild the business rules, build the graphical representation of the application. Services can bebuilt using Java (JCDs) can be exposed as web services. The web services are enabled throughHTTP/s protocol.

Integration applications can use messaging to communicate across application. Messagingallows applications to be loosely coupled. Java Message System Server (Sun SeeBeyond JMSServer and Sun Java System Message Queue, IBM Websphere JMS implementation support)provides reliable messaging in JCAPS. Sun SeeBeyond JMS server follows JMS 1.1 specificationand supports point-to-point (p2p) messaging via JMS queues implementation and multiplelisteners through JMS topics (pub-sub).

The JMS Server supports many of the performance features which can be tuned forperformance:

subscriber pooling (processing messages concurrently by only one receiver even thoughmultiple receivers are registered on the same queue; ServerSessionPoolsize andConcurrency set to Connection Consumer)

Throttling ( property which alerts the message producers and stops reading furthermessages as the messages are piled not subscribed)

The web services have a slight overhead compared to straight event driven message basedarchitecture. Wherever synchronous response is not needed and communicating systems cansupport JAVA/JMS, it is best to use the event driven, asynchronous JMS. Web service can beconsidered an option if there is an opportunity for multiple applications use this service

synchronously.

2.1.eWays - AdaptersSun JCAPS provides many adapters for easy integration with external applications ERPs(SAP,Oracle App), CRM (Siebel) and database (Oracle, SQL*Server)

When processing large volumes (of over 5-8 million records) of data through the JCD anddatabase eWays accessing the data through JDBC, performance issue is one of the mainconcern. Of course, care should be taken to analyze the database indexes for performance. Inmany instances, we have seen eTL tools do the heavy duty extraction and transformation andupload in fraction of the time taken through the first approach.

With MQ eways, appropriate version of the MQ is must as we have seen connections remainopen (observed with MQ 5.3) and patches to the MQ are needed to ensure open connectionsclosed and released to the connection pool.

SAP eWays use BAPIs, RFC and IDocs for communicating with SAP. We have also use SAPBusiness Transaction Event mechanism to have the SAP publish the data as trigger data forJCAPS. JCAPS would in turn look at the trigger data and process as needed. For postingtransactions to SAP, one can use the BAPI/Idoc.


6/15

SOA through Business Process Management eInsight


Page 6 of 15


Communicating with Siebel CRM can be done through Siebel EAI or through HTTP eWays.Typically large volumes can not be pushed through Siebel EAI or HTTP, so appropriate messagestructure (XSD) need to be in place to batch the messages and push.

3.SOA through Business Process Management eInsight

JCAPS eInsight enables easy modeling and orchestration of business processes. Business rulescan be created graphically. At runtime, the business processes instances can be persisted into adatabase. Any failed step in the business process can be corrected and restarted.

The business processes can be exposed as web services. Legacy services can be wrapped andcan be exposed as web services. Event based decisions need to be incorporated into thebusiness process if the business process is exposed as web services with multiple operations.

The business processes can be enabled with human workflow as well. For these the useractivities need to be defined as sub business processes. The sub-business processes can becreated manually with Receive and Reply activities or through WSDLs as well.

JCAPS eInsight allows exceptions to be caught at the business process level and namedexception level. Named exceptions are exceptions configured from WSDLs or JAVA activities.Named exceptions configured from the java processes/web services activities are caught andhandled properly from the tool.

The eInsight modeling tool creates the X-path expressions for the rules and are BusinessProcess Execution Language (BPEL) Compliant. However, the BPEL rules are not easily portedto other tools.

The transaction management is handled through XA. The activities within the business processbehave properly i.e. rolling back in case of failure and committing in case of success forexample on activities comprising databases and message servers. The transactions processedthrough invoking web services in the business process do not behave properly. The WS-

Transaction support is not yet present.

Some of the emerging WS- standards (e.g. Ws-Routing, WS-Notification) are yet to find spacein JCAPS eInsight. So, work-around need to be in place to handle the same as well. Forexample, a requirement could be any service that makes a change to the core data shouldpublish data through WS-Notification Broker standard. To overcome this support for thisstandard in JCAPS, we publish the input message as a JMS Topic after successful return codefrom the service. The message published is consumed by interested applications.

The web services exposed from the business processes are by default are exposed to HTTPprotocol. JCAPS is yet to support web services through JMS protocol (SOAP/JMS).

To handle the routing capability, we have approached using the canonical schema where inmessages are sent across to the bus. Router process at the local ESB picks up the message andif the message is to be processed outside local ESB (derived based on routing rules stored inpersistent database), the message would be routed to the central ESB. If has to be processedthrough local ESB, message would be sent to the relevant process as an event.


7/15

Version Control


Page 7 of 15


4.Version ControlJCAPS allows all the components (JCDs, Business Processes, CMAPS and deployment profiles)to be developed in eDesigner. The JCAPS artifacts are stored in Sun SeeBeyond JCAPSrepository. JCAPS itself provides version control capabilities for all the artifacts in the repositoryenabling maintaining multiple versions.

Enterprises prefer to have single source control system. JCAPS does not integrate withcommercial/open source version control software yet, although, there are plans for such in thefuture release. In our experience, on a daily basis, the projects are exported from developmentor testing environment and checked into Subversion/CVS. This would be fall-back in case of anyunforeseen issues with the code, one can get the latest project from Subversion and importthem and build on.

5.Deployments, Management and MonitoringDedicated environments for development, testing (system, integration and user acceptance)needs to be in place. This includes the development of artifacts (JCAPS repository, JCAPSProjects and related configurations) and runtime instances.

Appropriate change control mechanism is must for migrating from one environment to the otherenvironment. Application administrators need to be the sole administrators to manage theenvironments.

The default Sun SeeBeyond JMS server does not support clustering. But Sun JCAPS has anadditional Sun Java Message Service Grid (JMS Grid) which supports JMS clustering for high-availability. The load balancing can be achieved with or without the JMS Grid.

JCAPS Sun SeeBeyond Integration server can be deployed in clustered fail-over environments.For the Sun SeeBeyond Integration server, it needs the scripts at the operating system (OS)level for fail-over. Sun SeeBeyond Integration server does provide load balancing when eInsightbusiness processes are deployed across multiple Sun SeeBeyond Integration servers.

JCAPS developed projects can also be deployed under other J2EE capable application serverssuch as Web Logic or Sun Application Server. These provide application level clustering.

JCAPS did away with having a repository running for managing and monitoring compared toearlier versions. JCAPS eManager connects to the given application server/integration server foreasy management and monitoring. Business process instances can be mentored through theeManager. Application server configuration and management of the application can also behandled with eManager. However, lot of enterprises would have their own managementsoftware to manage the application servers. Sun SeeBeyond integration server and Sunapplication server can work with JMX API. Enterprises management software can integrate withSun SeeBeyond Integration server and Sun Application server to manage through the standardJMX API.

6.Adopt Best PracticesAny new tool in the systems landscape introduces certain challenges and new learning for theIT staff and introduces new processes. It is best to set up set of best practices. To set up bestpractices involve people with enough great credentials (been there and done that Sun and itspartner HCL) into the team from the inception stage of the project.


8/15

Adopt Best Practices


Page 8 of 15


6.1.Set Up Integration Competency Center (ICC)The Integration competency center (ICC/CoE) helps enterprises to evangelize the integrationmethodologies and standards to other teams in the organization. Typically ICC guides theother teams in the enterprise by helping the development teams with the integration skills,coming up with relevant training materials, methodologies and templates. ICC builds certaincommon services for the enterprise. These services could be business services or technicalservices. This would be available for use by the other teams in the enterprise.

6.2.Development Methodology StandardsIt is very important to standardize on the methodology to be adopted for the development. Thisprovides the necessary templates and processes to be adopted during the development andavoids developing from scratch.

We at HCL have applied 4-D (define, Design, Develop and Deploy) methodology to the projects.4-D Methodology is focused on implementation best practices for integration projects. 4-DMethodology has the flexibility to complement and co-exist with a customers or a consultingpartners established methodologies.


9/15



Page 9 of 15


Figure 2: 4D Methodology

For projects requiring one of type integrations, prototyping becomes important and adequatetime needs to be arranged.

6.3.Set Up Integration Testing MethodologyTest plans should be created for entire testing cycles unit testing, integration testing and useracceptance testing. It is best to identify the test data as well during design. Test cases shouldbe documented, results recorded and any bugs fixed should be regression tested.

HCLs CrossFit supports customizable testing process and enables the test automation. CrossFitis extensible via cartridges to interface with custom applications and standards. It addressesthe testing needs of distributed systems that are typical in enterprise application integrationscenarios.

6.4.Messaging Standards and Common Information ModelStandard messaging across applications enables services to look at the part of the messageinstead of going through the entire message to be processed, thus enhancing the performance.

Each industry there could be set of specific standards or one could come up a custom standardfor the message. Some common messaging used across the industry is:

XML based canonical data model relevant to the domain (e.g. ebXML) SWIFT HL7 ANSI X12/EDIFACTCommon Information model helps in defining the information model and its relationship amongdifferent entities. For example, due to different business scenarios, a Customer entity in SystemA is represented by set of X attributes; similarly a Customer entity in System B could berepresented by set of Y attributes. Common Information model brings about identifying theattributes for a Customer and its relationships and representing them in the XML schema.

Common Information model provides the following:

Data exchange between applications is standardized. This could be specific to the industrydomain (e.g. SWIFT/HL7).

Data exchange between disparate applications become seamless as any new applicationneeding integration can just exchange information adhering to the common format

Definition of data model to adhere to common information model is major task during theimplementation. Frequent changes to applications and interfaces to CIM would make thehandling of the messages complex as versioning need to be in place.

In many cases one can come up with a normalized information model as below:


10/15



Page 10 of 15


The above information model helps applications to easily route the data to target applicationsfor processing further.

6.5.Set up Generic Integration Services (Logging, Auditing andError Handling)

Transactions processed through the integration bus can error due to data formatting issues,business validation issues and connectivity with external systems issues etc. Auditing and Errorhandling are critical processes that help in reconciling the transactions. Auditing, Logging andError handling are provided by common services across the enterprise.

At HCL, we have developed Generic Integration services that provide logging, auditing anderror handling functions.

It is a best practice build this functionality or get from Sun or its preferred partners (HCL). This

provides a jump-start capability for integration projects.

Auditing captures the flow of the events in the integration bus. Certain key information (createduser, source system, transaction details, timestamp etc) of the transactions are captured andstored in persistence storage. Certain other integration products provide out-of-the-boxfunctionality on auditing as the messages are processed through the service bus. Sun JCAPSdoes not have such functionality.

Errors captured while processing the transactions are logged into files or database for lateranalysis. Errors can also be played back or reprocessed. Time bound errors may not be able tobe reprocessed as their validity expires after sometime. Appropriate coding techniques andmodel elements (e.g. try/catch in JCD and Scope and Exception handling in eInsight businessprocess model) need to be in place to capture the errors with proper stack trace and error textto aid in resolving the error.

6.6.SOA Naming StandardsEstablish naming standards for the components developed and publish to the team This wouldhelp the team building the components and artifacts for easier maintainability and any futureenhancements and modifications.


11/15

Case Studies


Page 11 of 15


7.Case Studies7.1.Case Study 1Implementation of Trade Promotion Management

One of the leading confectionery companies in the world has strong regional presence in the

beverages market in Americas and Australia. With origins stretching back over 200 years, thiscompany employs 60,000 people is trying to integrate disparate systems due to mergersacquisitions and various partners. A current process to measure the trade promotions given toits partners is not effective. By implementing the TPM, this company expects to measure thetrade spent accurately and target effective marketing programs.

Proposed Solut ion

Siebel solution is implemented for the Trade Promotion Management (TPM) across NorthAmerica

A Common Information Model (CIM) has been defined for applications to communicatewithin the enterprise.

An EAI solution based on SeeBeyond JCAPS has been conceived to extract the data fromdifferent source systems

Informatica was used to cleanse the data, remove the duplicates and harmonize the recordsand update the CIM golden copy.

Middleware SeeBeyond JCAPS (older version of Sun JCAPS) extracted the data from thecleansed datastore and updated the Siebel Trade Promotion Management (TPM) system


12/15

Case Studies


Page 12 of 15


Figure 3: SOA / ESB Solution

After the trade promotions are set in Siebel TPM, relevant incentives for trade effectiveness areto be sent back to retailers and other suppliers. This includes getting the vendor data fromsource systems for issuing payment. Also, to get the TPM system to be capable of measuringthe trade effectiveness the invoice and other performance information had to be brought infrom source systems.

Business transaction Event in SAP was enabled to send the event triggers on payments,invoices. Appropriate real-time data exchanged between the source systems (SAP) and SiebelTPM. Services were exposed for invoices and vendors wrapping the SAP BAPIs. Other JCAPSservices were built to handle payment and deduction data to be posted to SAP.


13/15

Case Studies


Page 13 of 15


7.2.Case Study 2SOA Based Solut ion for F inancia l Services F irm

One of the leading expert in the securities exchange industry in the Nordic region, clientprovides technology and services to companies in the securities industry around the globe. HCLis partnering with this client on creating the product for handling trading platform back-officessupport, removal of legacy systems. The solution is based on HCL One Foundation architecture

as depicted below.

Some of the issues faced with the exisitng

implementation: Time to market new products Conforming to regulatory compliance Extending services to application service provider (ASP) and Business Service Provider

(BSP) model

Changes to existing system difficultProposed Solut ion

Build a SOA and ESB based Securities Processing System Solution to handle ASP and BSP model First phase to get the data synchronization with operation systems to synchronize with SPS

data store (based on HCL One Foundation)


14/15

Case Studies


Page 14 of 15


The system is entirely built on services based architecture. The presentation services were builton .Net thick client. The business services are built on j2ee/Spring/Hibernate framework. JCAPSbusiness process (eInsight) exposed as a service receives the request from the .net client andinvokes the business web services and returns the response to the client. JCAPS MQ anddatabase eWays are used to retrieve the data from source systems and synchronize with theOne Foundation data store by invoking the business services.


15/15

Conclusion


Page 15 of 15

8.ConclusionJCAPS has all integration capabilities enterprises need to embrace the integration andenterprise-wide SOA/BPM initiatives. JCAPS is a open source and based on J2EE standards.JCAPS is yet to support quite a few of the web services standards. JCAPS is proven integrationtool in the market across many of the verticals and does require serious attention for the

enterprises to look into.

Documents

White Paper3