Upload
rob-paok
View
125
Download
0
Embed Size (px)
Citation preview
Technology, Tools and Trends
November 2015
Adapters & EAI
AgendaSession 1:
Background on software integrationBackground on adaptersOverview of adapter technology
Session 2:Overview of trends in adapter technologyJ2CA, Web Services, Semantic AdaptersOverview of adapter development tools
Background On Software Integration
Software integration is the #1 IT priority todayIt is also the most complex and expensive
problem to solveAbout 30% - 40% of all IT costs are integration-
relatedApproximately 1/2 of all integration costs are
adapter-relatedAdapters represent the highest incremental cost
of software integrationEach software system that needs to be
integrated requires an adapter
Background On Software IntegrationThe primary focus of software integration is
information systems (or business applications)The two main areas of software integration are:
DataFunction (Web Services)
The two main types of software integration environments are:Point to Point (Application Server based)Brokered (Integration Platform based)
The extended focus of software integration is business processesIntegrating information systems with business processesAutomating the integrated business processes
Background On Software IntegrationComponents of Software Integration Platform
Foundation componentsHost (Application Server, Integration Server, others)Messaging bus (JMS, others)Object brokers (OMG)Database engines (RDBMS, ODBMS, others)Transaction engines (XA compliant, others)
Value add integration componentsTransformer
Mapping, Object aggregation, Object decompositionBroker
Routing, FiltersAdapters
Current Perspective on EAIEAI is still a big, complex, expensive,
tedious and repeating problem faced universally by companies of all sizesApproximately 1/3 of all IT costs are EAI
relatedApproximately 1/2 of all EAI costs are
adapter-relatedThe definition of EAI is much broader today and
includes software systems within and outside the companies firewalls The term “EAI” has been changed and misunderstood with
the continuing evolution of EAI technologies
Current Perspective on EAIEAI is an ongoing initiative and requires
strategic thinking, strong risk management, and realistic goal oriented implementation plans
Industry studies have reported that the success rates of EAI project is very low < 5%
Existing methodologies, project management practices & processes are not enough to deal with the unique and specific risk management requirements of EAI
Current Perspective on EAINew tools have a strong commitment to
solve one of the toughest problems of EAI namely development of custom/customized adapters and lowering the total cost of ownership
The first step is to define EAI in a clear and simple form easy to understand, comprehend, and implement
Definition of EAIEAI is the process of integrating applications and database systems into a unified software system
capable of supporting:On-demand data exchange Collaboration of shared functionsConsolidation & Transformation of data into
informationAutomating data synchronizationAutomating business process
EAI DomainsThe EAI processes are grouped into a set of
logical processes (“Domains”) which simplifies the definition of specific EAI processesApplication To Application Domain (A2A - Domain)
The process of integrating two or more business applications
Application To Information Domain (A2I - Domain)The process of integrating a business application with
information producers (data warehouses), information access portals, etc.
Application To Process Domain (A2P - Domain)The process of integrating a business application with one
or more work flows, EDI gateways, supply chain automation networks, mobile workforce networks
EAI Domain Reference ModelsEvery EAI Domain is supported by a context
sensitive reference model that can be used to define, design, deploy a customized EAI solution
Each EAI domain reference model contains:A logical architecture that defines the components
and relationships of the EAI processA set of integration patterns that define the
different behavior and interactions of the EAI process
A technology stack containing layers of technologies and standards that enable the EAI process
Business ApplicationBusiness Application
Application to Application (A2A) EAI
Database DatabaseFunction Function
DataIntegration
BrokerAdapter Adapter
WebServicesBroker
Adapter Adapter
SOAP
XML
Transformer
UDDI
Information PortalBusiness Application
Application to Information (A2I) EAI
Database DatabaseFunction Function
Adapter
EIIServer
Adapter
ETL
DataWarehouse
PortalServer
Business Application Partner / SupplierSystems
Business Application
Application to Process (A2P) EAI
Database DatabaseFunction Function
Adapter
WorkflowEngine
Adapter
EDIGateway
XML
DatabaseFunction
Adapter
XML / EDIFACT
Firewall
WFMC
Data Replication Pattern
A2A PatternsData
ChangeDetector
DataExtractor
DataFilter
Data Source Data TargetDataLoader
Other PatternsData Synchronization (Bi-directional Data Replication)
Composite Data Extraction
Collaboration of Shared Services
Information Creation Pattern
A2I Patterns
DataRetriever
Meta-DataCreator
InfoAggregator
Data Source Information SinkInfoPresenter
Other PatternsExtract Transform Load (ETL)
Data Staging
Task Delegation Pattern
A2P Patterns
TaskAssignment
Agent
Task toFunction
MapWorkflow
Manager
Business Application
FunctionManager
Other PatternsBusiness Process Management
Generic EAI Solution Architecture
Foundation (Host, JMS, CORBA, Transaction and Database Engines)
Broker Transformer
Adapter
InformationSystem
Adapter
InformationSystem
Adapter
InformationSystem
Adapter
InformationSystem
Adapter
InformationSystem
Adapter Technology Status Quo
Adapters represent the highest incremental cost of integrationEach applications that needs to be integrated
requires a minimum of one to three adapters (A2A, A2I, A2P)
The average Fortune 1000 company1 has 48 applications and 14 databases for a total of 62 potential adapters
There are few specific tools and platforms available for development and maintenance of adapters
1IDC
Definition of an AdapterThere is no Single Definition of an AdapterBut there is a common understanding of
the adapters behavior and roleAdapters, Connectors, Plug-ins, Web
Services, Integration Components and other proprietary terms are used in different context to mean “Adapter Type Of Functionality”
Adapter definitions tend to be context sensitive
Generic Definition Of AdapterAn Adapter is a software component which
integrates two different instances of software systemsTypically the software systems integrated are
business applications (information systems)In a brokered integration environment a business
application is integrated with the broker by using an adapter
Adapter facilitates integration between the integrated software systemsProvide connectivity between the systemsManage the interactions between the systemsEnforce transformation and other business rules
Background On AdaptersThe concept of adapters was first introduced
by the Enterprise Application Integration (EAI) vendors in the later half of 1990’s
Adapters have become the most important aspect of software integrationEvery software that needs to be integrated
requires an adapterThe average Fortune 1000 company has 48
different business applications and 14 different databases¹
Adapters represent the highest incremental cost of software integration
¹ IDC
Background On AdaptersUntil recently there were no standards for
developing and supporting adaptersJ2EE:JCA is the first system level standard
designed to support adapters on the Java Application Server Major vendors are now supporting J2EE:JCA in their Java
application serversAdapters are becoming a commodity
System standards for adapters will result in more focus on integration functionality
Multiple providers of the same adapter will give users opportunity to compare adapter features
Background On AdaptersGenerally adapters are custom developed from
scratchMost information systems are proprietary or heavily
customizedThe average cost of developing and maintaining an
adapter for packaged applications like SAP, PeopleSoft, etc. over a five year period is over U$ 1 Million
The adapter market has few tools available for developersThe biggest challenge in creating adapter tools is the
expertise and experience required to design, develop, and maintain complex adapters
Adapter InteractionsAn adapter use case defines a specific interaction
between the two software systems integrated by the adapterAn adapter use case is a sequence of one or more
functional stepsThere are two types of steps in an adapter use case
Integration function (Connection, Parsing, Transformation, etc.)Support function (System Logging, Error handling, Audit Trail,
etc.)Two types of use cases
Simple Use Case Involves only one adapter (two software systems)
Complex Use Case Involves two or more adapters (four or more software systems)
Adapter ScenariosAn adapter scenario is a specific instance of
an adapter use case and is bound to a specific runtime environmentAtomic Scenario (Derived from simple use
case)Collaborative Scenario (Derived from
complex use case)It is possible for adapter scenarios to
change between atomic and collaborative scenarios depending on the runtime environment
Simple Use Case (Atomic Scenario)
Simple Adapter Use Case
SAPAdapter
SAP
APIClient
•Check customer•Add new order
Complex Use Case (Collaborative Scenario)
Simple Use Case
Complex Adapter Use Case
SAPAdapter
SAP
SiebelAdapter
Siebel
API
APIClient
Add neworder
Checkcustomer
Adapter Technical Architecture
Technology Stack
Client Interface
Service Gateway
Document Manager
Transformation Manager
Scenario Manager
Information System Manager
Host Environment Manager
Responsible for providing unidirectional or bidirectional access to adapter services
Responsible for parsing, filtering, and formatting data and documents
Responsible for managing single/multiple adapter scenarios involving one or more information systems
Responsible for defining and implementing service contracts and access control
Responsible for data type conversion, data mapping, object aggregation and object decomposition
Responsible for managing connectivity to the information systems
Responsible for managing the lifecycle and collaborating with the operating platform
Adapter Technical Architecture
Technology Stack
Client Interface
Service Gateway
Document Manager
Transformation Manager
Scenario Manager
Information System Manager
Host Environment Manager
Optional Layer
Optional Layer
Adapter Technical Architecture
Technology Stack
Client Interface
Service Gateway
Document Manager
Transformation Manager
Scenario Manager
Information System Manager
Host Environment Manager
SOAP/UDDI
J2EE:JCA
XML / XSLT
Simplifying Adapter ComplexityHow to manage the technical complexity of
adapters?Use model based software development
techniquesPre-defined software models based on integration and
design patterns will provide a robust framework for developing adapters
Developing multiple adapters for one applicationEncapsulate fewer interfaces into one adapter instance
Collaborate with external transformers instead of developing transformer functionality in each adapter
ImplementationAnalysis, Architecture and Design
Adapter Models
Abstract Adapter Model
Structural
Patterns
FunctionalPatterns
ConceptualPatterns
IntegrationPatterns
Concrete Adapter Model
Trends In Adapter TechnologiesAdapters are more sophisticated in functionality
Web services enable application function level collaboration
J2EE Connector Architecture enables plug-and-play support for adapters inside application servers
Adapters can function in a point-to-point or broker based integration environment
Standard adapter models will provide a consistent architecture and design for adapters
Adapter tools will enable faster, cheaper, and easier development and maintenance
Impact of Web ServicesThe most significant impact of web services is
the shift in software application architectureService Oriented Architecture “SOA” is
becoming the foundation for new internet based business applicationsThe concept of “Composing” business
applications from existing software servicesWeb services enables development of “SOA”
compliant applicationsAdapter have an even more critical role in
fulfilling “Services”
Impact of Web ServicesWhere is the Synergy between adapters and
web services?Web services provide a standardized platform for
defining, locating, and invoking business servicesWSDL, UDDI, SOAP
Adapter provide a standardized component mechanism for fulfilling the web services by accessing external software systemsJ2CA
Both web services and adapters are moving towards a common semantic environment enabling rapid “composition”UDEF, RDF
What is J2EE Connector Architecture?J2EE Connector Architecture (J2CA) is a sub-
specification of J2EEJ2CA is a system level interface for hosting
and accessing connectors to non-J2EE applicationsJ2CA connectors extend the J2EE servers ability
to interface with non-J2EE applicationsMost J2EE servers support J2CA specification
IBM WebSphere, BEA WebLogic, Jboss, Oracle ASJ2CA compliant adapters for popular applications
like SAP, PeopleSoft, are now available in the market place
What is J2EE Connector Architecture?J2CA Specification (Ver 1.5) defines system
contracts for managing security, transactions, and connections between J2EE application components and legacy systems
Support meta-data interfaces and bi-directional access between adapters and legacy systems
Message driven access to J2CA connectors is now possible
Moving Towards Semantic AdaptersWhat is a Semantic Adapter?
A Semantic Adapter provides an interface to the semantics of the adapterAdapter Behavior (Services provided by the adapter)Meta-data of the adapted systems data structuresMeta-data of the adapted systems service interfacesConfiguration properties of the adapters runtime
environmentA Semantic Adapter uses an Ontology based
approach to define the semanticsThis involves associating commonly understood
meaning to the definition of adapter properties, behavior, configuration, and associated meta-data
Adapter Development ToolsFundamental Requirements
Enable just in time adapter development and maintenanceCustomers do not purchase adapters ahead of requirementEnd points (information systems) change and therefore pre-
built adapters need customization tooExtend the model based design (MBD) paradigm to
adaptersModel based development significantly reduces
development time and cost, and increases adapter qualityDifferentiate and support abstract adapter models
and concrete adapter modelsEnables multiple implementations of a common abstract
adapter model
Adapter Development ToolsFundamental Requirements
Enable adapter developers to focus on the business requirements of integrating two software systemsDevelop and maintain platform controllers which
encapsulates the operating platform system interfaces from adapter developersEliminates the burden of learning and programming
system level interfaces and standardsE.g: J2EE:JCA controller which encapsulates J2EE:JCA
system contracts from the adapter developerGenerate adapter framework and component shell from pre-
defined adapter modelsEnsures design consistency, reduces bugs and enhances
the quality of adapters
Adapter TechnologyArchitecture:
Adapter architecture based on the concept of functional layersEach layer encapsulates specific adapter functionality
accessible as well defined servicesEach layer can be implemented as an isolated/distributed
component of the adapterArchitecture supports three types of interactions for
an adapterSynchronization
Keeping distributed data models in sync with each otherCollaboration
Enabling software systems to collaborate functionsDelegation
Enabling software systems to execute tasks on behalf of other systems
Adapter TechnologyAdapter Model
Is a collection of one or more adapter patterns managed by the E-Fusion framework Integrated Collaboration Environment (ICE)
E-Fusion supports two types of Adapter ModelsAbstract adapter model
Represents the architecture and design of an adapterAdapters cannot be created directly from abstract
adapter models as there are no implementation details defined
Concrete adapter modelRepresents the implementation reference model of an
adapter Includes one or more run-time scenariosAdapters can be created from concrete adapter models
Adapter Development EnvironmentAdapter Development Environment (ADE)
supports developers throughout the full lifecycle of adapter development and maintenanceFrom design to maintenance
Design, Develop, Assemble, Configure, Deploy, MaintainDevelop pre-built adapter models and support custom
adapter modelsEnable higher efficiencies in adapter development
without changing existing development workflow and processesDefine new adapter engineering and development
methodologies which can be applied with generic software development tools
Adapter Runtime EnvironmentAdapter Runtime Environment (ARE) supports
distributed and centralized adapter deployments across multiple platformsSupport dynamic distribution of adapter
components across the platforms in the adapter runtime environment
Support centralized administration, monitoring, and debugging of adapters at runtime
Support dynamic language (locale) translation for multi-lingual runtime environments
Support dynamic component code download and side-by-side runtime environments (production and QA)
SummaryAdapters are complex pieces of software
performing critical integration tasksThe definition of adapters continues to expand
and support new application architectures (“SOA”)
Adapter tools are now available for enabling rapid adapter development and deployment
The trend is towards developing semantically and functionally rich adapters will continue to strengthen the role of adapters