ESB Pack for Microsoft BizTalk Server October 13, 2017 ESB … · ESB Pack for Microsoft BizTalk Server October 13, 2017 Why use ESB Feature Pack? This feature pack provides the following

ESB Pack for Microsoft BizTalk Server October 13, 2017

ESB PACK FOR BIZTALK SERVER

Overview ESB Feature Pack complements BizTalk Server with new features that make it even more powerful and easier to use as Enterprise Service Bus. It extends BizTalk with new features, runtime capabilities, itinerary designer extensions and performance monitoring. In addition, it allows to replace analytic tracing capability introduced in Microsoft AppFabric for Windows Server for monitoring any applications built with Microsoft WCF and WF.

Goals Enterprise customers who currently use BizTalk as enterprise service bus can protect existing investments in Microsoft BizTalk Server by using ESB Feature Pack to reduce total cost of ownership of existing integration solutions. This pack offers more choices to BizTalk customers in

Infragravity �1

Metrics from ESB Pack


financial industry who cannot yet use cloud due government regulations or policies established by various standards.

In addition to new runtime capabilities to enable powerful messaging flows, ESB Pack significantly decreases cost of itinerary development, testing and BizTalk operations. This feature pack is integrated with modern open source monitoring platforms that can be used on-premise or in the cloud of your choice.

Customer Challenges The ESB Feature Pack addresses common and recurring customer challenges with BizTalk Server to make development, testing and operations easier. Most enterprise environments are complex and consist of many different operating systems, applications and platforms along with BizTalk. Thus, IT teams are looking to use same set of products and tools for system reliability engineers to combine these metrics along with BizTalk and applications built using .NET platform. In addition, Gartner predicts that more enterprise customers will be adopting open source software to enable additional business functions, which also may be part of integrations built with BizTalk Server.

The ESB Pack addresses the following challenges:

• Enabling “lift and shift” approach - for .NET applications built with WCF and WF use same monitoring platform and tools in hybrid enterprise, including Microsoft Azure, Amazon AWS, IBM BlueMix as well as datacenter.

• Monitoring complexity - for .NET services and applications built using WCF and WF. This capability was introduced in the past in Microsoft AppFabric for Windows Server.

• Limited operational insight - it is difficult to build integration solutions without instrumentation. For example, no telemetry for failed messages in BizTalk Server is emitted from exception management framework or itinerary-based routing.

• Anomaly Detection for BizTalk and ESB Toolkit - need for comprehensive telemetry and choices to store analyze operations insights from multiple products or platforms (Windows and Unix).

• Limitations of using ESB itineraries - including high level of complexity, inability to instrument, monitor and operate integration solutions based on runtime metrics.

• Hight level of complexity - for deploying BizTalk and versioning BizTalk Server artifacts requires downtime.

Infragravity �2

“The organizations in that are most advanced in their IT approach are most prone to embrace SLAs, but still, fewer than half (46%) do it.

KASEVA REPORT


Why use ESB Feature Pack?

This feature pack provides the following benefits:

• Data Privacy - store metrics and events in time series or relational database on-premise or in the cloud of your choice.

• Reduced Cost of Ownership - protect existing investment in BizTalk Server as enterprise service bus by adding new features and capabilities to reduce TCO and improve agility. Our internal data indicates that time needed for development, testing and troubleshooting can be significantly reduced: in some cases to minutes instead of days.

• Streamlined Operations - measure quality of integration solutions by collecting enterprise service bus metrics at runtime. It allows seamlessly replace AppFabric for Windows Server monitoring feature to collect runtime metrics for .NET services built on Microsoft platform using Windows Communication Foundation and/or Windows Workflow Foundation.

• Effective DevOps - increase development, test and operations productivity within enterprise or "IT as a Service" company. Analyze service level agreements(SLA), performance and health metrics from different sources with capabilities to create alerts, detect anomalies and prevent downtime.

• Faster Time to Market - increase capabilities of solution delivery by reducing complexity and cost of ALM with even more extension points for BizTalk enterprise service bus and monitoring.

• Integrated with open source monitoring platforms - leverage integrations with visualization tools and event stores that can be deployed on-premise or in the cloud of your choice. Using time series databases allows IoT approach for collecting telemetry from multiple operating systems and platforms. This allows using modern monitoring platform for processing and data visualization for metrics from hybrid cloud and/or heterogeneous environments on-premise. The collected telemetry makes it possible to enable anomaly detection using tools widely used by industry today.

What is included in BizTalk ESB Extension Pack? This feature pack is comprehensive update for ESB capabilities available in BizTalk Server today. It provides essential functions for development, testing and operations to significantly reduce time, cost and increase agility:

• Runtime enhancements - modernized itinerary runtime and services enable complex messaging patterns, branching, execution, composition and exception handling.

• Design improvements - developers can use new resolvers, filters, messaging services and designer extensions for building complex itinerary flows.

Infragravity �3


• Telemetry collection - allows measure performance for itineraries, on-ramps, off-ramps and exception management events without dependency on BAM. Similar to monitoring BizTalk and ESB itinerary metrics, the monitoring agent included in this feature pack replaces telemetry collection from Microsoft AppFabric for Windows Server. These metrics include telemetry from WCF, WF and the feature pack itself.

• Integration with multiple monitoring platforms - events can be stored in time series or relational databases. Performance engineers can easily leverage open source software to create dashboards for metric visualization, rules for anomaly detection and alerts using open source software.

• Simplified installation and configuration - includes install scripts, utilities and framework to automate registration of ESB components.

Features and Capabilities The below table shows how ESB pack complements features available in BizTalk Server:

The purpose of WMI metrics collection is to enable monitoring for application specific metrics that system reliability engineers find critical for running specific business workloads. These metrics can be configured without any coding and stored in time series database to enable anomaly detection

Description BizTalk Server ESB Toolkit ESB Pack

Mixed messaging patterns (on-ramps, off-ramps) N/A No Yes

Improved tracking for failed messages. N/A No Yes

Telemetry integration with time series databases. N/A N/A Yes

Advanced itinerary flows, sub-flows, continuations N/A No Yes

Itinerary variables N/A No Yes

Conditional flow branching using regular expressions. N/A No Yes

Monitoring SLA(on-ramps, off-ramps,flows, errors). N/A No Yes

User-defined telemetry events. N/A No Yes

Improved caching with multiple back-end stores. N/A No Yes

ESB component registration framework. N/A No Yes

Improved itinerary deployment. N/A No Yes

BizTalk metric collection (WMI) integrated with TSDBs N/A No Yes

SQL Server metric collection (WMI) N/A No Yes

WCF, WF analytics tracing N/A N/A Yes

BizTalk analytic events integration with TSDBs N/A N/A Yes

Infragravity �4


and alerting. The WMI metric collection in this pack does not replace Microsoft System Center for monitoring broad set of metrics for operations, security and compliance.

Operational Insights The monitoring features included in this pack allow reliability engineers to configure and monitor metrics they see as business critical for operational insights. to measure liveness and safety. Unlike other products related to BizTalk, ESB Feature pack uses fundamentally different approach for metric collections and data visualization. Instead of implementing monitoring only for BizTalk Server, it was designed to collect wide metrics and integrate with modern monitoring platform widely used in industry today in data center and any cloud. This enables getting operational insights for BizTalk, ESB, WCF, WF with proven approach used by industry today for monitoring complex cloud infrastructures today with Docker and/or Kubernetes in multiple clouds. The “integrated vs. build” approach is identical to other companies like IBM have done for the improving their core products to adopt modern DevOps using open source software and tools that are actively developed and easily accessible.

BizTalk Analytic Events This extension pack introduces new capability for collecting analytic events from BizTalk Server and storing them in time series database on-premise or in the cloud. The below picture shows analytic events collected by this pack stored in time series time series database:

This feature is only supported for BizTalk Server 2016 with Feature Pack 1 installed.

Infragravity �5

Analytic events from BizTalk Server collected by ESB Pack, stored in InfluxDb time series database, shown in Grafana dashboard.


Analytic Tracing Developers often face challenges when implementing common SOA and integration patterns with ESB Toolkit. Typically, when itinerary flows become more complex, they are more difficult to track. To address this challenge, this pack emits and collects telemetry for all ESB scenarios to measure performance of new features and capabilities without need to use BAM. The collected telemetry increases operational capabilities to detect when integration solution SLAs exceed defined limits and therefore can lead to performance bottlenecks. The collection is performed by agent (codename “Hydrofone”) deployed as service on Windows host. The levels of telemetry can be adjusted to select events that can be emitted and/or collected at runtime. This feature also solves critical challenge when business uses BizTalk as “shared infrastructure” when every change in application can negatively affect performance or function of existing apps deployed on the farm. The telemetry data also includes metrics for failed messages with and without itineraries. New telemetry collection capabilities included in ESB Pack allow to reduce time and cost of operations in the following cases:

• Increased data privacy by routing analytic events from BizTalk 2016 Feature Pack 1 using integration with time series databases.

• Collection of telemetry events for failed messages, itineraries, IIS and web services built using WCF and WF.

The below example of telemetry for BizTalk ESB scenarios to measure SLA for itineraries:

Infragravity �6

ESB events from BizTalk Server collected by ESB Pack, stored in InfluxDb time series database, shown in Grafana dashboard.


WMI Metrics In addition to collecting metrics from analytic tracing, this pack also supports WMI metric collection from Windows hosts or containers. This feature allows system reliability engineers collect specific metrics specific to the business without any coding by simply specifying queries and schedules for executing them on periodic basis:

BizTalk ESB Enhancements In addition to improvements for operation insight, this pack introduces additional capabilities for building BizTalk integration solutions and emits telemetry for instrumenting them.

Sub-flows and continuations In the past, developers asked for ability to compose itineraries together in designer and reusing steps across them. To address this requirement, this feature pack introduces new itinerary concepts: variables, sub-flows and continuations. In addition, it uses active monitoring and emits telemetry that can be used for test automation, user-defined events, troubleshooting and monitoring in production without dependency on BAM. Using sub-flows allows referencing steps in one itinerary from another that have to take place as part of main flow at runtime. In addition new itineraries can be initiated from orchestrations with different tracking options.

In terms of itinerary, continuation specifies whether or not main flow should resume after sub-flow is completed. In other words, after execution of sub-flow, steps in a main flow could be ignored or triggered.

Infragravity �7

WMI from BizTalk Server collected by ESB Pack, stored in InfluxDb time series database, shown in Grafana dashboard.


Exception Management To complement BizTalk ESB Toolkit, this feature pack provides runtime and design support for changing itinerary flows upon exception occurred in messaging or orchestration service. This mechanism does not require code change for messaging services and provides convenient way to do so in custom orchestrations.

Data Visualization This feature allows to visualize data from SQL Server database using Grafana software. It enables complex data visualization and drill down for exploring data in BizTalk ESB exception management database, BAM or any other database records.

Advanced Caching ESB Feature Pack provides alternative component to enable more robust caching for itinerary-based routing, including support for durable cache store. This allows implementing State Repository pattern for integration scenarios when needed.

Direct On-Ramps This feature provides orchestration and messaging services ability to spawn new itineraries with optional continuation of another known instance without the need to use on-ramps. While it is not universal solution to address all integration challenges, the benefit of this feature is ability to dynamically choose flows as itinerary is being processed, which is now truly compliant to the definition of the Recipient List pattern.

Extensibility ESB Pack includes several improvements for existing runtime and extensibility. It is much easier to create different types of broker services using messaging, orchestrations and new filtering. Runtime capabilities were improved with support of new message exchange capabilities for enabling SOA and EAI patterns. These runtime capabilities are completely additive and can be leveraged using new extensions for itinerary designer included in this pack.

Summary The ESB Pack for offers many features to modernize BizTalk Server with new integration capabilities and operational insights. Following the Microsoft trend for SQL Server 2017, which works on Windows and Unix platforms, this feature pack offers telemetry collection using modern monitoring platforms. For example, InfluxDb time series database and Grafana visualization software can be deployed as containers in five minutes or less with Docker or Kubernetes. Each of these deployments can be used to store and analyze telemetry from integration solutions and partitioned per business process or function to allow multiple IT teams effectively monitor shared BizTalk environment. The functional enhancements of ESB scenarios allow save time, cost and leverage new operational insight features to measure service level agreements. The ESB Pack allows customers to reduce cost, increase agility and unify operational insights in hybrid enterprise today.

Infragravity �8

Documents

ESB Pack for Microsoft BizTalk Server October 13, 2017 ESB … · ESB Pack for Microsoft BizTalk Server October 13, 2017 Why use ESB Feature Pack? This feature pack provides the following