Business Agility through Self-Service Messaging Deploy Applications in Hours, not WeeksHHM-2931Leif DavidsenLee Gavin

Please Note:

2

• IBM’s statements regarding its plans, directions, and intent are subject to change or withdrawal without notice at IBM’s sole discretion.

• Information regarding potential future products is intended to outline our general product direction and it should not be relied on in making a purchasing decision.

• The information mentioned regarding potential future products is not a commitment, promise, or legal obligation to deliver any material, code or functionality. Information about potential future products may not be incorporated into any contract.

• The development, release, and timing of any future features or functionality described for our products remains at our sole discretion.

• Performance is based on measurements and projections using standard IBM benchmarks in a controlled environment. The actual throughput or performance that any user will experience will vary depending upon many factors, including considerations such as the amount of multiprogramming in the user’s job stream, the I/O configuration, the storage configuration, and the workload processed. Therefore, no assurance can be given that an individual user will achieve results similar to those stated here.

© 2015 IBM Corporation

Digital Enterprise

Reliability, security and scalability for Business Critical systems•Always on, always available•Security, control and governance

Speed and agility to drive innovation and growth•Explore, adopt, adapt•Rapid, Iterative prototypes

LoB roles CIO rolesApplication Developer

LoB DeveloperIntegration Architect

Administrator/ Developer

A New Era of Teamwork – The Vision

3

© 2015 IBM Corporation

Pain point : “New information, systems and services are springing up everywhere, and all need to be connected!”“Configuration, maintenance and operation of infrastructure take too long”

Pain point : “Deployments take months instead of hours”

Pain Points : “Our developers need to create engaging new apps fast, and make them interact with existing infrastructure”“I want to use the skills I have and not be forced to waste time learning stuff I won’t need”

Too often the reality….

4

Business Pressures to be overcome

• All parts of the business are increasingly connected– Common sets of shared services– Single customer repositories– Pressure to reuse rather than re-invent

• Faster pace of application development– Need to implement and deploy solutions fast– Tension between structural imperatives, and application development pressures

• Barriers between Line of Business and Infrastructure– Lines of Business own the development of new applications– Can get frustrated at the seemingly different priorities in infrastructure team

5

Pressures within infrastructure team

• Enterprise infrastructure provides various services to different Lines of Business– A diverse team

• Lots of responsibilities• Lots of different ‘customers’

• Providing services to Line of Business will require signing a Service Level Agreement between parties– Infrastructure team generally responsible for uptime, security, performance– Challenged by LoB on responsiveness, costs, efficiency, skills

6

Pressures on Lines of Business in a Hybrid world

• Meet the business priority to attack new business opportunities• Rapidly develop applications to capture those opportunities

– Priorities in order: Build, deploy, run– Decreasing order of expertise…

• May focus on developing and running in the cloud– Multiple reasons – but speed of access and responsiveness may be key

factor– Wish infrastructure and backend systems were run and administered like

cloud systems• Likely creating ‘shadow IT’ to manage their deployments

7

Pressures on application developers

• Build application fast to meet urgent priorities– Unlikely to want to pay much attention to corporate structures– What matters is what works

• Reuse code where possible– Stackoverflow, Github– Use familiar tools and processes to develop

• Applications likely to be self-contained for ease of rapid development– Must be able to link to the wider business – leverage core back-end systems

• Deploy quickly and be ready for change– Ensure application can scale to meet success

8

How to address the issues that create the pain points….

• Failures still happen - application and network – cloud and mobile especially

• Don’t try to handle these failures in your business application logic

• Buffer your applications, simplify your networks and maintenance, and use a common shared and standardized API

• Configurable: Persistence, Performance, Management, Security, etc.

Messaging is important in today’s infrastructure to avoid issues becoming painful…

“business transactions must happen only

once”“adding new services or

applications is unpredictable”

“change is relentless”

“need to adopt new industry standards”

“business insight is key in today’s

market”

“consumer interaction is forcing us to respond faster”

“we need to become more event driven”

“we need to recover from IT failures

better”

“our applications are getting too complex”

“must become more agile”

“losing data costs time money and reputation”

9

Understanding the lessons for messaging middleware

Businesses are dynamic – new applications, new sources of data, new consumers of data• The challenge of delivering data to meet changing

demands needs a flexible infrastructure• Roll-your-own code in the applications • Storing the data in a database or file • Build your own ‘Shadow IT’ to provide your own

infrastructure• Work to better use your existing infrastructure

Messaging infrastructure needs to be responsive• Keeps the application simple and able to adapt to change• Can provide high levels of security, availability, reliability• Can be seen as a barrier to rapid change

10

Built on Apache

Hybrid Messaging is here with Message Connect

IBM Message ConnectIBM MQ IBM Message

Hub

Connecting your enterprise MQ backbone to the Cloud

HOT off the press!IBM announces Message Hub for Bluemix Dedicated!

11

IBM Hybrid Messaging

12

App AccessPartner

Enterprise MQ Backbone

Bluemix

Message HubBased on Apache Kafka

MQ Light API

REST Kafka Message Connect

IBM Message Hub offers ‘cloud-native’ messaging and usability

MQ is being deployed in cloud environments with growing demand for cloud-style usability: “self-service”

Digital IT Enterprise IT

What does IBM MQ do?Provides messaging services to applications and Web services from mainframe to mobile that need to exchange data and events with:

Universally supported by multiple platforms 20 years leading in transactional message delivery

• Inherent reliable delivery and transaction control• Native, high-speed handling of any type of

message or file

• Single point of control, visibility, and management for all data movement

• Applications become more flexible and data movement becomes more reliable

• Extensive support through years of ecosystem extensions• Comprehensive single solution reduces complexity of

deployment and operation

Message

Q ManagerQ Manager

Application ZApplication A

Channels

PervasiveDevice

Sensore.g. RFID

Regional Office

MobilePhone

PetrolForecourt

Refinery

BranchOffice

Retail Store

Z Systems

Financial Services & Banking Manufacturing

GovernmentRetail

13

Scripted Deployment of MQ

UrbanCode Deploy IBM tool for automating deployments of applications and middleware

IBM MQ supported with a plug-in

Configure and interact directly with IBM MQ in multiple ways with out-of-the-box steps

ChefChef automates configuration and management

Allows infrastructure to be described in code

Scripts already available for deployment of MQ

Ensuring MQ continues to be easier and quicker to deploy

14

Using MQ self service/as a service today

• Many clients have built MQ as a service today– Orchestration and automation of MQ resources

• E.g. IBM Urbancode Deploy, Chef, Puppet, etc.

– Running on bare metal or in a cloud

• Redpaper now available• http://ibm.biz/mqaas_red

15

Appeal of the Cloud: Perception of MQ Today:

ease of deployment

low skill

scalable

cheap

vs

high skill

hard to change

application complexity

old, for legacy use onlyLine-of-business resistance to using MQ

MQ not chosen for new projects

MQ not taken into the cloud by customers

No new growth in MQ use

MQ as a service

MQ systems able to respond to LoB requirements quickly

MQ suitable for cloud deployments

MQ shows vibrancy

16

MQ as-a-Service

MQ as-a-Service allows many of the practices associated with a cloud deployment to be applied to a critical MQ infrastructure where the overriding requirement for stability would typically outweigh any gains this can bring.

Speed of application development– Speed up new application development and deployment by enabling end users to provision and manage their own

enterprise MQ resources whilst MQ administrators retain overall control and confidence in the resiliency of the MQ system.

Responding to change– Remove impact on applications and MQ teams when dynamically scaling up and down MQ architectures.

Reducing skills and touch– Greatly reduce the level of MQ skill required when designing, developing and deploying applications into a highly

available and dynamically scaled MQ architecture.

17

Advantages of MQaaS

• Increased agility• Improved usability• Improved efficiency• Improved consistency

19

Customer Scenario #1Large UK Bank

Large UK Bank

• Deployed IBM MQ over a period of 20 years• Acquired three smaller banks smaller banks, each with their own

– IBM MQ administrative practices– Installations– Business applications

• IBM MQ used for every critical business transaction in the bank • Digital and mobile banking driving

– creation of new customer facing applications – at faster rate than at any time in the history of the bank

22

Large UK Bank

• Challenge– Elastically scale and rapidly provision development and production IBM MQ

infrastructures for new or updated business applications– Requires each line of business to be able to use a self-service interface to

provision the necessary infrastructure changes themselves within minutes or hours, not days or weeks• Line of business professionals do not have the IBM MQ administrative skills

necessary to accomplish this requirement• Highly specialized central team of IBM MQ administrators must focus on high value

architectural work instead of low value provisioning activities

24

Deployed on IBM Pure Application System

• IBM Pure Application System – Manage virtual applications in a cloud computing environment– Access resources in your cloud

• Manage multiple environments from a single system and remote interface

• Installs and configures your IBM MQ software

• Manages the application runtime by using policies that you define

• Virtual system patterns – Enable efficient and repeatable deployments of systems that include one or more virtual machine instances

and the applications that run on them– Completely automate the deployment and eliminate the need to perform multiple time consuming manual

tasks– Script packages run

• When the pattern is deployed as a virtual system

• When the virtual system is deleted

• Whenever you choose to run the scripts manually

26

IBM MQ virtual system pattern

• IBM MQ Virtual System Pattern Type contains IBM MQ software

• Install IBM MQ on a system image as part of a virtual system deployment, managed by the Pure Application System pattern engine

• Offers a number of features including:– Ability to apply a new fixpack to deployed pattern

instances through the Pure Application System management interfaces

– Support for multiple queue managers on the same VM

– Queue managers are automatically stopped/started when appropriate by Pure Application System

– Separate file systems can be used for IBM MQ logs, data, and errors

28

MQ Patterns

• Solutions to general problems faced– By infrastructure and development teams during the development, test, and production

phase

• Benefits of using patterns – Can be automated– Predict what comes next – Solve problems that would be very tedious to solve otherwise

• Infrastructure team creates a set of IBM MQ related patterns– LOB requirements– Application needs– Services invoked by the development and application teams

30

Customer Scenario #2Large US Retail Outlet

Large US Retail Outlet

• Large retail outlet – Over 2000 stores located throughout the United States– Each store must connect to its headquarters to share information about stock levels, trades,

and other business transactions

• Configured IBM MQ as the standard mechanism to interact between stores and headquarters

• Requirement– Must be able to rapidly respond to changes in workload driven from the stores

• Establish connectivity between new stores and headquarters within a working day

(5 to 8 hours)• Limited support from IBM MQ administrators

– A self-service interface is required that store-based employees can use to achieve this goal

33

Large US Retail Outlet - Objectives

• Infrastructure team plans machines and resources required for deployment – Shared network requirements – Shared memory requirements

• Objectives – Reduce the overall costs– Reduce number of servers– Make the deployment easier to administer – Provide better manageability– Ensure quicker IBM MQ resources provisioning

35

Employed Message Hub Architecture

• Message hub: A group of queue managers that a set of applications connect to is called an MQ hub

• Important to have a centralized IBM MQ infrastructure that can be scaled, managed, and administered independently from the application

37

Large US Retail Outlet - Challenges

• Challenges the infrastructure team faced while provisioning MQ hubs– Mixture of platforms, hardware resources, and hardware and software

versions– Complex dependencies making impact analysis difficult– Migrations are difficult due to a lack of standardization– Application downtime impacts messaging and affects other applications

39

Deployed on MQ Appliance

• IBM MQ Version 8.0 image delivered as a state-of-art appliance – Built on latest IBM DataPower® Gateway Appliance hardware and OS – Pre-tuned configuration – Provides maximum performance– Infrastructure team does worry about configuration parameters across servers

and operating systems

41

Benefits of deploying on MQ Appliance

• Easy to deploy• Firmware only

– Removes management costs of maintaining currency and inter-dependencies of operating system and middleware

• Familiar IBM MQ administration interfaces• Supports existing IBM MQ definitions and security• Easier administration with separate roles and browser-based MQ Console Tool• High availability provided with the appliance • Lower data center costs, space usage, and power requirements• Downtime reduced by separating applications and middleware

43

MQ Appliance - MQaaS

• Helps in building MQ as a service– Similar to the standard IBM MQ

software installation– Similar administrator concepts

and syntax – System administrators

• Reuse same patterns or catalog of services

• Create repeatable deployment patterns– Minimum configuration

– Minimum performance tuning

45

Thank YouYour Feedback is Important!

Access the InterConnect 2016 Conference Attendee Portal to complete your session surveys from your

smartphone, laptop or conference kiosk.

MQaaSBackup

MQaaS interested parties

• Lines of Business (LOB)– Not concerned with the details of the implementation– Need to build new solutions for the business quickly

• exploit business opportunities

• counter competitive threats

– Looking for technology that can be provisioned quickly

• Developers and testers– Looking to IBM MQ to provide interfaces

• Service being developed

• Service they are relying on

– Not concerned about high availability, scalability, and reliability because these requirements are usually concerns of the infrastructure team

– Want a way to provision the queues and topics they need for their applications as and when they need them for development and test activities

51

MQaaS interested parties

• Infrastructure team– Represents the traditional IBM MQ teams– Know the product and are familiar with configuring and administrating it in the various environments

and topologies– Tasked with guarding the operational integrity of the core infrastructure– Processes and procedures ensure that the core infrastructure remains

• secure

• highly available

• high performing

• resilient to disasters

– Change control process may take as long as three months for provisioning– IBM MQ as a service can increase the speed at which the infrastructure team can deliver new IBM

MQ infrastructure to developers, while ensuring that the needs of the business are met by the solution.

52

Discover MQ resource usage intent

• Main question -What will your code use the destinations for?– Exposing a new endpoint to other applications/components

• Via a queue: plan to use queues to receive explicit requests to perform work and/or return data

• A subscription on a topic: plan to use publish/subscribe topics to trigger one or more actions when data is emitted

– Sending messaging to an existing endpoint • Synchronous request/reply: code blocks waiting for responses from the remote

system. • Asynchronous request/reply (preferred coding pattern for request/reply): code is

executed when the response arrives at any point in the future.• Fire-and-Forget: code does not expect any replies

54

Information to provide in addition to infrastructure

• Discover quality of service requirements• Global transactions (XA) – exactly once delivery• Duplicate checking

56

Discover message ordering requirements

• Messages affinities to each other - intentional or not • Consider the following options:

– Store for later processing– Process messages in any order– Require IBM MQ to deliver the messages in the order sent– Detect occasional out-of-order delivery and drive manual/automated compensation

logic

59

Publish / Subscribe subscription requirements

• Options for configuring publish/subscribe features • Some features might be provided as options to project teams, whereas some might be

mandated. Examples are as follows:– Uniqueness of topics - can be chosen enterprise-wide– Isolation of topics for scale - can be isolated by default– Isolation of topics for security - can be isolated by default– Subscription durability - needs input from applications– Dynamic verses pre-defined subscriptions - needs input from applications

62

Establish an application scaling model

• Two approaches to workload balancing and sending of messages– Route the messages to the applications– Configure each instance of the application to listen to multiple queue managers

• Sample question to establish the application scaling - How will your application scale?– There is a single vertically scaled instance of the application, with active/passive high

availability– The application will be horizontally scaled with two or more instances and

• Each instance is only capable of listening to a single queue manager and can not be re-configured or changed to listen to multiple queue managers

• Each instance can be configured to listen to two queue managers concurrently.

64

Establish availability requirements

• Multiple approaches to achieve message availability and service availability – All queue managers configured for high availability failover using the same technology – All queue managers for tier-1 applications configured for disaster recovery replication to a

secondary data

• Sample question to establish availability requirements - What is the maximum acceptable time for which the MQ service can be unavailable to send/receive new messages during planned maintenance or an unplanned infrastructure failure?

• Up to 5 minutes– Just a guide– Test the real world maximum failover time

• Less than 1 minute– Application can be engineered to exploit multiple active queue managers

66

Self service interfaces

• Web portal – Allow users to request and manage resources– RYO based on

• IBM UrbanCode Deploy• IBM Cloud Orchestrator

– Interfaces to allow portals to integrate with messaging system• MQ Script (MQSC)• Client MQSC• MQ Programmable Command Format (PCF)• MQ Administration Interface (MQAI)• MQ Explorer• MQ Console

68