Upload
syncsort
View
36
Download
1
Embed Size (px)
Citation preview
Syncsort Mainframe Customer Education Webinar
Syncsort Ironstream® Version 1.4 New Features For Enhanced z/OS Analytics – Part 2
4Q 2017
Today’s Presenters
Ed Wrazen Director, Mainframe Product Management is responsible for the product strategy & roadmap for Syncsort’s Mainframe products and solutions. With a career in Enterprise IT spanning 35 years, Ed has held roles in software development, database administration, product management, consulting and marketing in global businesses and enterprise technology companies. Ed has experience in Enterprise systems architectures, performance management, database and data management technologies and is a regular speaker at industry events worldwide.
2Syncsort Confidential and Proprietary - do not copy or distribute
Ed Hallock is a highly experienced Information Technology Professional with a broad experience base in software product development, support, product management, marketing, and business development. In his diverse career Ed has benefited from working for some of the largest independent software vendors, in a variety of roles, providing enterprise solutions to Global 1000 corporations. Ed has extensive experience in performance and availability management for systems and applications. He holds a bachelor’s degree in Computer Science from Montclair State University in Upper Montclair, New Jersey and has presented at numerous industry events as well as corporate related conferences and seminars.
Agenda
Introduction to Ironstream®
Recent New Features:– SMF Versioning
– Enclave support for RMF III data
– Multi-Send API
– Transaction Tracing
– Advanced Filtering for SMF data
– Data Loss Protection
Splunk: The Industry-Leading Platform For Machine Data
Syncsort Confidential and Proprietary - do not copy or distribute
Machine Data: Any Location, Type, Volume
Online
ServicesWeb
Services
ServersSecurity
GPS
Location
StorageDesktops
Networks
Packaged
Applications
Custom
AppsMessaging
TelecomsOnline
Shopping
Cart
Web
Clickstreams
Databases
Energy
Meters
Call Detail
Records
Smartphones
and Devices
RFID
On-
Premises
Private
Cloud
Public
Cloud
Platform Support (Apps / API / SDKs)
Enterprise Scalability
Universal Indexing
Answer Any Question
DeveloperPlatform
Report &analyze
Custom dashboards
Monitor & alert
Ad hoc search
Mainframe
4
Critical Mainframe Data Normalized and Streamed to Splunk with Ironstream®
Log4jFile
Load
SYSLOGSYSLOGD
logs
security
SMF
50+
types
RMF
Up to 50,000
values
DB2SYSOUT
Live/Stored
SPOOL Data
Alerts
Network
Components
Ironstream
API
Application Data
AssemblerC
COBOL
REXX
USS
Value of an End-to-End View, Inclusive of Mainframe
Extend What Splunk Does Already, to include critical z/OS systems:– 360ᵒ Degree View: Make the Splunk View of the Enterprise Complete via
Including Mainframe Data
– Same Splunk Dashboards, Bigger, More Complete Data Sets; Free Ironstream Splunk Apps and Modules
Security and Compliance/SIEM- Ensure Audits Passed
IT Operational Analytics/ITOA-Ensure Ops SLAs Met
IT Service Intelligence/ITSI-Ensure Services Health
Polling Question #1
What analytics platforms are you using today for z/OS IT operational intelligence:
Splunk
Hadoop
ELK (Elastic Stack)
Spark
Custom/Home Grown solution
None
7
8Syncsort Confidential and Proprietary - do not copy or distribute
SMF Versioning
New in 4Q 2017
z/OS SMF Changes
SMF record structures can and do change from one release of z/OS to another
Significant No. of SMF changes with z/OS 2.3 (avail Sep 29, 2017)– Up to 2048 record types supported
– Up to 65k subtypes per record type
– DSECT changes - Fields Added or Moved
– Extended Headers now available
Changes with subsystems– Different versions of CICS or Db2
– Different versions running on same LPAR
9Syncsort Confidential and Proprietary - do not copy or distribute
SMF Processing requirement for different z/OS levels
Technologies that process SMF data from z/OS need to be mindful of changes between releases
An SMF Processing Solution compiled on one z/OS may need to process SMF data from another z/OS
Ability to process SMF data on a z/OS level that did not generate the original data
Ability to detect subsystem version that generated SMF
– CICS 4.2, CICS 5.1, CICS 5.2 CICS 5.3, CICS 5.4
– Db2 v10, Db2 v11, Db2 v12
10Syncsort Confidential and Proprietary - do not copy or distribute
New Ironstream Support for SMF Versioning
Dynamically detects the z/OS level
Determines the structure of z/OS SMF records
Determines the subsystem SMF record structure
Supports multiple versions of z/OS SMF data in a single instance
Ironstream now includes a framework for supporting multiple versions of z/OS SMF data
• Positioned for future z/OS releases
• Correctly formats the SMF record for Splunk
• Multiple versions/formats supported
11Syncsort Confidential and Proprietary - do not copy or distribute
12Syncsort Confidential and Proprietary - do not copy or distribute
Enclave support for RMF III data
New in 4Q 2017
What are Enclave Attributes?
An enclave is a transaction that can span multiple dispatchable units (SRBs and tasks) in one or more address spaces and is reported on and managed as a unit.
The enclave is managed separately from the address spaces it runs in.
A program can create an enclave, schedule SRBs into it, or join tasks to it.
A multisystem work manager can process a transaction on multiple systems by using a multisystem enclave.
An enclave represents a “business unit of work” – Used by a variety of workloads – Db2, Db2 DDF, Websphere, MQ, LDAP, TCP/IP
– Enclave created and used for work across multiple address spaces and systems
13Syncsort Confidential and Proprietary - do not copy or distribute
Enclave support for RMF III data
Enclave is used to keep track of address space independent transactions
– Can consist of multiple TCBs or SRBs executing across multiple address spaces
Enclave used by Db2
– Distributed Data Facility (DDF)
– Db2 Stored Procedures
– Db2 Sysplex query parallelism
– Db2 sequential prefetch
Other users
– MQSeries
– Websphere
– TCP/IP
– LDAP
14Syncsort Confidential and Proprietary - do not copy or distribute
Displaying Enclave activity using RMF
Configuring Ironstream to collect Enclave Attributes
1. Activate IronstreamDesktop Browser
2. On Admin menu, select Ironstream instance
3. Select RMF Filters– Automatically built using RMF
DDS
4. Select Enclave attributes from MVS Image Attributes
16Syncsort Confidential and Proprietary - do not copy or distribute
Ironstream API EnhancementMulti-Send API
New in 4Q 2017!
Ironstream API Enhancements
The Ironstream API enables Ironstream instances to programmatically collect user-defined EBCDIC or ASCII data and forward it on to Splunk.
The API includes a configurable IRONSTREAM_API data source and a SSDFAPI routine, which is a standalone load module that can be called or link edited statically into the calling program
New capability allows an API instance to be INITiated and TERMinated
– Enables multiple records to be sent while the connection is active
17Syncsort Confidential and Proprietary - do not copy or distribute
Advantages of the Multi-Send API
Good for multiple, sequential send requests
– E.g. Processing file input and forwarding to Splunk
RACF check performed only on the initiation of the connection
Connection persists over multiple send requests
Eliminates the need for the API to allocate and release storage for each operation
Performance Improvement over Single-Send API
18Syncsort Confidential and Proprietary - do not copy or distribute
Using the Multi-Send API
19Syncsort Confidential and Proprietary - do not copy or distribute
CALL SSDFPAPI,(NUMPARM,REQUEST,CLASS,TYPE,SUBTYPE,TOKEN,RETCODE, RSNCODE)
|||
PROCESS DATA 1-NCALL SSDFPAPI,(NUMPARM, REQUEST, TOKEN, DATA, LENGTH, RETCODE, RSNCODE)
|END DATA |
||
CALL SSDFPAPI,(NUMPARM,REQUEST,TOKEN,RETCODE, RSNCODE)
Requesting Program Ironstream
INIT Request
SEND Request…1
SEND Request…N
TERM Request
▪ Validate Request▪ Perform RACF Check▪ Build Control Block Chain▪ Send a valid Token to Requestor
▪ Validate request▪ Process record for each SEND
request
▪ Validate request▪ Release Storage
1. Request: Request type can be “INIT”, “SEND” or “TERM”
2. Class: Identifies the Ironstream instance running with the same class
3. Type: Identifies the Ironstreaminstance running with same class and type combination.
4. Subtype: Identifies the Ironstream instance running with Same class, type and subtype combination
5. Token: Used by API 6. Data: Data Address7. Length: Data Length
API Parameter Options
20Syncsort Confidential and Proprietary - do not copy or distribute
Transaction Tracing
Announced in 3Q 2017
What is Transaction Tracing
• Enables organizations to get deep insight into web-based and mobile transactions’ impact on the mainframe.
• Unprecedented granularity that enables you to monitor and improve application performance.
• Lightweight solution with a minimal footprint
Syncsort Confidential and Proprietary - do not copy or distribute
• Leverages Syncsort Ironstream® to
deliver IBM z/OS machine data in
real-time to leading platforms like
Splunk® for operational analytics.
How Does It Work?
• Uses a transaction identifier to correlate transaction workloads through CICS and Db2 on z/OS.
• Ties the transaction identifier to SMF 110 and 101 records generated by CICS and Db2.
• Provides the correlated SMF data to Splunk for a visualization of various performance attributes of the units of work.
• Time spent in the CICS and Db2 sub-systems along with resources consumed to support the transactions is clearly reported.
Syncsort Confidential and Proprietary - do not copy or distribute
23Syncsort Confidential and Proprietary - do not copy or distribute
Advanced Filtering for SMF Data
Announced in 2Q 2017
Why Filter SMF Data?
SMF volumes can be enormous – large CICS and Db2 installations can generate TBs of data daily
Transferring data that is not useful puts a strain on network and other system resources
Need to provide control over volume of SMF data processed and forwarded by Ironstream to Splunk
Need to eliminate data clutter by forwarding only those fields that are truly needed
24Syncsort Confidential and Proprietary - do not copy or distribute
SMF Filtering and WHERE Processing
Ability to select only desired fields within individual SMF records– INCLUDE statement in configuration file or via field selection in the Ironstream
Desktop GUI
New extension enables selection of fields based upon the value of field
– WHERE clause in configuration file
25Syncsort Confidential and Proprietary - do not copy or distribute
Basic WHERE Syntax
"SELECT":"SMFnnn"
"INCLUDE":"field_1,field_2,...,field_n" Optional Statement– If omitted, INCLUDE defaults to ALL
"WHERE":"search_condition AND/OR search_condition“– Any number of search conditions can be specified
– If multiple search conditions are given, each must be separated by a logical AND or OR operator
– Search_condition: Field_1 operator operand• Field_1 must be the name of a field from the SMF record
• The operator can be: EQ, NE, LT, LE, GE, GT
• Operands can be another field name, character strings, decimal values, hex values, date, time
• Wildcards supported for character strings
26Syncsort Confidential and Proprietary - do not copy or distribute
27Syncsort Confidential and Proprietary - do not copy or distribute
Data Loss Protection (DLP)
Announced in 2Q 2017
Why is DLP Needed?
To prevent loss of data forwarded by Ironstream to Splunk
– Early implementations of Splunk did not include any mechanism for ensuring that data forwarded by Ironstream was both received and successfully indexed by the Splunk platform
• If Splunk encountered an error prior to indexing the data it received from Ironstream, that data was lost even though Ironstream had successfully forwarded it
– Network failures preventing Ironstream from forwarding data for a long enough period would cause the in-storage data buffers to overflow resulting in data loss
28Syncsort Confidential and Proprietary - do not copy or distribute
New Feature: Data Loss Protection (DLP)
Minimizes data loss during times of network or other external failures.
Uses IBM’s Coupling Facility’s System Logger functions, and Splunk’sIndexer Acknowledgement feature.
– Splunk indexer acknowledgement feature allows Ironstream to detect when data it has forwarded has been successfully received and indexed by the Splunk platform.
29Syncsort Confidential and Proprietary - do not copy or distribute
New Feature: Data Loss Protection (DLP)
Minimizes data loss during times of network or other external failures.
Uses IBM’s Coupling Facility’s System Logger functions, and Splunk’sIndexer Acknowledgement feature.
– Splunk indexer acknowledgement feature allows Ironstream to detect when data it has forwarded has been successfully received and indexed by the Splunk platform.
Optional feature that must be enabled….– Must define and configure a log stream within a coupling facility and make
Ironstream configuration parameter changes.
– No modifications required to existing Ironstream configuration files for those customers not requiring DLP.
– More information is available in the Ironstream Configuration and Users Guide.
30Syncsort Confidential and Proprietary - do not copy or distribute
Coming Soon! – IMS Log Data
31Syncsort Confidential and Proprietary - do not copy or distribute
Mainframe
TCP/IPSSL or non-SSL
Data Forwarder DCE IDT
Ironstream DesktopData Collection Extension
Data ForwarderData Forwarder
z/OS
Enterprise
Security
IT Service
Intelligence SPLUNK
DB2SYSOUT
Live/Stored
SPOOL Data
Alerts
Network
Components
Ironstream API
Application Data
Assembler
C
COBOL
REXX
USSLog4jFile
Load
SYSLOGSYSLOGD
logs
security
SMF
65+
types
RMF
Up to 50,000
values
IMS
Polling Question #2
What analytics platforms are you considering or evaluating to use for z/OS IT operational intelligence within the next 12 months:
Splunk
Hadoop
ELK (Elastic Stack)
Spark
Custom/Home Grown solution
Other
32
Summary: Value Today for Enterprises with a z/OS Mainframe
Syncsort Confidential and Proprietary - do not copy or distribute
Less ComplexityCollect mainframe data; correlate with data from other platforms; no mainframe expertise required
Clearer Security InformationIdentify unauthorized mainframe access, other security risks; prepares and visualizes key data for compliance audits
Healthier IT OperationsReal-time alerts identify problems in all key environments View latency, transactions per second, exceptions, etc.
Effective Problem-Resolution ManagementReal-time views to identify real or potential failures earlier; view related 'surrounding' information to support triage repair or prevention
Higher Operational EfficiencyEnhanced event correlation across systems; Staff resolves problems faster; “do more with less”
Eliminate Your Mainframe “Blind-Spot”Splunk + Ironstream = Your 360ᵒ Enterprise View
Industry Leader in Mainframe Software Products
What Now?
35
Get Ironstream® for SYSLOG for free
VISIT: HTTP://WWW.SYNCSORT.COM/EN/PRODUCTS/MAINFRAME/IRONSTREAM
CONTACT: [email protected]
http://www.syncsort.com/en/TestDrive/Ironstream-Starter-Edition
Thank You.Questions?