The Benefit of Leveraging System z with IBM Cognos 8with IBM Cognos 8

AgendaAgenda

Sh t i t d ti N i• Short introduction Numius• Cognos in Information Management from the

partner perspective• A short functional demonstration of Cognos on g

System z• Results of a client PoC for Cognos on System zResults of a client PoC for Cognos on System z

in a joint effort between IBM SG Bel, IBM Montpellier and NumiusMontpellier and Numius

Numius team

Leading provider of Performance Management Solutions and Services in Benelux.

30 hi hl kill d d f i l lt t30+ highly skilled and professional consultants.

Full range services, from Vision Creation, Business Analysis to ImplementationBusiness Analysis to Implementation, Architecture, Operations and Outsourcing.

Multi-industry, multi-function experience.

Focus on long-term partnerships.

Trusted advisor roleTrusted advisor role.

Numius activitiesNumius activities

© Numius nv© Numius nv

Numius-IBM relationshipp

• 2009 Most Distinguished Achievement Award Southwest IOT• 2009 Belux Partner of the Year• 2008 Only Platinum Partner in BeLuxy• 2007 Platinum Reseller• 2007 Partner of the Year• 2007 Reseller of the Year• 2006 Partner of the Year• 2005 Enterprise Planning Partner Award• 2004 Business Intelligence Partner Award2004 Business Intelligence Partner Award

IBM Advanced Business Partner

Cognos in Information Managementg g

The Performance Management Wheel –Performance Management is Every Employee’s Concern

How are we doing? Why?What should we be doing?

Warehouses (Relational & OLAP)

Transaction Systems Flat, Legacy or Modern

ExternalSources

The Cognos 8 Performance Management System –Th Pl tf I f t t D t il d A hit tThe Platform Infrastructure Detailed ArchitectureUser Experience Universal CapabilitiesAccess and Interactivity Business Modeling

ContentIT Tools Shared Set of Purpose-Built Services

Common Business Model

Model

Admin

Upgrade

Shared Dimensions

Personal Datasets

Presentation Service

EventService

Security Service

Cognos 8 Bus – Dispatcher SOAP, XML

SchedulingService

Cognos 8 A hit t

Content

Security

Services API

Plan & Report

QueryService

MovementService

Calculation Service

Streaming Service

Attachments, AnnotationsArchitectureInitiatives,

MetricsSystem ContentMetadata, Events

Optimized OLAP64-bit in-Memory

MetadataPlan & Report Saved Objects

Open Data Access

Metadata, Events Search Index, Audit Logs, System Metrics

64 bit in MemoryHigh PerformanceStreaming Cache

Exploit existing infrastructure choices Integrated & optimized with IBM infrastructureI f t t

Modern and Legacy Sources

Application Sources

OLAP Sources

Access to All Data

Message Sources

Relational Sources

p g g p

Data Integration & Data Quality Tools

Platforms & Databases

Security Providers & Firewalls

Application & Web Servers

Infrastructure Fit

The Cognos 8 Platform easily Integrates with Existing Environments

Any J2EE application serverApache Tomcat (incl ) IBM webSphere

Environments (Simultaneously available)

App ServersSOAP, XML, WSDL, HTTP(S), OMG

Interoperable and Extensible

Standards Apache Tomcat (incl.), IBM webSphere, SAP NetWeaver, BEA WebLogic, Oracle Application Server 10G

Any combination of Windows, Unix or LINUX (RedHat)

Operating Systems

App ServersSOAP, XML, WSDL, HTTP(S), OMG CWM, XML/XMI, WSRP,RESTStandards

ExtendOpen, Web Services API to programming language of choice including Java, Microsoft .NET Framework, COM & Perl

Industry Standard Portals including IBM, y

Routers/Firewalls All standard gateways supported

y g ,SAP, Plumtree, WRSP)

Enterprise Search, BPM, Office, 3rd party Applications

Embed

Any combination of LDAP, NTLM, Active Directory, Netegrity, SAP, Cognos 7, customSecurity

Intel, AIX, HP/UX, SolarisPlatforms

Firewalls

Metadata Integration Open Data Access

y, g y, , g ,

Cognos 8 Platform

CA ERwin, Oracle Designer, Business Objects Designer, Sybase PowerDesigner, CA COOL, Rational Rose, XML/XMI, OMG CWM

Modeling tools

Relational

XML, WSDL, LDAP, JDBC, JDBC…

IBM DB2 UDB, Oracle, Microsoft SQL Server, Teradata, other…

Modern

Source catalogs Relational databases, ERP

PowerCubes: High Performance Dimensional CacheI d t St d d OLAP P id

SAP, PeopleSoft, Oracle, Siebel, salesforce…ERP

OLAP

Cognos Data Manager, Informatica PowerCenter, IBM Information Server (Ascential) Metabroker, Composite, IBM II

ETL & EII

gIndustry Standard OLAP Providers: Microsoft, IBM, SAP, Essbase

Short Demonstration of just a little piece f Cof Cognos on z

Results of PoC of Cognos on System z

Beyond demonstrations...

A joint effort of IBM Software Group Belgium, IBM Montpellier and NumiusIBM Montpellier and Numius

Description Objectives and Technical architectureDescription, Objectives and Technical architecture

Mission statement of the joint IBM-Numius project

T f ll t i ti b i i t lli• To successfully port an existing business intelligence environment from an Intel-HP / Windows – HPUX –Oracle 10g – MS SQL server 2000 architecture to anOracle 10g MS SQL server 2000 architecture to an IBM Cognos 8.3 on System z – IBM DB2 9 for z/OS architecture.

• To prove that the end-user functionality is maintained.• To perform tests to determine the performance p p

boundaries of this real-life application on a simple System z hosted architecture.

• To learn what are real-life best practices when executing a “port” of this type.

Client contextClient context

• Numius’s client provides administrative and financial• Numius s client provides administrative and financial (data) services to about 10.000 Belgian companies, mainly in the retail sector.y

• Numius’s client uses IBM Cognos 8 for internal data l i d f lanalysis, process management and for external

communication with its 10.000 B2B clients and with about 200 B2B business partners.about 200 B2B business partners.

Logical architecture at Numius clientLogical architecture at Numius client

Input of data Intermediate OperationalInput of data

Application in Oracle FORMS

Intermediate datasource

Operational datasource

Application in Oracle FORMS

Client on internet Once per night

File based interface(client generates the neccesary files)

Logical architecture continuedLogical architecture - continued

Operational datasource

Data warehouse Powercubes Users

OLAP

ETL

Regeneration of cubes

Internal users

Incremental process, every night External users

RDBMS

OLAP RDBMS

The illusion: big bang for

no bucks

uctiv

itysi

ng p

rod

Incr

eas

Cost level

© Numius nv

The actual client (distributed) Cognos architecture

© Numius nv

Typical Distributed situationTypical Distributed situation

Client challengesClient challenges

• Numius’s client faces strongly degrading performance• Numius s client faces strongly degrading performance, both for database queries as well as for OLAP processing.g

• Numius’s client wants to outsource entire computer infrastructure to a central computing centre.N i ’ li hi i f l b• Numius’s client wants to achieve economies of scale by simplifying the heteregenous distributed computing architecture.architecture.

• Numius’s client wants to use its business intelligence tools as an individualised communication channel with

(its stakeholders (clients, business partners, shareholders,...)

Th ill i The vision:The illusion: big bang for small bucks

The vision: big bang for

correct bucksuc

tivity

The start: asing

pro

d

The start: a small fizz for a few

bucksIncr

eas

Cost level

The distributed environment the client should have

The distributed high availability equivalentThe distributed high availability equivalent

The vision:The illusion: big bang for small bucks

The vision: big bang for

correct bucksuc

tivity

bucks

Th t t Theng p

rodu

The start: a small fizz for a few

bucks

The nightmare: big bucks

but no fi k

Incr

easi

bucks fireworks

Cost level

The PoC z technical architectureThe PoC z technical architecture

Logical infrastructure of the PoC

Client Machine

W b BWeb Browser

HTTPHTTP

Application Server Machine

Application ServerppJVM

Cognos 8 BI Services

DataSQLSQL

RDBMS Server

Sourceson z/OS Content

Store

Test 1: Determine top capacityTest 1: Determine top capacityity

Marginal productivity > 0System “underutilized”

Marginal productivity near 0System at top capacity

Marginal productivity < 0System “overutilized”

Pro

duct

ivi

P

Increasing load

Test 2: Determine end-user comfort zone

mes End-user comfort zone

pons

e tim

ppl. Controller

Injector 1

Injector 2

System Z

user

resp

in w

eb a

p

Injector 3

d-to

-end

pe

rienc

e

Increasing numberf i t ti

End

exp

of interactive users

Performance test structurePerformance test structure

Lane 1 of 1 x 4 reports

Lane 2 of 1 x 4 reports

RDBMS OLAP RDBMS OLAP

p

Lane 3 of 1 x 4 reports

...Lane 100 of 1 x 4 reports

T0 T+2 T+4 T+6 T+8 T+10 T+12T+14 T+n

Explanation of Performance test dynamics• Performance tests environment• Performance tests environment

– During all the tests, the same architecture on System z was used.

– The following parameters where changed to create different test scenarios:

• Number of parallel 1 x 4 lanes started at a single moment in time• Number of parallel 1 x 4 lanes started at a single moment in time.– 1 x 1 x 4– 10 x 1 x 4

21 1 4– 21 x 1 x 4– 51 x 1 x 4– 100 x 1 x 4

Wi h i h /OS WLM Di i S i G l• With or without z/OS WLM Discretionary Service Goal• With or without DB2 9 for z/OS parallelism

Delicate team play between architectural layers

Linux for System z processing power

IBM Cognos layer

IBM Cognos 8.3 Number of Processes tuning for Batch Report Service

IBM DB2 z/OS layer DB2 9 for z/OS processing powery

Phase 1: N x 4 reports are submittedPhase 1: N x 4 reports are submitted

IBM Cognos layer

IBM DB2 z/OS layery

Phase 2A: Simple reports/queries are quickly finished

IBM Cognos layer

IBM DB2 z/OS layery

Phase 2B: Complex reports/queries take longer to process “wait time” in output

IBM Cognos layer

IBM DB2 z/OS layery

Phase 3: Uniform distribution as easy and complex reports are processed randomly

IBM Cognos layer

IBM DB2 z/OS layery

IBM Cognos 8 Report flow for 100 x 1 x 4 test without z/OS WLM Discretionary Service Goal

• The IBM Cognos 8 batch report• The IBM Cognos 8 batch report service is tuned to allow 6 parallel processes, generating a constant processing flowconstant processing flow combined with a steadily decreasing queue of reports that

iti f i l tare waiting for a processing slot to become available (in the IBM Cognos 8 layer).

IBM Cognos

Linux for System z processing power

IBM Cognos layer

IBM DB2 Z/os layer

IBM Cognos 8.3 Number of Processes tuning for Batch Report Service

DB2 9 for z/OS processing power

Linux for System z CPU for 100 x 1 x 4 test /o /OS WLM Discretionar Ser ice Goalw/o z/OS WLM Discretionary Service Goal

• At the Linux for System z level, we see peaks of processing and peaks of I/O

• These peaks are caused by the large PDF reports y g pthat need to be generated.

WLM Workload• WLM = Workload Manager

SQL activities for 100 x 1 x 4 test w/o z/OS WLM Discretionary Service Goal

• At the DB2 9 for z/OS

L100RP01# SQL Activities

1000

1200

level, we also notice an increase in corresponding SQL requests.

400

600

800

SQL

0

200

3:28:00

3:30:00

3:32:00

3:34:00

3:36:00

3:38:00

3:40:00

3:42:00

3:44:00

3:46:00

3:48:00

3:50:00

3:52:00

3:54:00

3:56:00

3:58:00

4:00:00

4:02:00

4:04:00

4:06:00

4:08:00

4:10:00

4:12:00

4:14:00

13:

13:

13:

13:

13:

13:

13:

13:

13:

13:

13:

13:

13:

13:

13:

13:

14:

14:

14:

14:

14:

14:

14:

14:

DB2 9 for z/OS CPU for 100 x 1 x 4 test comparison

L100PR01# CPU Utilization 46 mins

• 6 CPs = 100% blue line• 1 zIIP = 100% pink line• IIPCP yellow line 150 00%

200.00%

250.00%

300.00%

CPU-Busy%zIIP-Busy• IIPCP yellow line

e g first peak approx. 250% equates to additional 2.5 zIIP 0.00%

50.00%

100.00%

150.00%

00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

zIIP BusyIIPCP%

additional 2.5 zIIP capacity

• We notice a big difference in elapsed time

13:28

:0013

:30:00

13:32

:0013

:34:00

13:36

:0013

:38:00

13:40

:0013

:42:00

13:44

:0013

:46:00

13:48

:0013

:50:00

13:52

:0013

:54:00

13:56

:0013

:58:00

14:00

:0014

:02:00

14:04

:0014

:06:00

14:08

:0014

:10:00

14:12

:0014

:14:00

104 mins

in elapsed time• Total CPU time is

equivalent in both runs

Comparison of Linux for System z CPU with and without z/OS WLM Discretionary Serviceand without z/OS WLM Discretionary Service Goal

With t WithWithout With

• Higher processing power of DB2 9 for z/OS generates more intensive

• Lower processing power in DB2 9 for z/OS leads to longer elapse time in g

peak load on Linux for System z. • If the OLAP processing – which is entirely handled on the Linux for

g pquerry resolution. • It translate in a more uniform load distribution on Linux for System z, and y

System z – coincides with one of these peaks, then the OLAP processing time will go up.

y ,no impact on OLAP processing.

g g

Comparison of results with and without z/OS WLM Discretionary Service Goal

With t WithWithout With

M h h t t t l d ti• Much shorter total duration• Shorter “silence” in beginning of test

Comparison of query execution time with and without z/OS WLM Discretionary Servicewithout z/OS WLM Discretionary Service Goal

With t WithWithout With

• Significantly shorter query processing times.

CPU distribution over zIIP / IIPCP / CPCPU distribution over zIIP / IIPCP / CP

With t WithWithout With

Performance Test Phase 2 zIIP eligibility in %

100%

Performance Test Phase 1 zIIP eligibility in %

100%

40%

50%

60%

70%

80%

90%

100%

zIIP%zIIP-CP%CP%

50%

60%

70%

80%

90%

100%

zIIP%zIIP-CP%CP%

0%

10%

20%

30%

40% CP%

zIIP% 53.5% 19.6% 20.0% 20.0% 24.3% 20.1%

L01RP01 L10RP01 L21RP01 L51RP01 L51RP02* L100RP01 0%

10%

20%

30%

40% CP%

zIIP% 55 0% 53 8% 54 4% 76 2% 76 4% 54 4%

L01RP01 L10RP01 L21RP01 L21RP02* L51RP01* L100RP01

• zIIp-CP processing takes up a substantially larger proportion

• zIIPCP processing only for none discretionnary periode

zIIP-CP% 1.5% 35.4% 35.0% 35.0% 53.6% 34.9%

CP% 45.0% 45.0% 45.0% 45.0% 22.0% 45.0%

zIIP% 55.0% 53.8% 54.4% 76.2% 76.4% 54.4%

zIIP-CP% 0.0% 0.0% 0.0% 1.7% 1.5% 0.5%

CP% 45.0% 46.2% 45.6% 22.1% 22.1% 45.0%

substantially larger proportion. discretionnary periode

Teamplay of different layersTeamplay of different layers

• We notice how the production of complex reports on complex queries• We notice how the production of complex reports on complex queries requires coordinated teamplay between the different layers of the architecture.Q er resol tion is follo ed b PDF generation hich in t rn res lts in• Query resolution is followed by PDF generation which in turn results in report delivery to the end-user.

L100RP01# SQL Activities

600

800

1000

1200

SQL

0

200

400

13:28

:0013

:30:00

13:32

:0013

:34:00

13:36

:0013

:38:00

13:40

:0013

:42:00

13:44

:0013

:46:00

13:48

:0013

:50:00

13:52

:0013

:54:00

13:56

:0013

:58:00

14:00

:0014

:02:00

14:04

:0014

:06:00

14:08

:0014

:10:00

14:12

:0014

:14:00

Conclusions on performance testsConclusions on performance tests

• The platform behaves in a stable and predictable• The platform behaves in a stable and predictable manner.

• System behavior is driven strongly by the report types.System behavior is driven strongly by the report types. Architecture behaves in function of usage.

• A good architecture is “TeamPlay”. The teamplay must b d h f h l fbe geared to the usage type of the platform.

• All components must be tuned appropriately.A lti ti IBM C 8 hit t i l t• A multi-tier IBM Cognos 8 architecture remains relevant, but System z can significantly reduce TOC.

• A SINGLE 1x4 lane on the Numius customer systemA SINGLE 1x4 lane on the Numius customer system took approximately 45 minutes of processing time.

Results of Scaleability Test on a IBM DB2 9 for z/OS RDBMS datasource

Li f S t i h d h l l b i thi• Linux for System z is charged much more regularly because in this end-to-end test, interactive end-user behavior must be accomodated too by the Linux layer.

Results of Scaleability Test on an IBM DB2 9 for z/OS RDBMS datasource

• The test ran successfully for the full 130 virtual concurrent users that were included in the testthe test.

• As of 114 users we experience slower “surfing” behavior.• We learnt that this was caused by the fact that we artificially limited each virtual

user’s bandwith, thereby unnecessarily “occupying” processing power of the Linux for System z at larger user numbersSystem z at larger user numbers.

• In real life, this number of virtual users should be multiplied by 10 to 50 to achieve a number of potential “named” users.

• On the client’s system, this test could only be run for 8-12 users.

Conclusions from end-user perspective• Numius, as a representative to the client, can conclude , p ,

the following:

– The application was successfully and without loss ofThe application was successfully and without loss of functionality ported to the System z platform. This required no redevelopment.

– The client’s application would not require a redesign to pp q gaccomodate its growth in data volumes or in terms of users.

– Reports that are not practically useable at client’s site now become relevant again. Reports that did not run at client’s site

blnow are runable.– Client would be able to serve many multiples of current

number of users with the very simple architecture from this PoCPoC.

– Client could scale out to more complex architecture without increased hardware complexity.

The illusion:

The z: big bang for correctThe illusion:

big bang for small bucks

correct bucks

duct

ivity

The start: a The nightmare:si

ng p

rod

e sta t asmall fizz for a few

bucks

nightmare: big bucks

but no fireworks

Incr

eas

fireworks

C t l lCost level

Contact us atContact us at

J C tJo Coutuer+ 32 478 20 40 18 32 478 20 40 18Jo.coutuer@numius.eu

NumiusNumiusInterleuvenlaan 1515D3001 Heverlee