Informix User Group France - 30/11/2010 - Nouveautés IDS 11.10 & 11.50

User Group Informix France

Fonctionnalités majeures IDS 11.10 & 11.50Fonctionnalités majeures IDS 11.10 & 11.50

Laurent Valenza

[email protected]

Mardi 30 novembre 2010Mardi 30 novembre 2010


OnBar Whole System Parallel Restore

Detailed Thread Wait Information

Create Index Automatically Creates Distributions & Statistics

Automatic real-time statistics

created for temp tables

Dynamically Rename Objects in ER

Multiple Trigger per table

Enhanced dbinfo()

Online Reconfiguration of the Physical log

Non-Block Checkpoints

Auto LRU Tuning

Auto AIO VP Tuning

Automatic Checkpointssysdbopen & sysdbclose

Private Memory Caches for Virtual Processors

SQL History Tracing

Last Committed Isolation

Database Scheduler

ANSI Join Directives

Database Restart Policy

SQL Admin API

Enhanced sysmaster

ontape Directory Backup

Encrypted

Communication

For HDR

Global configuration of

non-logging temp tables

Enhanced Explain


Développement

Scalabilité

Sécurité

Performance

Disponibilité � RESILIENCE

� OLTP DATA SERVER

� FAIBLE ADMINISTRATIONAdministration

� SECURITE DES DONNEES ET DES ACCES A LA BASE

� CAPACITE DE TRAITEMENT HAUTEMENT SCALABLE

� GESTION DES RESSOURCES DE LA BASE DE DONNEES

CheetahCheetah


Mise en place de la fonctionnalité HDR facilitée, uniquement en cas de besoin

HDR

OLTP

System A

N

System B

� Serveur totalement

indépendant

�Mise en place de la HDR facilitée

�Copie des bases sur les deux systèmes

�Aucun impact sur la production

�Synchronisation asynchrone des log buffers

�Chiffrement des données entre les noeuds HDR

�ER : Amélioration du parallèlisme

Network

Ne t w o r k

N

Network

Remote Standalone Secondary ServerRemote Standalone Secondary Server


Remote Standalone SecondaryRemote Standalone Secondary


Shared Disk SecondaryShared Disk Secondary


Shared Disk SecondaryShared Disk Secondary


� Sauvegarde et restauration sur des partitions du système, répertoires

– Modification des paramètres TAPEDEV, LTAPEDEV

– Ontape & Onbar

� Continuous Logical Restore (utile pour la création d’un système de backup)

– Alternative à la fonctionnalité HDR

� Amélioration des sauvegardes et restaurations parallèles (valable aussi aujourd’hui pour les “whole backup”)

� RTO_SERVER_RESTART, action sur le temps de redémarrage du moteur en cas de crash

Sauvegarde et RestaurationSauvegarde et Restauration


Développement

Scalabilité

Sécurité

Performance







CheetahCheetah


� AMELIORATION DE LA REPLICATION D’ENTREPRISE

– Exécution Parallélisé de l’application des modifications sur les enregistrements

– Amélioration remarquable des performances globales

� IMPROVED STATISTICS MAINTENANCE

– Mise à jour automatique des statistiques lors de créations d’index

� NON BLOCKING CHECKPOINT

– Nouvel algorithme de synchronisation, suppression du mécanisme de fuzzy-checkpoint

OLTP Data ServerOLTP Data Server


• Tremendous performance on low cost hardware

• Customers scale for high volume throughput

• Financial services: 100,000 messages/sec on 4-CPU Linux system

• Pharmacy system: 1B transactions per day on a 8-CPU machine

• Large retailer: Over 225 transactions/sec on a 32-CPU system

with space for growth

• IDS 11.5: Up to 50% faster than previous versions

“In our benchmarking process, Informix came out above all other leading solutions for its performance, scalability and reliability.” - Jon Tasker - Project Manager, Trafficmaster, UK

Build for Speed – less hardware requiredBuild for Speed – less hardware required


Développement

Scalabilité

Sécurité

Performance







CheetahCheetah


� Architecture multithread

– Faible consommation de ressources

– Architecture 32 & 64 Bits

– Thread Migration pour le load balancing

– Ajout “on the fly” de processeur virtuel

� Gestion des utilisateurs

– Allocation dynamique des ressources en fonction de la charge

� Ressources physiques

– Gestion des I/O

– Parallélisme

– Ajout automatique d’espace disque

ScalabilitéScalabilité


Développement

Scalabilité

Sécurité

Performance







CheetahCheetah


Encryption Expert

Common Criteria Certification

Label Based Access Control

Chiffrement au niveau réseau

SécuritéSécurité


� Chiffrement :

– Communication entre le client et le serveur, end to end

– Requêtes distribuées

– Enterprise Replication, sélection des serveurs à chiffrer, communication totalement sécurisée

– HDR, chiffrement totale ou partielle des journaux, chiffrement entre le primaire et le secondaire

� Contrôle :

– Chargement des datablades

– Création des bases de données et autres…

– Séparation des rôles

> DBSA : Database system Administrator> DBSSO : Database system security officer> AAO : Audit Analysis Officer> BAR : Backup and Restore Group

� Audit :

– On-Audit, ShowAudit géré par AAO et DBSSO



� Label Access Control (LBAC):

– Permet de définir qui peut accéder aux données mais aussi aux colonnes

– Gestion très précise des données

– Système flexible dans l’utilisation mais aussi grâce à la richesse de ses options

– Puissant et robuste dans l’application des règles de sécurité



� Common Criteria Certification

– Certification gouvernementale rigide :

• Identification, Authentification, Controle d’accès, Audit, Communications et la protection des données

• EAL 4 - Niveau maximum qu’un logiciel peut obtenir pour être utilisé dans les pays signataires en terme de sécurité

– Bénéfices :

• Niveau de sécurité du produit

• Compatibilité avec : SOX, HIPAA, Gramm-Leach-Bliley



Développement

Scalabilité

Sécurité

Performance







CheetahCheetah


25% report no DBA time at allAn Average of 7 minutes per user week

67%

11%

8%

14%

Less than 1 Day

1 to 2 Days

2 to 3 Days

3 Days or More

IDS Survey results : Average DBA Days/YearIDS Survey results : Average DBA Days/Year


� Nouvelle Database (sysadmin) :

–Fonction d’administration

–Gestion des alertes

–Collecte de données

� Améloriation de sysmaster et onstat :

–Checkpoint

–Environnement User/Server

–Online & onbar log files

–Information Réseau (IO counts and times)

–SQL Statement Historique

� Scheduler interne

–Execution de SQL

–Procédure stockée

–UDR

AdministrationAdministration


� Nouvelles UDRs pour l’administration du serveur Informix

� Thématique d’administration :

– Gestion de l’espace

– Gestion de la configuration

– Maintenance des routines

– Oncheck

� Avantages :

– Administration SQL

– Remote Administration

– Suivi de l’exécution des commandes dans des tables Informix

– Reporting personalisable

– Integration dans le système d’audit de l’entreprise

Fonction SQL AdministrationFonction SQL Administration


� L’API SQL Admin stocke les informations dans une table de la base sysadmin

Return messagelvarcharcmd_ret_msg

Return codeIntegercmd_ret_status

The command executed

varcharcmd_executed

Host the command was executed from

varcharcmd_hostname

User executing the command

varcharcmd_user

Time the command was started

datetime year to second

cmd_exec_time

serialcmd_number

DescriptionTypeColumn

Fonction SQL AdministrationFonction SQL Administration


Improvement to view I/O and lock wait informationsysrstcb

The OnBar Activity Log filesysbaract_log

Network information based on client typesysnetclienttype

Global Network Informationsysnetglobal

General information about the SQL tracingsyssqltrace_info

The list of iterators for the SQL statement.syssqltrace_itr

The sql statements which have been recently executed on the systemsyssqltrace

Displays the profile information about each SQL cachesyssqlcacheprof

Oracle compatibility featuresysdual

View the network I/O generated by database sessionsysnetworkio

Improvement to view the memory used by sessionsysscblst

View the online.log for the serversysonlinelog

View the servers environment variablessysenv

View Informix’s session environment variablessysenvses

Modified the existing table to add wait stats.systcblst

The information about the checkpoint and associated statisticssyscheckpoint

DESCRIPTIONTABLE NAME

Sysmaster : Nouvelles tablesSysmaster : Nouvelles tables


� Information de détail sur les requêtes SQL

� Visualisation de toutes les

couches

� Lecture simple, claire du

contenu de sysmaster et

onstat

� Configuration dynamique

� Granularité au niveau

utilisateur•Estimated # of rows•Estimated Cost•# of rows returned

•Statement Type

•Database Isolation Level

SQL statistics

•Buffer Reads & Writes•Page reads & Writes•Memory Sorts, disk Sorts,•Lock requests, waits

•Logical Log Records

•Index buffer reads

RSAM statistics

SQL statement text or the procedure stack trace with statement type

Statement Text

Date & Time this statement completedTime of Execution

Duration of the SQL statementStatement Execution time

Type of SQL statement being executedStatement Type

Current Database NameDatabase Name

Database Session IDSession ID

User idUser ID

DescriptionName

SQL Drill DownSQL Drill Down


Database: sysadmin

Statement text:

SELECT MAX(run_task_seq) FROM ph_run A, ph_task B WHERE A.run_task_id

= ?

AND A.run_task_id = B.tk_id AND A.run_time + B.tk_delete < CURRENT

Iterator/Explain

================

ID Left Right Est Cost Est Rows Num Rows Type

3 0 0 1 1 1 Index Scan

4 0 0 19 545 1 Index Scan

2 3 4 20 5 1 Nested Join

1 2 0 1 1 1 Group

Statement information:

Sess_id User_id Stmt Type Finish Time Run Time

21 0 SELECT 10:51:11 0.0023

Statement Statistics:

Page Buffer Read Buffer Page Buffer Write

Read Read % Cache IDX Read Write Write % Cache

0 77 100.00 0 0 0 0.00

Lock Lock LK Wait Log Num Disk Memory

Requests Waits Time (S) Space Sorts Sorts Sorts

0 0 0.0000 0.000 B 0 0 0

Total Total Avg Max Avg I/O Wait Avg Rows

Executions Time (S) Time (S) Time (S) IO Wait Time (S) Per Sec

220 78.8463 0.3584 1.9557 0.000000 0.000000 439.9908

Estimated Estimated Actual SQL ISAM Isolation SQL

Cost Rows Rows Error Error Level Memory

20 1 1 0 0 DR 41552

SQL Drill Down – vue onstatSQL Drill Down – vue onstat


� Outil Web d’administration

� Un seul outil pour superviser plusieurs instances déportées

� Information géographiques sur la localisation des serveurs

� Surveillance pointue des instances

� Lecture claire des erreurs

� Vision centralisée des serveurs

� Géo-Visualisation des serveurs

Open Admin ToolOpen Admin Tool


Développement

Scalabilité

Sécurité

Performance







CheetahCheetah


� Trigger :

– Multiple INSERT, UPDATE, DELETE,SELECT dans une table avec aussi support du INSTEAD OF

– Support des triggers sur les vues

� XML publishing :

– SQL sur XML stockée en base

– XPATH

– Génerer du XML

– Manipuler du XML

� Index Hierarchic :

– Support des données hiérarchiques

– Monde MVS, MainFrame

� Basic Text Datablade :

– Recherche de mots, phrases

– Documents non structurés

– Colonne : CHAR, VARCHAR, LVARCHAR, TEXT, CLOB

– Operations boolean type OR, AND et NOT

DéveloppementDéveloppement


Nouvelles fonctionnalitésde « Cheetah » IDS 11.50


Dallas

Las Vegas

FriscoAustin

TokyoSao PauloParis

OLTP

CATALO

GM

AR

T

IDS 11.5 introduces the Online Connection Manager and Server Monitor (oncmsm):

• a daemon program which

accepts a client connection request and then re-routes that connection to one of the best fit nodes in the IDS cluster.

• Monitors instance activity

levels.

Whi

ch c

atal

og in

stan

ce? La

s Veg

as

Connection ManagerConnection Manager


SLA = Service Level Agreement

• A “contract” between client applications and IDS instances.

• Can be based on probability of performance and data “currency”: • If user requires the most current data as opposed to slight lag

time while replication occurs, user must connect to primary or HDR secondary instance.

• If user can tolerate some degree of latency, then user should be connected to another secondary instance based on “currency” requirement (SDS vs HDR secondary vs RSS).

• Can also be based on geographic or functional requirements:• Connect to instances within X miles/km.or• Connect to instances designated to support specific types of

application workloads. but you must define this by “known” instance names.

SLASLA


For example:

SLA oltp=primary

SLA report=rss_1+rss_2+rss_3

SLA accounting=SDS+HDR

SLA catalog=rss_4+rss_5+rss_6

SLA test=RSS

The following are “reserved words” for SLA definitions:

� primary – the cluster primary.

� SDS – any SDS instance.

� HDR – the HDR secondary.

� RSS – any RSS instance.

An SLA definition can include “reserved words” as well as specific

instance names.

NOTE: each SLA has a separate SQLHOSTS entry on CM

server.

SLASLA


Connection attempts using SLAs:

� SLA oltp=primary

� SLA payroll=HDR+primary

� SLA report=SDS+HDR

SLASLA


Primary HDR secondary

RSS

HDRTraffic

RSS

Traffic

PrimaryDown?

Is Primary

Really Down?

� Monitors all nodes checking for the primary failover.

� Performs failover (i.e. promote secondary to primary) when it is confirmed that the primary is down.

� Released as part of the Connection Manager.

� Will support failover to an RSS node.

� The Failover Arbitrator provides automatic failover logic for high-availability clusters.

Failover ArbitratorFailover Arbitrator


Example:

FOC serv1+(serv2+SDS)+HDR+RSS,10

Failover order if primary doesn’t come “back” after 10 seconds, for

example network connectivity hiccup:

• serv1 instance.

• best choice of serv2 or one of the (surviving?) SDS instances.

• HDR secondary.

• Any RSS node.

� The FOC can be passed to the connection manager (oncmsm) on the

command line as “-f serv1+(serv2+SDS)+HDR+RSS,10”.

Failover ArbitratorFailover Arbitrator


� Client applications can update data on secondary servers!

� The secondary server is not updated directly.

Updatable secondaries give the appearance that updates are occurring directly on the secondary server when in fact the transaction is transferred to the primary server and then the change is replicated to the secondary server.

Updatable SecondariesUpdatable Secondaries


� Attempted writes (insert, update, and delete) are redirected

from the secondary to the primary instance.

� Only supports DML statements, can not create tables,

indexes etc.

� Uses optimistic concurrency to avoid updating a stale copy

of the row.

� Works on all MACH-11 secondary instances.

Updatable SecondariesUpdatable Secondaries


Nouvelles fonctionnalitésde « Cheetah » IDS 11.50xC4


� Ability to store data rows in compressed format on disk

� Saves up to 90% of row storage space

� Space returned can then be used by any table in the dbspace

� Ability to estimate possible compression ratio

� Fits more data onto a page

� Fits more data into buffer pool

� Reduces logical log usage

CompressionCompression


Performance Impact

� IO-bound workloads

– Compression may improve performance by reducing IOs (both data page and logical log)

– More data fits on a page, so more in buffer pool

– Log records are smaller, so less logging

� For CPU-bound workloads

– Additional CPU used to compress and expand rows

– Should not be a large impact



� All are supported on compressed tables

� HDR

– Tables will be compressed on secondary iff they are compressed on primary

� ER

– Compression status of tables is independent between source and target, specified by user



Things That Cannot Be Compressed

� Out-of-row data (e.g. blobs)

� Indexes

� Catalog tables

� Temp tables

� Partition tables

� Dictionary tables

� Tables in the following databases:

sysuser,sysmaster,sysutils,syscdr,syscdcv1



� Free Pages

� Used Pages

� Estimate of used after compression

Individual fragments for table big2



Run your BI/Analytics on Informix

Business Intelligence

and Analytics

• Informix customers want the same characteristics for their core BI/Analytics as they have in IDS

•They want warehousing tools integrated with IDS

•Many customers already use IDS for their BI/Analytics

•Leverage functionality from XPS:

•External tables

•Lite scans

•Hybrid fragmentation

•Administer with OAT

•Integration with other DW/BI products:

•Cognos 8

•SQL Warehouse

•Informix Warehouse for IDS

Informix WarehouseInformix Warehouse


MerciMerci

Mardi 30 novembre 2010Mardi 30 novembre 2010

Laurent Valenza

[email protected]