DB2 Z/OS Temporal tables - · dsnt408i sqlcode = 20523, error: table vxb5b1.tempotest_hist was specified as a history table, but the table definition is not valid for a history table

_____________________________________________________________________

0. Abstract As part of data management scenarios, any update and deletion of data

requires and saving old data called version data in tables. In RDMS this is

possible by way of temporal tables. Temporal tables are the tables used to manage

system time sensitive data (providing capability to query old snapshot of data for

auditing) and business time sensitive data (providing period where the the data is

valid for business). This paper caters to the explanation how all these can be

implemented in DB2 for Z/OS and also learn about the features of temporal

tables in DB2 10 and DB2 11

Thanks & Regards

Venkata Rama Rajesh

DB2 DBA for Z/OS

IBM Certified Database Associate (DB2 10)

IBMGlobal Technology Services

Email : [email protected]

Phone: +919492770394

DB2 Z/OS Temporal tables

© Copyright IBM Corporation, 2013

1. Introduction

What is temporal tables

A temporal table is a table that records the period of time when a row is valid.

Temporal Data Management in DB2 Provides automated feature that help you

easily record and query past , current and future data conditions in tables.

Types of temporal tables / temporal data management

There are 3 ways of temporal data management

• System Period Management

• Application Period Management

• BiTemporal Management

System Period Management:

It provides automated feature that helps easily record and query current and

past data conditions in our tables, which provides the capability to query past

snapshot of data.

Application Period Management:

It provides feature to provide period where data is valid for business.

BiTemporal Management:

It provides the combination of system period and application period data

management.

Note :

• Temporal tables were introduced in DB2 10 for z/OS and enhanced in V11 and V12.

• System period temporal tables is for versioned data and system time sensitive data

• Business period temporal tables for business time sensitive data.


2. System period temporal tables

2.1 Implementation

Implementation can be made in 3 steps

1. Create a systemperiod temporal table, CREATE TABLE statement with the

following requirements

Row begin timestamp column

Row end timestamp column

Transaction start Id timestamp column

PERIOD SYSTEM_TIME / PERIOD SYSTEM TIME clause

2. Create a history table for systemperiod temporal table with same defination

(DDL) excluding SYSTEM_TIME, row begin, row end, transaction start Id

clauses.

3. ALTER TABLE ADD VERSIONING statement with the USE HISTORY TABLE

clause to define systemperiod data versioning on the table.

Example

Step : 1 Create base table

CREATE TABLE VXB5B1.TEMPOTEST

(

ID INTEGER,

C_DATA CHARACTER(12) FOR SBCS DATA,

R_B_TS TIMESTAMP(12) NOT NULL GENERATED ALWAYS AS ROW BEGIN,

R_E_TS TIMESTAMP(12) NOT NULL GENERATED ALWAYS AS ROW END,

T_I_TS TIMESTAMP(12) NOT NULL

GENERATED ALWAYS AS TRANSACTION START ID,

PERIOD SYSTEM_TIME (R_B_TS,R_E_TS)

)

IN DATABASE MALLIV;


• These three columns (row begin, row end and transaction start Id) should be NOT

NULL columns and GENERATED ALWAYS columns.

• The system time consists of a pair of columns with systemmaintained values that

indicate the period of time when a row is valid.

• The begin column contains the timestamp value for when a row is created. The end

column contains the timestamp value for when a row is updated or deleted.

Transaction start Id specifies the timestamp value, whenever a row is inserted into

the table or any column in the row is updated.

Step : 2 Create history table

CREATE TABLE VXB5B1.TEMPOTEST_HIST

(

ID INTEGER,


R_B_TS TIMESTAMP(12) NOT NULL ,

R_E_TS TIMESTAMP(12) NOT NULL ,


)

IN DATABASE MALLIV;

Step : 3 Add versioning use history table

ALTER TABLE VXB5B1.TEMPOTEST

ADD VERSIONING USE HISTORY TABLE VXB5B1.TEMPOTEST_HIS;

Note: If history table is not created properly according to its norms set by DB2, then

the history table cannot be successfully for the base table and the following error

message is received.


ADD VERSIONING USE HISTORY TABLE VXB5B1.TEMPOTEST_HIST;


DSNT408I SQLCODE = 20523, ERROR: TABLE VXB5B1.TEMPOTEST_HIST

WAS SPECIFIED AS A HISTORY TABLE, BUT THE TABLE DEFINITION IS

NOT VALID FOR A HISTORY TABLE. REASON CODE = 4

2.2 Data manipulation

INSERT

INSERT INTO VXB5B1.TEMPOTEST(ID,C_DATA)

VALUES (1,'PHANI')

When a row is inserted in system period temporal table, row begin timestamp &

transaction ID timestamp coloumns will be updated with timestamp of inserted row.

End row timestamp column will be updated with maximum timestamp value.

Nothing will be updated in the history table.

Column Base table History table

ID 1 NA

C_DATA PHANI NA

R_B_TS 2016120510.45.18.723150848000 NA

R_E_TS 9999123000.00.00.000000000000 NA

T_I_TS 2016120510.45.18.723150848000 NA

UPDATE

UPDATE VXB5B1.TEMPOTEST

SET C_DATA='DEEPU'

WHERE ID=1;

When a row is updated in system period temporal table, row begin timestamp & →

transaction ID timestamp columns will be updated with timestamp of the

updated row. End row timestamp column will be updated with maximum

timestamp value.


Row before update will be copied into history table, and its row begin timestamp →

remains unchanged. End row timestamp column and transaction ID timestamp

column will be updated with timestamp associated with update.


ID 1 1

C_DATA DEEPU PHANI

R_B_TS 2016120511.00.03.4094.. 2016120510.45.18.7231..

R_E_TS 9999123000.00.00.0000.. 2016120511.00.03.4094..

T_I_TS 2016120511.00.03.4094.. 2016120510.45.18.7231..

DELETE

DELETE

FROM VXB5B1.TEMPOTEST

WHERE ID = 1;

When a row is deleted in system period temporal table, row moves into history

table, and its row begin timestamp remains unchanged. End row timestamp

column and transaction ID timestamp column will be updated with timestamp

associated with delete.


ID NA 1

C_DATA NA DEEPU

R_B_TS NA 2016120511.00.03.4094..

R_E_TS NA 2016120511.10.07.4104..

T_I_TS NA 2016120511.00.03.4094..

Note: DML activities can also be performed on history table. In order to maintain

data consistency between various versions stored in the history table, no DML

activities should be performed on history table.


2.3 Data retrieval

During the execution of DELETE or UPDATE the versioned data (before image) will

be moved to the history table. To retrieve the versioned data (history table data)

along with existing data (base table data), SELECT query requires period

specification (SYSTEM_TIME clause), which can be done in one of the following

three ways as listed below:

• FOR SYSTEM_TIME AS OF …..........................

SELECT ID,C_DATA,R_B_TS,R_E_TS,T_I_TS


FOR SYSTEM_TIME AS OF '2016120512.03.36.376430285000';

• FOR SYSTEM_TIME FROM ........... TO …........



FOR SYSTEM_TIME FROM '2016120512.03.36.376430285000'

TO '2016120512.05.36.376'

• FOR SYSTEM_TIME BETWEEN ..... AND …...



FOR SYSTEM_TIME BETWEEN '2016120512.03.36.376430285000'

AND '2016120512.05.36.376'


3.4 Internal query transformation

If the query contains SYSTEM_TIME clause as a predicate, the query internally

converts into couple of SELECT queries (one on the base table and another one on

the history table) joined with UNION ALL clause. DB2 generates implicit

predicates for each query. Consider the below query with SYSTEM_TIME clause.



FOR SYSTEM_TIME FROM '2016120512.03.36.376430285000'

TO '2016120512.05.36.376'

WHERE ID < 3

DB2 implicitly generated equivalent query as shown below.


FROM

( SELECT ID,C_DATA,R_B_TS,R_E_TS,T_I_TS


WHERE R_B_TS < '2016120512.05.36.376'

AND R_E_TS > '2016120512.03.36.376430285000'

UNION ALL


FROM VXB5B1.TEMPOTEST_HIS

WHERE R_B_TS < '2016120512.05.36.376'

AND R_E_TS > '2016120512.03.36.376430285000'

) AS TEMPOTEST

WHERE ID < 3


3. Application period temporal tables

3.1 Implementation

An application period temporal table has two usermaintained and NOT NULL Date

or Timestamp without time zone columns denoting the period of time for which data

is valid in the table. Both columns must be of the same datatype.

Below DDL is an example to create an application periodtemporal table.

CREATE TABLE VXB5B1.APPTEMPO

(CUST_ID INTEGER

,C_DATA CHARACTER(12) FOR SBCS DATA

,BUS_ST_DT DATE NOT NULL

,BUS_EN_DT DATE NOT NULL

,PERIOD BUSINESS_TIME (BUS_ST_DT,BUS_EN_DT)

) IN DATABASE MALLIV

3.2 Data manipulation

There is no implicitly updated columns for application period temporal tables.

Columns have to be updated manually as per the business logic.

INSERT INTO VX$B5B1.APPTEMPO(CUST_ID,C_DATA,BUS_ST_DT,BUS_EN_DT)

VALUES (1,'PHANI','20161201','20161231');

PORTION OF BUSINESS_TIME clause

With DB2 10 for z/OS, to delete or update data in a portion of time on an

application period temporal table, DELETE/UPDATE clauses need to be used with

the FOR PORTION OF BUSINESS_TIME clause. Synatx and example is shown

below.

Syntax:

DELETE FROM..FOR PORTION OF BUSINESS_TIME FROM value1 TO value2;

UPDATE ..FOR PORTION OF BUSINESS_TIME FROM value1 TO value2;


Example:1

UPDATE VX$B5B1.APPTEMPO

FOR PORTION OF BUSINESS_TIME FROM '20161201' TO '20170131'

SET CUST_ID=9

WHERE CUST_ID< 4

Example:2

DELETE

FROM VX$B5B1.APPTEMPO

FOR PORTION OF BUSINESS_TIME FROM '20161201' TO '20170131'

WHERE CUST_ID< 4

3.3 Data retrieval

SELECT query requires period specification (BUSINESS_TIME clause) extract

the data which is applicable for a specific period, which can be done in one of the 3

ways as below.

• FOR BUSINESS_TIME AS OF …..........................

SELECT CUST_ID,C_DATA,BUS_ST_DT,BUS_EN_DT


FOR BUSINESS_TIME AS OF '20170101' ;

• FOR BUSINESS_TIME FROM ........... TO …........



FOR BUSINESS_TIME FROM '20170101' TO '20170131' ;

• FOR BUSINESS_TIME BETWEEN ..... AND …...



FOR BUSINESS_TIME BETWEEN '20170101' AND '20170131' ;


3.4 Internal query transformation

SELECT / INSERT / UPDATE statements may contain the BUSINESS_TIME

clause. DB2 will implicitly convert the FOR clause to a WHERE clause predicate

for such queries as shown in below examples.

Example1



FOR BUSINESS_TIME AS OF '20170101'

WITH UR;

DB2 implicity generated equivalent query as shown below.



WHERE BUS_ST_DT <= '20170101'

AND BUS_EN_DT > '20170101'

WITH UR;

Example2

DELETE


FOR PORTION OF BUSINESS_TIME FROM '20161201'

TO '20170131'

WHERE CUST_ID< 4

DB2 implicitly generated equivalent query as shown below.

DELETE

FROM POLICY_ATT

WHERE BUS_ST_DT < '20170131'

AND BUS_EN_DT > '20161201'


3.5 Period overlapping

BUSINESS_TIME WITHOUT OVERLAPS clause indicates that the values for the

nonperiod columns and expressions of the index key for a row must be unique with

respect to the time represented by the BUSINESS_TIME period for the row. DB2

enforces that multiple rows do not exist with the same key values for the columns or

expressions of the index, with overlapping time periods. Below is the DDL for

unique index creation with BUSINESS_TIME WITHOUT OVERLAPS clause.

CREATE UNIQUE INDEX VX$B5B1.APPTEMPOIX

ON VX$B5B1.APPTEMPO (CUST_ID,BUSINESS_TIME WITHOUT OVERLAPS);

Consider the below example, let's assume a row is present in the above table with

CUST_ID=1. Now one another INSERT is trying to insert a new row with

CUST_ID=1 with an overlapped time period between 20170202 and 20170329 as

shown, which will eventually fail with duplicate key issue (SQLCODE=803).

CUST_ID C_DATA BUS_ST_DT BUS_EN_DT

1 RAVI 20170202 20170329

INSERT INTO VX$B5B1.APPTEMPO

(CUST_ID,C_DATA,BUS_ST_DT,BUS_EN_DT)

VALUES (1,'RAVI','20170202','20170329');

DSNT408I SQLCODE = 803, ERROR: AN INSERTED OR UPDATED VALUE IS

INVALID BECAUSE INDEX IN INDEX SPACE APPT1LDS CONSTRAINS

COLUMNS OF THE TABLE SO NO TWO ROWS CAN CONTAIN DUPLICATE

VALUES IN THOSE COLUMNS.

RID OF EXISTING ROW IS X'0000000203'.

Note: If we try to add a BUSINESS _TIME WITHOUT OVERLAPS index on the

table containing overlapped data for the application time period, then the CREATE

INDEX will fail with SQLCODE = 603.


4. Advanced topics

4.1 Bi Temporal tables• BTTs manage both system time and business time, and combine all the capabilities

of systemperiod and applicationperiod temporal tables. This combination enables

applications to manage the business validity of their data while DB2 keeps a full

history of any updates and deletes . Every BTT is also an STT and an ATT.

Example:

CREATE TABLE VXB5B1.TEMPOTEST

(ID INTEGER,


R_B_TS TIMESTAMP(12) NOT NULL GENERATED ALWAYS AS ROW BEGIN,

R_E_TS TIMESTAMP(12) NOT NULL GENERATED ALWAYS AS ROW END,


GENERATED ALWAYS AS TRANSACTION START ID,

PERIOD SYSTEM_TIME (R_B_TS,R_E_TS),

BUS_ST_DT DATE NOT NULL,

BUS_EN_DT DATE NOT NULL,

PERIOD BUSINESS_TIME (BUS_ST_DT,BUS_EN_DT)

)IN DATABASE MALLIV;

4.2 Temporal registers

CURRENT TEMPORAL SYSTEM_TIME and CURRENT TEMPORAL

BUSINESS_TIME are couple of special registers were newly introduced in DB2 11.

The values in this registers can be set using SET statement as shown below.

SET CURRENT TEMPORAL SYSTEM_TIME = NULL

SET CURRENT TEMPORAL BUSINESS_TIME =TIMESTAMP('20080101')+5 DAYS;

SET CURRENT TEMPORAL BUSINESS_TIME '2017010600.00.00.000000000';


SYSTIMESENSITIVE YES:

If we specified this option during bind. when BINDing the application, DB2 will

prepare two separate access path strategies for queries against system temporal

tables. As shown in Figure, DB2 will then use the relevant access strategy

depending on whether the CURRENT TEMPORAL SYSTEM_TIME register is set.

This approach provides the best of both worlds, with applications able to access

historical data when required but avoid unnecessary performance overheads when

it is not.


4.3 Catalog information• If base table has any history table then that table and its schema are recored in

VERSIONING_SCHEMA and VERSIONING_TABLE columns of sysibm.systables

catalog table, otherwise these columns are blank.

SELECT CREATOR, NAME, TYPE, VERSIONING_SCHEMA,VERSIONING_TABLE

FROM SYSIBM.SYSTABLES

WHERE VERSIONING_TABLE <> ' '

WITH UR;

CREATOR NAME TYPE VERSIONING_SCHEMA VERSIONING_TABLE

VXB5B1 SPTT T VXB5B1 HIS_SPTT

VXB5B1 HIS_SPTT H VXB5B1 SPTT

VXB5B1 TEMPOTEST T VXB5B1 TEMPOTEST_HIS

VXB5B1 TEMPOTEST_ H VXB5B1 TEMPOTEST

• If the entry is for history table, then VERSIONING_SCHEMA and

VERSIONING_TABLE columns has corresponding base table schema and base

table name. To avoid such records in result table, below query will help.

SELECT CREATOR, NAME, TYPE, VERSIONING_SCHEMA,VERSIONING_TABLE

FROM SYSIBM.SYSTABLES

WHERE VERSIONING_TABLE <> ' '

AND TYPE = 'T'

WITH UR;

CREATOR NAME TYPE VERSIONING_SCHEMA VERSIONING_TABLE

VXB5B1 SPTT T VXB5B1 HIS_SPTT

VXB5B1 TEMPOTEST T VXB5B1 TEMPOTEST_HIS


• PERIOD and DEFAULT columns of SYSCOLUMNS table will help to obtain

SYSTEM_TIME and BUSINESS_TIME clause information.

• If DEFAULT column has 'X' means corresponding column is defined with the AS

TRANSACTION START ID attribute.

• PERIOD column indicates whether the column is the start or the end of the period

for a SYSTEM_TIME or BUSINESS_TIME period.

S Column is the start of period SYSTEM_TIME

T Column is the end of period SYSTEM_TIME

B Column is the start of period BUSINESS_TIME

C Column is the end of period BUSINESS_TIME

blank Column is not used as either the start or the end of a period

Below is the sample query to get above column information.

SELECT TBCREATOR,TBNAME, NAME, PERIOD, DEFAULT

FROM SYSIBM.SYSCOLUMNS

WHERE PERIOD <> ' '

OR DEFAULT='X'

WITH UR;

TBCREATOR TBNAME NAME PERIOD DEFAULT

VXB5B1 SPTT R_B_TS S Q

VXB5B1 SPTT R_E_TS T R

VXB5B1 SPTT T_I_TS X

VXB5B1 APT_T BUS_ST_DT B N

VXB5B1 APT_T BUS_EN_DT C N

VXB5B1 TEMPOTEST R_B_TS S Q

VXB5B1 TEMPOTEST R_E_TS T R

VXB5B1 TEMPOTEST T_I_TS X

VXB5B1 APPTEMPO BUS_ST_DT B N

VXB5B1 APPTEMPO BUS_EN_DT C N


4.4 Differences

System time Business time

Captures the time when changes

happened to data inside DB2

Captures the time when changes

happened to business data

Past to the present time Past, present, and future time

DB2's physical view of time The application's logical view of time

System validity (transaction time) Business validity (valid time)

Present data will be available in base

table and past data will be available

in history table.

Total data is present in base table only.

There is no need of history table.

Supports queries such as what prices

were stored in the database on June

30?

Supports queries such as: What price

policy was active on June 30?

4.4 System temporal recoveryBoth the base table and history table can be recovered to different points of recovery.

But it is always recommended to recover both tables to a common point of recovery

so as to maintain data consistency aross versions stored in both tables. For

example

RECOVER TABLESPACE (MALLIV.TEMPOTES DSNUM ALL)

TABLESPACE (MALLIV.TEMP1KYW DSNUM ALL)

TOLOGPOINT X'0D1BB9FC5CE6'

VERIFYSET YES

Note: VERIFYSET Specifies whether the RECOVER utility verifies that all related

objects (can be History or LOB or XML tables) that are required for a pointintime

recovery are included in the RECOVER control statement. VERIFYSET is valid

only for pointintime recovery TORBA or TOLOGPOINT.


4.5 System Temporal Table restrictions1. No utility operation can that deletes data from a systemperiod temporal table can

be performed. These utilities include LOAD REPLACE, REORG DISCARD, and

CHECK DATA DELETE YES.

2. Column name or table name of a systemperiod temporal table or a history table

cannot be renamed.


RENAME COLUMN ID TO C_ID ;

DSNT408I SQLCODE = 750, ERROR: THE SOURCE TABLE VXB5B1.

TEMPOTEST CANNOT BE RENAMED OR ALTERED AS SPECIFIED

3. Schema (data type, check constraint, referential constraint, etc.) cannot be altered

for a systemperiod temporal table or history table; however, a new column can be

added to a systemperiod temporal table.

4. No clones can be defined on the systemperiod temporal table or the history table.

5. Both the systemperiod temporal table or history table have to be inside a single

table tablespace.

6. No CHECK DATA utility with the options LOBERROR INVALIDATE, AUXERROR

INVALIDATE, or XMLERROR INVALIDATE can be run on a systemperiod

temporal table. The CHECK DATA utility will fail with return code 8

7. History table or its table space cannot be dropped explicitly. (If base table is

dropped, then history table will implicitly get dropped)

DROP TABLE VXB5B1.TEMPOTEST_HIS

DSNT408I SQLCODE = 478, ERROR: DROP OR REVOKE ON OBJECT TYPE

TABLE CANNOT BE PROCESSED BECAUSE OBJECT VXB5B1

TEMPOTEST OF TYPE TABLE IS DEPENDENT ON IT

Note: None of the above restrictions are eliminated from DB2 10 to DB2 12 for z/os


4.6 Temporal views

Views can be created on temporal tables

EX: CREATE VIEW TEMO_VIEW AS (SELECT

ID,C_DATA,R_B_TS,R_E_TS,BUS_ST_DT,BUS_EN_DT

FROM VX$B5B1.BITEMPOTEST)

With V11 can use BUSINESS_TIME clause in DELETE and UPDATE statements

on temporal table based views

EX: UPDATE TEMO_VIEW

FOR PORTION OF BUSINESS_TIME FROM ‘20130101' TO ‘20130601'

SET ID = ID + 1;

4.7 Inclusive / exclusive period

With DB2 V12, EXCLUSIVE / INCLUSIVE keywords are implicit check

constraints which ensure the relationship of the value of endcolumnname to the

value of begincolumnname as follows

EXCLUSIVE : Specifies that the value of the end column for the BUSINESS_TIME

is not included for the row inserted (or) modified. The BUSINESS_TIME period is

defined as begincolumnname (inclusive) and endcolumnname (exclusive).

INCLUSIVE: Specifies that the value of the end column is included in the period.

The BUSINESS_TIME period is defined as inclusiveinclusive.


5. Bibliography

http://www.ibmbigdatahub.com/blog/temporaltablesdb2

https://www.ibm.com/support/knowledgecenter/SSEPEK_12.0.0/admin


http://www.ibmbigdatahub.com/blog/temporal-tables-db2

https://www.ibm.com/support/knowledgecenter/SSEPEK_12.0.0/admin

Documents

DB2 Z/OS Temporal tables - · dsnt408i sqlcode = 20523, error: table vxb5b1.tempotest_hist was specified as a history table, but the table definition is not valid for a history table