HANA Bootcamp SQLScript and ABAP Integration

Building SQL Script based Applications

Christian Behrens, Mirco SternJune 2012

© 2012 SAP AG. All rights reserved. 2

The Customer

Headquarter: Vernier Switzerland (Geneva) Fragrance and flavor industry Global leader

Highly regulated products Compliance checks


The Problem: Compliance Checks

Imagine being one of 100 creators

You just happened to compose this great new fragrance

Then:

Enter the composition into “Sphinx”

AND: Check compliance

(> 1/2 hr, i.e.: Have lunch!)

The case for SAP HANA


Requested Solution

Engine to check formulas against compliance rules

“Compliance Engine 2”, powered by SAP HANA

Business Facts

Phase I (“Fragrance”) Phase II (“Flavor”)

Project Start July 2011 February 2012

Go-Live November 2012 September 2013

Project Effort ~2200 Person Days ~4000 Person Days


Agenda

Introduction

Reference Architecture

Architecture & Design of Compliance Engine 2 High-Level Architecture Best Practices & Lessons Learned: Building Applications in SQLScript

Lifecycle Management, System Landscape & Transports

Integration with ABAP Invoking SQLScript-Procedures from ABAP

Conclusion

Reference Architecture


SAP Custom Development

Make customers’ core business processes real real-time

≠ Reporting

Provide customer-specific in-memory solutions

– Enhance SAP standard applications

– Develop next generation applications

For Compliance Engine 2 Enhancement of EHS (SAP Standard) SAP HANA solution is accessed from EHS

Custom DevelopmentOfferings for

SAP HANA

Accelerators

Support

Enhance-ments

Applications


SAP HANA as a Secondary DB

Side-by-side scenario Replicate data into SAP HANA

ABAP Platform

SAP HANADBSLT

SAP Suite New App


Next Generation ApplicationsThere’s more to it than just changing the underlying database

SQLScript: Move calculations into DB – Benefit from power of SAP HANA DB

Traditional Model: “Data to Code”

DB Server

Application

Server Code

New Model: “Code to Data”

SAP HANA DB

Application

Server

Code


Reference Architecture (SAP HANA as a Secondary DB)

Side-by-side scenario Replicate data into SAP HANA

ABAP Platform

HANADBSLT

SAP Suite New App

New App

SQLScript

Architecture & Design of Compliance Engine 2

High-Level Architecture

Best Practices & Lessons Learned: Building Applications in SQLScript


SAP HANA

Calculation results

SAP Application ServerKernel 7.20

3rd Party

Master Data


Legacy Systems

Mid

dle

wa

re

EH&S

ComplianceEngine

DB

SAP HANAInterface

Proxy

SLT

Constraints

SubstancesCompositions

CheckResults

CompositionsWhere Used

Check Execution

R

R

Integration

Architecture & Design

Architecture & Design of Compliance Engine 2


Best Practices & Lessons Learned: Building Applications in SQLScript


Bring Code to Data: SQLScript

CREATE PROCEDURE getOutput (IN cnt INTEGER, OUT output_pubs tt_publishers, OUT output_year tt_years) LANGUAGE SQLSCRIPT READS SQL DATA ASBEGIN

big_pub_ids = SELECT publisher FROM books -- Query Q1 GROUP BY publisher HAVING COUNT (isbn) > :cnt;

big_pub_books = SELECT title, price -- Query Q2 FROM :big_pub_ids, publishers, books WHERE pub_id = pid AND pub_id = publisher;

output_pubs = SELECT SUM(price) -- Query Q3 FROM :big_pub_books GROUP BY publisher;

output_year = SELECT SUM(price) -- Query Q4 FROM :big_pub_books GROUP BY year;

END;

SQL ScriptCompiler

Publisher

PUBS Year

Q3 Q4

Q2

Q1


SQLScript: SQL vs. Build-Ins

ot_books_agg = SELECT pub_year, COUNT(title) AS cnt FROM :lt_books GROUP BY pub_year;

lt_books_proj = CE_PROJECTION(:lt_books, ["TITLE", "PUB_YEAR"]);ot_books_agg = CE_AGGREGATION(:lt_books_proj, [COUNT("TITLE") AS "CNT"], ["PUB_YEAR"]);

Disadvantages of Build-Ins

Development effort increases

Optimizer is bypassed

Advantages???


SQLScript: Imperative vs. Declarative Statements

lt_remaining_reals = …

SELECT COUNT(*) INTO lv_remaining_reals FROM :lt_remaining_reals;

WHILE lv_remaining_reals > 0 DO lt_remaining_reals = …

SELECT COUNT(*) INTO lv_remaining_reals FROM :lt_remaining_reals;

END WHILE;

Avoid imperative statements! – Negative impact on performance


Data Flow for Compliance Checks

Application is a huge data flow

Thin orchestration layer Bars in the diagram

Aggregate Results

Extract Constraints

Composition 2Composition 1 Composition n

Check Type 1 Check Type 2 Check Type m…

…


Best Practices I

Manage Complexity

Encapsulate as usual using procedures

Structure as usual in packages

Parallelization

Writing Data breaks parallelization

Encapsulate writing and non-writing code separately


SQLScript: ReadOnly vs. ReadWrite

SQL Script Client

ReadWrite Procedure

ReadOnly Procedure


Best Practices II

Client Handling

Encapsulation in views

CREATE VIEW "EHS_CE"."V_CE2_BB_HIER" ( "OBJ_ID", "CHILD_ID", "CHILD_SEQ_NO", "CHILD_IMPACT" ) AS SELECT ZOBJ_ID AS OBJ_ID, ZCHILD_ID AS CHILD_ID, ZCHILD_SEQ_NO AS CHILD_SEQ_NO, ZCHILD_IMPACT AS CHILD_IMPACT FROM EHS_CE.ZGL_CE2_BB_HIER WHERE MANDT = SESSION_CONTEXT('client');

Requires SET SESSION statement prior to execution

SET SESSION client = '100';


Best Practices III

Avoid ROW Store! EXPLAIN PLAN

Example: SELECT GREATEST(1, MIN(status)) AS status FROM :tab; SELECT CASE WHEN status > 1 THEN status ELSE 1 END AS status FROM :tab;

Avoid UNION (ALL)

Use Build-In CE_UNION_ALL

One and only exception when to use build-ins

Requires proper data type handling

Example: stat1 = SELECT 1 AS status FROM :tab; -- Results in a float as data type for status stat2 = SELECT TO_INTEGER(1) AS status FROM :tab; -- Results in an integer for status Compilation of stat = CE_UNION_ALL(:stat1, :stat2); fails due to incompatibility of tables


Best Practices IV

Encapsulate Imperative Statements in Separate Procedure

No Infinity-Values in SAP HANA DB Our solution: | att_value | is_infinity |

– Is_infinity: -1, 0, 1 for -, not , – If (is_infinity ≠ 0) then att_value = NULL

MIN and MAX operations work as expected

Otherwise: Absence of Best practices Try it! Examples:

– CASE vs. Branch & CE_UNION_ALL – GROUP BY on many columns vs. GROUP BY on key + additional JOIN


Debugging – Trace Data

Debugging Debugging requires modifications Problem in customer system

Debugging example:

-- Intermediate result to be inspected:lt_composition = SELECT … FROM :lt_raw_composition;

lt_composition = TRACE (:lt_composition);

-- Consume intermediate resultsot_composition = SELECT … FROM :lt_composition;


Debugging – Analyze Trace

Get Trace Result Tables

SELECT * FROM sys.sqlscript_trace;

Get Trace Result

SELECT * FROM SQLSCRIPT_TRACE_4F95E715495CC1CEE10000000A428043;


Debugging – Impact on Compilation and Execution

Statements

lt_constraints_ext = SELECT request_id, cst_id, status, CASE WHEN (status = 1) THEN to_double(NULL) ELSE to_double(0.0) END AS maximum_dosage, CASE WHEN (status = 1) THEN to_integer(1) ELSE to_integer(0) END AS maximum_dosage_infinit FROM :lt_const_ext AS ce;

lt_constraints_ext_compl = SELECT concomp.request_id AS request_id, concomp.cst_id AS cst_id, conext.status AS status, conext.maximum_dosage AS maximum_dosage, conext.maximum_dosage_infinit AS maximum_dosage_infinit, FROM :lt_constraints AS concomp LEFT OUTER JOIN :lt_constraints_ext AS conext ON concomp.request_id = conext.request_id AND concomp.cst_id = conext.cst_id;



are equivalent to

lt_constraints_ext_compl = SELECT concomp.request_id AS request_id, concomp.cst_id AS cst_id, conext.status AS status, conext.maximum_dosage AS maximum_dosage, conext.maximum_dosage_infinit AS maximum_dosage_infinit, FROM :lt_constraints AS concomp LEFT OUTER JOIN ( SELECT request_id, cst_id, status, CASE WHEN (status = 1) THEN to_double(NULL) ELSE to_double(0.0) END AS maximum_dosage, CASE WHEN (status = 1) THEN to_integer(1) ELSE to_integer(0) END AS maximum_dosage_infinit FROM :lt_const_ext AS ce ) AS conext ON concomp.request_id = conext.request_id AND concomp.cst_id = conext.cst_id;



That‘s what the optimizer does

This results in a Bug

See Internal message 810472/2012

CASE statement is not working properly In optimized statement status may be NULL So the ELSE part is executed and 0.0 resp. 0 is returned instead of NULL

Workaround Consider NULL explicitely as third possible value



TRACE statement breaks this optimization

i.e. two statements remain

lt_constraints_ext = SELECT request_id, cst_id, status, CASE WHEN (status = 1) THEN to_double(NULL) ELSE to_double(0.0) END AS maximum_dosage, CASE WHEN (status = 1) THEN to_integer(1) ELSE to_integer(0) END AS maximum_dosage_infinit FROM :lt_const_ext AS ce;

lt_constraints_ext = TRACE(: lt_constraints_ext);

lt_constraints_ext_compl = SELECT concomp.request_id AS request_id, concomp.cst_id AS cst_id, conext.status AS status, conext.maximum_dosage AS maximum_dosage, conext.maximum_dosage_infinit AS maximum_dosage_infinit, FROM :lt_constraints AS concomp LEFT OUTER JOIN :lt_constraints_ext AS conext ON concomp.request_id = conext.request_id AND concomp.cst_id = conext.cst_id;


Unit Tests

General Concept

Create own schema for test data tables

Create own package(s) for unit test procedures

Create overall test procedure that calls every single unit test

Single Unit Tests for each procedure

Create tables for Input data Expected output

Unit test procedure Reads input data Calls procedure to be tested Compares calcualted output with stored output Returns differences


User Setup

Coding refers to a default schema

Background: Transporting

Avoid qualified names in coding

To simplify transports, make sure everything is in one schema No need for dedicated replication user Replicate into app user’s schema

Have a dedicated Application User (Technical User)

Second Schema for Test Data

Use schema mapping for test execution


Links

Next Generation DB Wiki

https://wiki.wdf.sap.corp/wiki/display/ngdb/Home

Service Market Place

https://websmp109.sap-ag.de/hana

Book

Joe Celko's Thinking in Sets: Auxiliary, Temporal, and Virtual Tables in SQL





http://www.amazon.de/Joe-Celkos-Thinking-Sets-Management/dp/0123741378/ref=sr_1_1?ie=UTF8&qid=1336631485&sr=8-1



Exercise


Business Background

Products need to adhere to a huge number of constraints

Constraints are specified by laws, authorities, industry standards, organizations, customers, …

Constraints are hierarchally grouped with four levels

Regulators that contain constraints

Sub-profiles that may contain constraints and regulators

Super Sub-profiles that may contain constraints, regulators and sub-profiles

Profiles that may contain constraints, regulators, sub-profiles and super sub-profiles

Example

Baby food sold in Germany

Two regulators: Baby Food and Food in Germany and additional constraints

All constraints of both regulators need to be fulfilled


Data Model

Compliance Objetcs

Table ZGL_CE2_COBJ

View V_CE2_COBJ

Hierarchy

Table ZGL_CE2_BB_HIER

View V_CE2_HIER

Request

Table ZGL_REQUEST

View V_REQUEST

(all objects are located in Schema HANA_BOOTCAMP

Object IDObject TypeMaintenance State

Compliance Object

Object IDChild Object IDChild Sequence NumberChild Impact

Compliance Object Hierarchy

is child*

is father

*

Request IDObject ID

Request* 1..*


Exercise

Create Procedure to extract constaints from the hierarchy

Use own package

Create a procedure Input is one single request ID

– One value to simplify the procedure call Output is a table of all constraints with constraint type

– Master data maintenance error is indicated by NULL for constraint type

Integration with ABAP


Access SAP HANA via Native DB Calls

Set up Secondary Connection

Then it’s just a DB call…









lo_dbconn = cl_sql_connection=>get_connection( con_name = 'DEMO_111' ).

CREATE OBJECT lo_sql EXPORTING con_ref = lo_dbconn.

lo_result = lo_sql->execute_query( 'SELECT * FROM MY_HANA_TABLE"' ).


Invoking SQLScript Procedures from ABAP

Major issue: Table transfer

lo_sql->execute_query( 'SET SESSION ''client'' = ''' && sy-mandt && '''' ).

lo_sql->execute_ddl( 'TRUNCATE TABLE "IT_PROD_IDS"' ).LOOP AT it_prod_ids INTO lv_prod_id_input. lo_sql->execute_update( 'INSERT INTO "IT_PROD_IDS" VALUES(''' && lv_prod_id_input &&''' )' ).ENDLOOP.

lo_sql->execute_ddl( 'TRUNCATE TABLE "OT_COMPOSITION"' ).

lo_result = lo_sql->

execute_query( 'CALL "_SYS_BIC"."sap.teched/EXPLOSION"( "IT_PROD_IDS", "OT_COMPOSITION" ) WITH OVERVIEW ' ).

lo_result->close( ).

lo_result = lo_sql->execute_query( 'SELECT * FROM "OT_COMPOSITION"' ).

lo_sql->execute_query( 'SET SESSION ''client'' = ''' && sy-mandt && '''' ).

lo_sql->execute_ddl( 'TRUNCATE TABLE "IT_PROD_IDS"' ).LOOP AT it_prod_ids INTO lv_pid. lo_sql->execute_update( 'INSERT INTO "IT_PROD_IDS" VALUES('''&& lv_pid &&''')' ).ENDLOOP.

lo_sql->execute_ddl( 'TRUNCATE TABLE "OT_COMP"' ).

lo_result = lo_sql->execute_query(

'CALL "_SYS_BIC"."sap.teched/EXPLOSION"( "IT_PROD_IDS", "OT_COMP" ) WITH OVERVIEW').

lo_result->close( ).

lo_result = lo_sql->execute_query( 'SELECT * FROM "OT_COMP"' ).

Exercise


Exercise

Customer wants to know which particular checks are executed

i.e. call existing SAP Hana procedure from ABAP

Configure a Data Base Connection

Create an ABAP Class that encapsulate the call of SAP HANA

One method to call the procedure created

Create an ABAP Report to test the method

Template

You may copy the report and class from local objects of user D030764

Conclusion


Key Takeaways

CD is ready to deliver “next generation applications“ powered by SAP HANA First customer projects started SSRS revenue (coding must be IP relevant)

The key differences in a SQLScript project are The need for data Testing is very time-intensive

In a real project, you need to account for the evolving nature of SAP HANA DB Transports are difficult impacts testing Debugging is difficult esp. in customer landscape

Thank you

Contact information:

Christian BehrensPrincipal Solution Architect SAP Custom Development

Documents

HANA Bootcamp SQLScript and ABAP Integration