What's new on SAP HANA Smart Data Access

1© 2014 SAP AG or an SAP affiliate company. All rights reserved.

SAP HANA SPS 11 - What’s New? Smart Data Access

SAP HANA Product Management December, 2015(Delta from SPS 10 to SPS 11)

© 2015 SAP SE or an SAP affiliate company. All rights reserved. 2Public

Agenda

Overview Motivation, Key Concepts, Architecture Modelling

New Features in SPS11 SDA monitoring SDA adapters

– Use of SDI adapters for Smart Data Access– SDA MII adapter– SDA SPARK SQL adapter

PowerPC support

Best Practices

Overview


Motivation

IssueCustomers are facing a heterogeneous system landscape across different locations, storing huge amount of data in different formats

RequirementCustomers require a cost-efficient, easy-to-deploy solution to get real-time data visibility across their fragmented data sources, e.g. for operational reporting, monitoring, predictive analysis, search

Virtualized Access

….

Enterprise databases

Warehouse databases

Master data database

….

Report

HDFS cluster

….


Some typical use cases

USE CASE DESCRIPTION

Developing Apps Using Dispersed Enterprise-Wide Data

• Build applications on HANA, but access data from other sources using HANA smart data access, without moving data into HANA

• Initially supported sources – HANA, ASE, IQ, Teradata, HIVE/Hadoop

Federation of data in Hadoop • Access Hadoop/HIVE/Spark data from HANA

Big Data access • Access big data stores (e.g. Sybase IQ), without loading data into HANA

HANA-to-HANA • Federated HANA databases (e.g. cross-reporting)


Introducing SAP HANA smart data access

Key Value Proposition Enables access to remote data just like “local”

table Smart federated query processing Supports data location agnostic development No special syntax to access heterogeneous

data sources Cost savings Benefit from HANA functionality without

moving all data to HANA Rapid deployment of high-performance, data

intensive transactional and analytical applications


Key Concept of SAP HANA smart data access

SAP HANA

DB catalog

Column Tables

Row Tables

Remote Data System

Remote object catalog

Remote Object

(Table/View)

Virtual Table

The SAP HANA smart data access is based on local virtual tables that maps to an existing object at the remote data source site


Technical Architecture

StoredProcedures Geospatial

SAP HANAStudioKXEN, Lumira

Columnstore, Rowstore

Text XS AFL / PAL

Query Compilation and Execution

SAP BI 4.0

Calculation Engine SQL Engine

Graph Federation

Main features: • Integration with Hana Studio and DDL

support (create remote sources and virtual tables)

• Optimized for reducing data transfer

• Support of virtual tables in the calculation views

SAP Hana

SAP Hana

Client Apps

....

SAP IQ

SAP ASE

MSSQL

DB2

Oracle

HadoopMaxDB

ODBC

Adapters


Modelling HANA Smart Data Access

Create Virtual Tables

CreateFederated Model Consume Define

Data Sources

Data Source Virtual Tablesrefers to

HANA Viewincludes

HANA Tables

includes

Query HANA Client

User credentials

refers to

ODBC Driver

Install SourceDriver

uses

New Features in SPS11


SDA monitoring in the SAP HANA Cockpit

Installation Download content from the SAP Service Market Place (http://service.sap.com) Note: You can search for “SMARTDATA”

http://service.sap.com/



Extract and import content as Delivery Unit Open HANA Cockpit and select the additional SDA tiles from the catalog



No differences to smart data access monitoring in HANA Studio, but additionally remote user and transaction-id of the remote query are displayed in the SDA HANA Cockpit


SAP HANA SDA Adapters - Introduction

All supported SAP HANA SDA adapters are listed in a central SAP-note 1868209.

With HANA SPS11 smart data access can use SDI adapters to access more 3rd party remote sources and their versions.

New 3rd party remote source will only be supported by SDI adapters that can be used by SDA.

http://service.sap.com/sap/support/notes/1868209


Use of SDI adapters for Smart Data Access

Conversion of a Smart Data Access remote source into a Smart Data Integration data source

Pre-requisites

• The Data Provisioning Server (DP Server) has been started

• The Data Provisioning Adapters (SDI Adapters) have been configured

Supported Conversions

SDA Adapter SDI Adapter

MSSQL (GENERIC ODBC) MssqlLogReaderAdapter

ORACLE (GENERIC ODBC) OracleLogReaderAdapter

DB2 (GENERIC ODBC) DB2LogReaderAdapter


Configure SAP SDI adapters for Smart Data Access

Installation of SDI components• Activate DP-Server in the HANA Studio and deploy the DP-Agent on your remote source host

• Configure DP-Agent and register new SDI adapter (will be visible in HANA remote source definition screen)

Use SDI adapter• Create remote source on new SDI adapter (you can also alter an existing remote source to the new SDI adapter)

• Browse remote object catalog and create virtual table


Activate DP-Server


Install DP-Agent


Example: SDI File-Adapter

DP Agent Config Tool

HANA Studio (Create remote source)


SDA adapter for SAP MII

The SAP HANA SDA adapter for SAP Manufacturing Integration and Intelligence (SAP MII) can be used to connect an SAP MII back-end system to an SAP HANA system

• Delivered with SAP HANA (base) installation

• Installation and Configuration is described in SAP-note 1984859 (see attached PDF-files)

• Queries on the SAP MII side are exposed as virtual tables to SAP HANA

• The adapter transforms queries to their virtual tables into appropriate SAP MII service queries

• The results from SAP MII are then transformed into rows of the virtual tables

http://service.sap.com/sap/support/notes/1984859


HANA SDA SPARK adapter

SAP HANA SDA has been providing the SPARK SQL adapters since SPS10

• SAP HANA Spark SQL Adapter is a plugin for HANA Smart Data Access framework

• It provides access to the Spark Controller and moderates query execution and data transfer

• SAP HANA Spark Controller enables virtual tables on underlying Hive tables (SPS10)

• New SPS11: SAP HANA Spark Controller enables virtual tables on HANA Vora tables

• Pre-Requisite: Download and Install HANA Spark Controller (see also SAP note: 2177933)

https://css.wdf.sap.corp/sap/support/notes/2177933


SDA Performance tuning

• SAP HANA SDA optimizes the data transfer between local data and remote data primarily

• SQL-Optimizer techniques: filter pushdown, join-relocation

• SQL Optimizer needs statistic information about the remote table:

CREATE STATISTICS ON “SCHEMA".“VIRTUAL_TAB”(“COL1", “COL2", “COL3") TYPE HISTOGRAM;

• Virtual Tables in HANA views

• Use virtual tables in calculation views, as SDA optimizations works with calculation views only

• You might save unneeded remote accesses by using the new union node enhancements in calculation views with SPS11

• The union node enhancements allow you to define pruning rules, which may prevent remote SDA accesses - if you define unions between virtual tables and local tables

• Configure DML Mode (readonly/readwrite): if applicable, just use readonly DML-Mode as it provides performance advantages for remote select-access

SDA Adapters

• You can use HANA SDI adapters for SDA (pre-requisite: license for HANA EIM option)

• More information: http://help.sap.com/hana_options_eim?current=hana_platform

• Best practice on SDI adapter deployment: http://scn.sap.com/community/developer-center/hana/blog/2015/08/05/hana-adapter-sdk--setup

• Future support of new remote sources and their versions based on SDI adapters

Best Practices

http://help.sap.com/hana_options_eim?current=hana_platform

http://help.sap.com/hana_options_eim?current=hana_platform

http://scn.sap.com/community/developer-center/hana/blog/2015/08/05/hana-adapter-sdk--setup

http://scn.sap.com/community/developer-center/hana/blog/2015/08/05/hana-adapter-sdk--setup


Disclaimer

This presentation outlines our general product direction and should not be relied on in making a purchase decision. This presentation is not subject to your license agreement or any other agreement with SAP.

SAP has no obligation to pursue any course of business outlined in this presentation or to develop or release any functionality mentioned in this presentation. This presentation and SAP’s strategy and possible future developments are subject to change and may be changed by SAP at any time for any reason without notice.

This document is provided without a warranty of any kind, either express or implied, including but not limited to, the implied warranties of merchantability, fitness for a particular purpose, or non-infringement. SAP assumes no responsibility for errors or omissions in this document, except if such damages were caused by SAP intentionally or grossly negligent.

© 2015 SAP SE or an SAP affiliate company. All rights reserved.

Thank you

Contact information

Ruediger KarlSAP HANA Product [email protected]

mailto:[email protected]

Data & Analytics

What's new on SAP HANA Smart Data Access