Oracle Data Integrator 12c (ODI12c) - Powering Big Data ... · PDF fileOracle Data Integrator 12c (ODI12c) - Powering Big Data and Real-Time Business Analytics ... Oracle Data Integrator

Oracle Data Integrator 12c (ODI12c) - Powering Big Data and Real-Time Business Analytics

An Oracle White Paper

October 2013



2

Introduction:

The value of analytics is so widely recognized today that all mid to large scale enterprises have

deployed some level of Business Intelligence (BI) tools and technologies. In spite of this

widespread implementation of analytics, the promise of operational effectiveness is not being

fully realized. According to a recent survey conducted by independent analyst firm, BI

Scorecard, adoption of Business Intelligence (BI) remains surprisingly low—reaching only 25

percent of potential users. For many, the Business Intelligence (BI) tools available are difficult

to use and slow to respond and the content they deliver is of little relevance. As a result,

business users default to making decisions based on incomplete information or “gut feeling.”

The same survey also noted that while 26 percent of respondents reported that their Business

Intelligence deployments were successful, the vast majority of Business Intelligence (BI)

programs are falling short of their potential to increase revenues, improve customer service,

reduce costs, and use strategic insights to create innovative new products or services

Is Data Integration Limiting Business Analytics?

There are a number of factors why organizations have not realized their full potential in analytics.

These are three key factors where Data Integration technologies can help play a key role in solving:

Business Analytics systems are running on stale and incorrect data. Without real-time good

quality data in the data warehouses, data loses its relevance. This negatively impacts operational

efficiencies and jeopardizes the carefully crafted customer experiences.

Business Analytics systems are not designed for Big and Fast Data. Future-proofing business

analytics and data warehousing system is becoming a growing concern to address the increasing

volume, velocity and variety of big data.

Business Analytics systems are disconnected. When Business Intelligence (BI) systems are

fragmented from various planning systems, transactional systems and packaged applications (i.e. ERP,

CRM), facts and assumptions are misaligned. The pitfalls include cost overruns, inventory shortage and

surplus, unnecessary risks, and delayed responses to threats and opportunities.

Each of these obstacles can be overcome by utilizing key best practices in data integration as the core

foundation to business analytics and data warehousing. These strategies include integrating data in real-

time, effectively tapping the value by integrating big data, and finally, integrating data end-to-end with

business analytics systems. In this paper we shall discuss these requirements and how the latest release

of Oracle Data Integrator 12c (ODI12c) along with supporting Oracle Technologies addresses this

growing need of powering Business Analytics for organizations.


3

The Enterprise Data Warehouse – Feeding Business Analytics

Even in today’s world of Big Data, the enterprise data warehouse plays a critical role in feeding

Business Intelligence applications for businesses. Most organizations have a traditional Data warehouse

that needs to be kept current and integrated with their reporting structures to make sure that relevant

data is being surfaced to line of business operational reports and management dashboards. These are

increasingly being integrated with new analytic data warehouses and Big Data clusters and real time

data feeds to provide a comprehensive data store to the Analytics technologies. This population of the

data store requires a Data Integration technology that

Handles bulk loads of traditional and Big Data in and out of the sources and data warehouses

Incorporates Change Data Capture technologies to enable Real Time data movement.

Should be part of a Fast Data ecosystem to handle Big Data, and most importantly

Should be tightly integrated with an Enterprise Business Analytics Solution.

Analyzing Big and Fast Data

Businesses today need to analyze huge amounts of data, including social media activity, web logs,

warranty claims, call center activity, movement of assets with RFID tags, and sensor data. In most

cases, organizations need to integrate unstructured data with data that resides in multiple data sources

such as data warehouses, enterprise business applications, and online analytic processing (OLAP)

cubes. This ‘big data’ is having a tremendous impact on the solution space for business analytics, data

warehousing and data integration. This has to do primarily with the fact that big data poses new

questions for the best ways to process volumes and varieties of data at higher speeds and at faster

velocity. One recent new standard that has emerged, Fast Data, specifically deals especially with this

velocity component of big data and it’s increasingly seen as an important element that ties together

concepts in real-time and in-memory processing with big data and analytics.


4

Fig1: Big Data and Enterprise Data should be used in conjunction for best results.

Loading and Transforming Big Data using Oracle Data Integrator

Oracle Data Integrator is the flagship Oracle Data Integration product. Oracle Data Integrator is

critical to big data projects in providing much needed connectivity to load and transform data from

Hadoop sources and to support structured and relational sources. Oracle Data Integrator leverages

unified tooling for both big data and enterprise data which translates into a faster learning curve as well

as seamless usability so that the data scientist or data analyst can focus on integration versus usability.

Flow based declarative designs in the new release of Oracle Data Integrator, ODI12c helps build

complex expressions that are easy to maintain and support. Two representations of the same ELT

mappings, the logical representation and the physical representation, provide customized working

environments for business analysts and data scientists. Oracle Data Integrator Application Adapter for

Hadoop simplifies data loading and movement between Hadoop and an Oracle Database through

Oracle Data Integrator 12c (ODI12c)’s easy to use rebuilt interfaces.


5

Fig 2: By providing efficient connectivity between Oracle Database and Hadoop, Oracle Big Data Connectors enables

analysis of all data, both structured and unstructured, in enterprise data warehouses.

The Need for Speed - Real-time Data Warehousing and Analytics:

Real time data is absolutely required for efficient operational decision-making. As latency increases,

data becomes less relevant and the ability to act on it decreases, robbing a company’s opportunity to

improve operational efficiencies and increase revenue. Real-time data integration ensures that the data

used in decision-making is as current as possible and in some cases, virtually in sync with operational

information.

Traditional Business Intelligence and Data Warehousing solutions give users an excellent view of past

events and entities by accessing historical data. These solutions typically monitor slowly moving trends,

such as product defects or sales performance. Users gain insight into these activities by querying a data

warehouse, which is updated periodically.

Historical analysis is important, but it mainly targets strategic, long-term decision-making by analyzing

trends, and it doesn’t let people respond to operational events happening in the organization. As data

latency increases, its relevance to operational events diminishes. BI solutions for day-to-day operations

need to leverage the most current data to enable employees to take action right away. Data integration

solutions that leverage heterogeneous Change Data Capture (CDC) technology can establish real-time

links to production data sources in addition to performing bulk updates. These solutions work

efficiently by moving only the changed data to the decision support system.


6

Real-time Business Analytics - Oracle Data Integrator with Oracle GoldenGate

Real time reporting is best delivered when coupled with effective Change Data Capture (CDC)

technology. Oracle’s strategy for enabling real time is realized through Oracle GoldenGate. To enable

real-time data acquisition, Oracle GoldenGate uses log-based, real-time CDC capabilities to provide

continuous capture and delivery of the most recently changed data between OLTP systems and the

data warehouse. The application offers transactional, real-time data capture, routing, transformations,

and delivery, using the push approach. Real time changes to the data are moved to the data warehouse

where it can drive enhanced, strategic, and operational BI capabilities. Oracle GoldenGate can perform

basic, row-level transformations at the point of capture or at the time of delivery.

For heavy transformation requirements Oracle GoldenGate delivers transactional data to a staging area

in the data warehouse for in-database transformations with Oracle Data Integrator to support an E-LT

architecture. Oracle Data Integrator utilizes the native database or execution platform to ensure that

performance gains are not just limited to within the Oracle technology but also in the platform

investments that forms a large part of the Data Integration landscape. Because of this E-LT

architecture, query performances are faster and consequently transformations are faster and more

efficient.

Fig 3: Oracle GoldenGate and Oracle Data Integration combine to provide real time Data warehousing.


7

Integrated BI Applications – Oracle BI Powered by Oracle Data Integrator

Oracle’s latest Business Intelligence Applications (OBIA) uses Oracle Data Integrator (ODI) for its

data integration requirements. Built into the OBIA solution, Oracle Data Integrator provides load

plans, pre built Knowledge Modules and user roles to populate and maintain fact and dimension tables

for OBIA. Load plans, are executables that comprises of multiple dependent and parallel ETL steps

that can be configured to load fact tables from sources. A load plan generator plug is provided that

helps in creating load plans. For more information on understanding how ODI works with OBIA click

here.

Previously, for OBIA versions before 11.1.1.7.1, this ETL part was performed via 3rd party tools. With

the integration of ODI into the solution monitoring and user management is seamless with the ODI

console acting as a monitoring and restarting during failovers and or job statuses.

Dedicated BIApps Knowledge Modules can be used for updating Facts, Slowly Changing Dimensions,

targets and periods. The use of Knowledge Modules eases development cycles in integrating data into

source and target table definitions and allows your whole Business Intelligence solutions to be

maintained as a single unit, delivering a single point of support, following a logical product lifecycle and

lowering TCO.

Oracle Data Integrator enables organizations using BI Applications to increase IT efficiency and

reduce costs with a comprehensive data integration platform that covers all data integration

requirements – including big data, application integration, as well as BI / data warehousing.

Fig 4: Complete integrated Data Integration components for multiple use scenario


8

Summary

Only Oracle provides a complete and best-of-breed platform for integrated business analytics solutions

to handle increasing volume, velocity & variety of big data. The new releases of Oracle Data Integrator

12c (ODI12c) and Oracle GoldenGate 12c has cemented Oracle’s commitment to generate increased

value for customers looking to implement real time analytics use cases, better tap big data sources. The

components of this platform are engineered to work together. In addition, each component is based

on open standards and interfaces that allow customers to leverage their existing enterprise standard

components in areas where investments may have already been made. Oracle is the only vendor today

that can add even more increased value in analytics with unmatched performance, an eye to the future

trends and lower TCO.

To learn more about Oracle Data Integration: Oracle Data Integration

To learn more about Oracle Business Analytics: Oracle BI

http://www.oracle.com/goto/dataintegration

http://www.oracle.com/us/solutions/business-analytics/overview/index.html

White Paper Title

October 2013

Author: Madhu R Nair

Contributing Authors: Dain C Hansen

Oracle Corporation

World Headquarters

500 Oracle Parkway

Redwood Shores, CA 94065

U.S.A.

Worldwide Inquiries:

Phone: +1.650.506.7000

Fax: +1.650.506.7200

oracle.com

Copyright © 2013, Oracle and/or its affiliates. All rights reserved.

This document is provided for information purposes only, and the contents hereof are subject to change without notice. This

document is not warranted to be error-free, nor subject to any other warranties or conditions, whether expressed orally or implied in

law, including implied warranties and conditions of merchantability or fitness for a particular purpose. We specifically disclaim any

liability with respect to this document, and no contractual obligations are formed either directly or indirectly by this document. This

document may not be reproduced or transmitted in any form or by any means, electronic or mechanical, for any purpose, without our

prior written permission.

Oracle and Java are registered trademarks of Oracle and/or its affiliates. Other names may be trademarks of their respective owners.

Intel and Intel Xeon are trademarks or registered trademarks of Intel Corporation. All SPARC trademarks are used under license and

are trademarks or registered trademarks of SPARC International, Inc. AMD, Opteron, the AMD logo, and the AMD Opteron logo are

trademarks or registered trademarks of Advanced Micro Devices. UNIX is a registered trademark of The Open Group. 0113

Documents

Oracle Data Integrator 12c (ODI12c) - Powering Big Data ... · PDF fileOracle Data Integrator 12c (ODI12c) - Powering Big Data and Real-Time Business Analytics ... Oracle Data Integrator