Numberate master presentation

NUMBERATE 8.0Rhino

Technology Overview

December 2, 2014Numberate Data Warehousing & Business Intelligence Solutions

The Numberate Rapid Warehouse Solution is a data integration solution that maximises the cost efficiency of data oriented business intelligence environments.

It builds on over a 15 Years of experience delivering these solutions to a range of corporate and government clients who needed to leverage business growth from their organisations underlying data assets.


Numberate Rapid Warehouse Solution

The Numberate Rapid Warehouse Solution differs from other solutions because of its flexibility, its non-proprietary architecture and its speed of implementation.

Based around the concept of business rules, it incorporates a high degree of reusability allowing customers to leverage and extend their existing investments in data warehousing and business intelligence.



The Numberate Rapid Warehouse Solution is the right solution to meet the needs of your business for going forward. You need to give your IT professionals tools that;

1. Allow them to focus on your business rather than our technology.

2. Provide options and alternatives rather than mandating yet another upgrade or add on.

3. Deliver in weeks rather than months.

4. Are driven by your business rules and processes rather than driving your business broke.

Data Warehousing & Business Intelligence Solutions



Solution Requirements: Business

THE CHALLENGE

Business users don’t understand their data

Information is power “Its distribution is threatening”

New information, or new views, changes requirements

Timeliness is everything

“Yesterday’s issues are meaningless today.”

THE NEEDS

• Involve business to

– Confirm solutions

– Change their mind

• Rapid development through automation

• No penalties for rework

• Return control of data and distribution to the business


Solution Requirements: ETL

THE CHALLENGE

Data from disparate systems doesn’t match

Cross systems business rules difficult to model

Dirty data blows out exception handling

Users change their mind

Upstream systems constantly change

THE NEEDS

Capture business rules once

Allow users to prototype inside the ETL process

Automate the process

• Maintaining the rules

• Defining the transforms

• Defining the target environment

Provide capacity for rework


Solution Requirements: Project

THE CHALLENGE

Development program midstream

Fixed price contract

Existing tool sets

Existing development partners

Cost constrained project

Project schedule pressure

THE NEEDS

No upfront investment

Fixed price

Reduce development schedules

Reduce cost and time for work packages

Provide alternate approaches

Data Warehousing & Business Intelligence SolutionsDecember 2, 2014Numberate Data Warehousing & Business Intelligence Solutions

A different approach

The Numberate Rapid Warehouse Solution automates the process of building and maintaining a warehouse

→ Open, off the shelf, commodity technologies.

→Solution prototyped before decision.

→ Performance benchmarked before full implementation

→ Independent of target technology

Value Proposition


A different approach to building a data warehouse

Solution Space

Warehouse Development Environment Rapid transformations Prototyping Maintenance

Warehouse Integration Environment All major tools and RDBMS’s

Warehouse Enhancements Storage Front end Backup archive and recovery


Front EndPower Users

Numberate Architecture

Standard Reporting Tools

ODBC

Detailed Data Cache or Store

Lo

gic

al

Vie

ws

RD

BM

S A

PI

RDBMS ODBC

RDBMSRepository

or DataWarehouse

ETL Environment

Operational Systems

Operational Systems

Operational Systems


Numberate Matrix

PhysicalDataElements

AdditionalLoad Code

Raw Data

Raw Data

LoadExecuteData

Code Generation Engine

User Data Request

Result Set

End User

LogicalDatabaseDefinition

User Query Builder

User Data

Request

Result

Metadata Manager

User ElementSecurity

Intersection

Numberate Matrix

Numberate Matrix Manager


Technology Features

Platform Transparency

Uses inexpensive, standard components: hardware, software Disk

Uses any storage devices orthogonally disk, tape, CD ROM,

Independent of front end tools and back end data stores

Horizontally scalable on any platform

Does not require identical hardware


Value Proposition

Single metadata repository

Extract and transform data

Define warehouse environment

Define third party environments

Maintain, schedule and operate warehouse

Scalable storage solution

Terabyte enabled

Uses standard server / disk environments

Proprietary partitioning technology


Metadata- Magic or Myth

Multiple standards

Multiple repositories

Residing in multiple separate systems

Addresses only a partial view of the environment

Not integrated with operational environments

Usually managed as a task additional to the implementation


Metadata

SOURCES

Numberate metadata generates efficient, error free code for the entire warehouse environment

NumberateMetadata

Business Rules

User Regts

Manual Inputs

Existingwarehouse

Datamart

Sourcedatabase

ExistingMetadata

USES

NumberateMetadata

ExtractionRoutines

DefineStaging

area

Transformationprograms

Definewarehouse

tables

Definetableloads

Generatetests

Loadfrontendmetadata


Numberate Metadata – It’s Alive..

Active Automated Accurate Essential Self documenting End to end Reusable Extensible Pervasive

One and done approach to build a warehouse

The tool manages the project work

Reduces effort, time and cost

Maintain the warehouse through the metadata


Numberate Kernel

Numberate Unplugged

Code Repository

RWSAdmin Interface

RWSUser Interface

3rd Party Metadata Interfaces

Metadata Repository

Metadata activates program objects


Staging & Test area

Source transactional data is partitioned for parallel processing.

Partitioned at table level.

Can be repartitioned for performance tuning with minimal rework.

Parallel operations come from standard systems multi threading.

SummariesAnd Derived

Data

Standard Operating System

Java Environment

Numberate Environment


Unstrained Integration

By architecture Hub and spoke Enterprise Focused Data Marts Normalised / Schemas

ETL Direct database load Parallel ETL processes Auditable Low maintenance costs

Prototyping/ Front end Derived from metadata Low cost Rapid development times


Integration Services

Numberate Integration Layer

DW

Legacy Repositories

New Datamarts Emerging Technologies

Data & Event Bus

Front End Environments


TransformationEngine(s)

TransformationEngine(s)

Typical Transformation Architecture

Target logical models cannot be validated by users without the data

Data quality makes inferred relationships

difficult to code

Transformation Engines are single stream, linear processes

Disparate systems cannot sustain desired relationships

Data Architectures and models may need to be radically altered:PerformanceData QualityBusiness needs


Iterate the Stream

Traditional Transformation Process

OracleSourceSystem

OracleSourceSystem

Teradata Target System

Teradata Target System

Coboldata

extractionprocess

Coboldata

extractionprocess

Fastloador

Multiload

Fastloador

Multiload

TeradataSQL

Programs

TeradataSQL

Programs

Transformation processes occur at all points

• Difficult to plan• Hard to code manually• Very hard to change or rework

Single stream approach• Limits transformation models• Requires many iterations.


Traditional Transformation Process

1. Use the warehouse for ETL

2. Co-ordinate many environments

3. Co-ordinate multiple teams

4. High project management load

5. High documentation load

6. Target environment is not independent of ETL stream


Non Proprietary Architecture

The Numberate Rapid Warehouse Solution leverages the power of modern multi- tasking operating systems and advanced storage solutions to provide scalable data environments that can operate in near real time and provide rapid access to complex

queries in business time.


ETL Strengths

ETL has full units of parallelism Multi staged ETL developed as one process

Implemented as different streams

SAS

Utilities

SQL

C / Cobol Sophisticated high speed pattern

matching Matrix Input Loader Generic Data Conversion Module Saves tens of thousands of lines of code

Generated code is error free Metadata driven End to end – including warehouse definition


Elements of Compatibility

STORAGE

Disk / tape / CD-rom

Appear as RDBMS views

Low cost storage

Suitable for transactional histories

DATA ARCHITECTURE

Normalised data models

Star Schemas

Snowflake Schemas

DATA STORAGE

Mix and match capability

Oracle / DB2 / Teradata/SAS

Native file systems


Meeting Your Needs

End to end automation reduces time and cost -Provide faster, cheaper work packages

Capture existing sources of metadata-Warehouse work already completed-Definitions in old repositories and data marts

Provide you with more options-Transformation can be done on

• Mainframe• Data Warehouse• Unix

-Move between platforms as run time option

Solution includes a RAD / prototyping environment-Model and confirm business requirements-More accurate reporting specifications



If your organisation is ready to move to a new level of robust solutions that actually

reduce your IT spend, then you are probably ready for the



Small Business & Entrepreneurship

Numberate master presentation