The ABC of ETL with SSIS

The ABC of ETL with SSIS

James Beresfordwww.bimonkey.com

To cover the basics of ETL using Microsoft BI tools

AIM

Agenda Tools Database Structure SSIS Package Level Components

•Database Engine•Integration Server•SQL AgentSQL Server

•dtexec•dtutil

Command Line

•SSMS: SQL Server Management Studio•BIDS: Business Intelligence Development Studio•SQL ProfilerGUI Tools

SQL Server Tools for ETL

SQL Server•Database Engine

stores data – optional

•Integration Server

executes packages

•SQL Agentschedules packages - optional

GUI Tools – SSMS

GUI Tools – BIDS

Where you will spend 95% of your time

GUI Tools – SQL Profiler

Command Line

dtutil - moves

dtexec - does

Database Structure

Audit and Control

Staging

Persistent Consumed

Control Structures

Audit and Control

Control Mechanisms

Logging

BIDS doesn’t work properly!

Use dtutil batch scripts

Deployment

File Systemvs

Package Storevs

SQL Server

Deployment

Create an Execution Account

Do not use SA account

Execution

SSIS SSIS functional units are Packages…

…then within Packages:

Control Flow

Data Flow

SSIS Packages

• Connection managers • Checkpoints• Event Handlers • Configurations• Variables & Expressions• Logging

Connection Managers

Components which refer to a data source or destination

CheckpointsKey Properties at Package Level:

• CheckpointFileName• CheckpointUsage• SaveCheckpoints

FailPackageOnFailure = True for all restart points

Event Handlers

As packages execute, a series of events occur, e.g: • Validation Completing• Errors• Variable values changing

Configurations

Configurations are External Sources

of

Package Execution Settings

Variables & Expressions

Variables provide changeable values to be used by packages

SSIS has its own expression language

Logging

Logging Providers record execution details

SSIS Packages

Control Flow

&

Data Flow

Control Flow

Mission Control!

Control Flow

Containers

Precedence Constraints

Non Data Tasks

Containers

Provide looping

Support Transactions

Precedence Constraints

Control flow of package

Success / Failure & Expressions

Non Data Tasks

• Execute SQL• Execute Package• FTP• Send Mail• Process SSAS Objects• etc…

Data Flow• Sources and Destinations• SCD• Lookup• Data Conversion• Derived Columns• OLEDB Command• Custom components

Learning Resources

learnintegrationservices.com

http://blogs.conchango.com/jamiethomson/