View
222
Download
0
Tags:
Embed Size (px)
Citation preview
SQL Server 2005Integration ServicesSQL Server 2005Integration Services
Dave GloverDave GloverMicrosoft AustraliaMicrosoft Australiahttp://blogs.msdn.com/dgloverhttp://blogs.msdn.com/dglover
AgendaAgenda
Cleanse Data InputSplit an Output ChannelScript to Branch ControlLookup Fuzzy ValuesLoop through FolderConfigure and Deploy
OverviewOverview
Part of SQL 2005 Std and Ent Editions:Integration Services with Basic Transforms. Provides graphical extract, transform, and load (ETL) capabilities Ent only: Integration Services Advanced Transforms Includes data mining, text mining, and data cleansing
Collaborative DevelopmentSeparate Management ToolsData Quality and profilingScalable
SSIS Runtime
WMIWMI
Sources
Flat FileFlat File
DestsOLED
B C
onnectio
n F
TP
HTTP
FIL
E O
DB
C M
anagers C
US
TO
M
FLA
TFIL
E
CustomCustom
XMLXML
SQLSQLServerServer
Flat FileFlat File
FTPFTP
FileFile
OLED
B C
onnectio
n F
TP
HTTP
FIL
E O
DB
C M
anagers C
US
TO
M
FLA
TFIL
EWEBWEBFTPFTP
DBDB
XML TaskFILE
FTP Task FTP
Web Services Task
HTTP
Data Flow Task
OD
BC
…
FILE
…
CustomCustom
Containers
WMI Event Task
WM
I
Event Handlers
Send Mail Task Send Mail Task
Variables Lo
g P
rov
ide
rs
Task HostTask Host
Custom TaskCustom Task
VariablesVariables Lo
g P
rov
ide
rsXMLXML
Cleansing Data InputCleansing Data Input
Connection-Related ObjectsConnection ManagerFlow TypesData AdapterMetadata ValidationData Flow Components
Flow TypesFlow Types
Two different types of flowControl Flow = Runtime = TasksData Flow = Pipeline = Transforms
Managed in DesignerUsed to be single view in SQL 2000Separate views in SQL 2005
Control Flow handles tasks and precedenceData Flow handles transformations - zoomed contents of Data Flow task
Data Flow ComponentsData Flow Components
Components
PathsData route between one component and the nextIncludes metadata about columns moving aroundLineage Identifier tracks item transformations
PipelineComponents connected by a path
Source Transform Destination
Splitting an Output ChannelSplitting an Output Channel
Distributors and CollectorsPrecedence Constraints
Distributors and CollectorsDistributors and Collectors
Distributor transformationsMulticastConditional Split
Collector transformationsUnion AllMergeMerge Join SrcSrc
TfmTfm
DstDstDD CCTfmTfm
TfmTfm
Precedence ConstraintsPrecedence Constraints
Connect one task to anotherGive sequential relationship to tasks
Success/Failure/Completion workflowEstablish concurrency
Connected sequence of tasks is a task listIndependent Task Lists execute concurrentlyTasks within a list execute sequentially
•Script and Branch Control•Script and Branch Control
Package VariablesScript TaskComplex PrecedenceContainers
Package VariablesPackage Variables
ScopeEach container can have variablesDefine namespace for user variablesContainers can access variables from higher levels
Accessible fromExpressions (such as loops and constraints) – Use @Parameters in Execute SQL taskParent Package (as part of configuration)Script
Script TaskScript Task
Currently VB .NetCan read or modify properties throughout the packageCan’t access inner workings of tasks or transformsCan’t modify pipeline metadata (e.g., number of columns piped)
ContainersContainers
Container providesGrouping of task lists (list of one is allowed)Transaction scopeVariable scope
A package is a containerAdd your own tasksInsert your own containersLoops are containers, too
Fuzzy Lookup TransformationFuzzy Lookup Transformation
Proximity algorithm to find matchesBuilds index – Index can persistCreates metrics
SimilarityConfidence
Uses a separate connection for reference table
LoopsLoopsFOR LOOP
Loops while expression is TRUEManually add loop counter
Init: @N = 1Eval: @N <= 25Increment: @N =@N + 1
Execute tasks in container on each iterationMore control—and more complex than For Each
FOR EACH LOOPLoops over set of objects
FilesXML nodesDatabase objects
Set variable (e.g. file name) for each iterationExecute tasks in container on each iteration
Package Lifecycle SupportPackage Lifecycle Support
Data VisualizersDebugging
Stop execution during packageStop control flow before and afterStop transformation during (visualizer)
LoggingEvent HandlersLog Providers
ConfigurationsConfigurations
Take something from the systemEnvironment variables, registry, XML option fileApply it to some part of your packageRun the package with the new setting
Useful for multiple “similar” jobsSimilar to Dynamic Properties from SQL 2000Facilitate reusability, different environments
Configurable at runtime or during execution
DeploymentDeployment
Same server deploymentDeploy from BI Development Studio
Multiple server deploymentDeployment UtilityPackage configuration files with packageDeploy to SQL Server (msdb) or file (dtsx)
ExecutionExecution
Command-line executionDTEXEC
User Interface executionDTEXECUICan generate command line for DTEXEC
SchedulingSQL Server Agent
© 2004 Microsoft Corporation. All rights reserved.This presentation is for informational purposes only. Microsoft makes no warranties, express or implied, in this summary.
Report Builder ArchitectureReport Builder Architecture
SQL Server CatalogSQL Server CatalogSQL Server CatalogSQL Server Catalog
Web Service / URL interfaceWeb Service / URL interfaceWeb Service / URL interfaceWeb Service / URL interface
Report ServerReport Server
DeliveryDeliveryRenderingRendering
QueryQuery SecuritySecurity
Report BuilderReport BuilderClientClient
Report BuilderReport BuilderClientClient
ModelModelDesignerDesigner
ModelModelDesignerDesigner
Data Sources(SQL Server,
Analysis Services)
Drill through report generationDrill through report generation
Query generationQuery generation Model SecurityModel Security
Report ProcessingReport Processing
ReportReportDesignerDesigner
ReportReportDesignerDesigner
SQL Mgmt SQL Mgmt StudioStudio
SQL Mgmt SQL Mgmt StudioStudio
ReportReportManagerManagerReportReport
ManagerManager
Why Report Builder?Why Report Builder?
Report Report ConsumersConsumers
Business Business UsersUsers
Report Viewer
Report Builder
Power UsersPower UsersDevelopersDevelopers
Report Designer
What is Report Builder?What is Report Builder?A new ad-hoc report design tool for SQL Server Reporting Services Targeted at business users who want to find and share answers to interesting questionsDriven from a business model of the data so users do not need to understand the underlying data structuresNot a full analytical client or replacement for Pivot TablesFully integrated with Reporting Services and delivered in SQL Server 2005
Report Builder vs. Report DesignerReport Builder vs. Report DesignerReport Builder Report Designer
Targeted at business users Targeted at IT pros and developers
Ad hoc reports Managed reports
Auto-generates queries using semantic layer on top of the source
Native queries (SQL, OLE DB, XML/A, ODBC, Oracle)
Reports built on templates Free-form (nested, banded) reports
Click-once application, easy to deploy and manage
Integrated into Visual Studio
Cannot import Report Designer reports
Can work with reports built in Report Builder
Wrap-upWrap-upReport Builder is…
a new ad hoc report design tool for SQL Server Reporting Servicestargeted at business users who want to find and share answers to interesting questionsdriven from a business model of the data so users do not need to understand the underlying data structuresis fully integrated with SQL Server Reporting Services and delivered in SQL Server 2005