SQL Server Stored Procedures Architecture & Performance Victor Isakov MCT, CTT, MSCE, MCDBA [email protected]

SQL Server Stored Procedures

Architecture & Performance

SQL Server Stored Procedures

Architecture & Performance

Victor IsakovVictor IsakovMCT, CTT, MSCE, MCDBAMCT, CTT, MSCE, [email protected]@informare.com.au

© Victor Isakov [email protected]

Victor Isakov ([email protected])Victor Isakov ([email protected])

Victor Isakov has been involved with SQL Server Victor Isakov has been involved with SQL Server since Version 4.21a, enjoying a reputation as one of since Version 4.21a, enjoying a reputation as one of the best "SQL Gurus" in the countrythe best "SQL Gurus" in the country

He has taught the Administration, Development, He has taught the Administration, Development, Data Warehousing and Performance Tuning courses Data Warehousing and Performance Tuning courses since 1995 in the Australasia-Pacific region and since 1995 in the Australasia-Pacific region and provided consultancy services to companies like provided consultancy services to companies like the University of NSW, Department of Education and the University of NSW, Department of Education and Training, Westpac and the United Nations Training, Westpac and the United Nations

He specialises as a consultant in Optimising, He specialises as a consultant in Optimising, Troubleshooting and Architecting Database Troubleshooting and Architecting Database Solutions based on Microsoft technologySolutions based on Microsoft technology


AgendaAgenda

Stored Procedure ArchitectureStored Procedure Architecture

Stored Procedure OptimizationStored Procedure Optimization

Managing Query Plan PerformanceManaging Query Plan Performance

Undocumented Microsoft Stored ProceduresUndocumented Microsoft Stored Procedures


AgendaAgenda



Managing the Query Plan for Managing the Query Plan for Performance.Performance.

Undocumented Microsoft Stored Undocumented Microsoft Stored ProceduresProcedures



SPs are SQL Server programsSPs are SQL Server programsTraditionally written in Transact-SQLTraditionally written in Transact-SQLIn Yukon will include VB.NET, C#In Yukon will include VB.NET, C#

ProvideProvideQuery ManagementQuery ManagementData ModificationsData ModificationsData / Referential IntegrityData / Referential IntegrityMaintenance TasksMaintenance TasksProtect DataProtect Data

Provide Gateway to Database ObjectsProvide Gateway to Database Objects



Stored in Database Stored in Database Protected like other DB objectsProtected like other DB objects

Logic moved to serverLogic moved to server

Increased Developer Productivity Increased Developer Productivity Reusable CodeReusable Code

More structured 3-Tier solutionsMore structured 3-Tier solutions

Eliminate needless query plan Eliminate needless query plan constructionconstruction

Cached query plans Cached query plans cancan improve improve performance performance


Compiled Compiled Execution Plan Execution Plan

QueryQuery

ChangesChanges


SQL Server

Security

UnauthorizedUnauthorized

Stored Procedure

SELECT logic SELECT logic

Business Rules Business Rules

Constraints Constraints


Execution Plan in DetailExecution Plan in Detail

Query PlanQuery PlanRe-Entrant, Read-Only StructureRe-Entrant, Read-Only Structure

SharedShared

Separate Parallel / Serial PlansSeparate Parallel / Serial Plans

Execution ContextExecution ContextUser ContextUser Context

Parameter Values, etcParameter Values, etc

Not SharedNot Shared

Can be Re-UsedCan be Re-Used


AgendaAgenda



Managing the Query Plan for Managing the Query Plan for Performance.Performance.



CREATE PROCEDURE…


Parse TSQL Syntax

Resolve References

Save in Database

EXEC @RC=MyProc…

Resolve references

Optimize

Compile

Cache (RAM)

Execute


Compilation in DetailCompilation in Detail


RAM Cache


SQL Server

1st instance compiled and QP loaded2nd instance shares loaded QP (new EC)

3rd instance shares first QP (& 1st EC)

1st instance finishes

Data Pages

EXEC @RC=MyProc…


Compilation / Execution in DetailCompilation / Execution in Detail


Stored Procedures OptimizationStored Procedures Optimization

Ad-hoc QueriesAd-hoc QueriesCompiled when first usedCompiled when first used

EP EP potentiallypotentially cached cached

Subsequent referencesSubsequent referencesQuery Optimizer compares Query Optimizer compares query with query with Query-TemplatesQuery-Templates

Conservative Conservative

Use cached plan Use cached plan if if it’s it’s recognizedrecognized

Auto-ParameterizationAuto-Parameterization

Cach

e (R

AM

)

Data Pages


Processing T-SQL in DetailProcessing T-SQL in Detail



SPs compiled on first useSPs compiled on first useQuery plan cached in RAMQuery plan cached in RAM

Subsequent calls generally Subsequent calls generally use cached planuse cached plan

Recompiled if….Recompiled if….

Cach

e (R

AM

)

Data Pages


RecompilationRecompilation

WITH RECOMPILE in CREATE WITH RECOMPILE in CREATE PROCEDURE or EXECUTE statementPROCEDURE or EXECUTE statement

Schema changes to referenced objectsSchema changes to referenced objectsTable referenced by stored procedureTable referenced by stored procedure

Adding / dropping constraints, defaults, Adding / dropping constraints, defaults, or rulesor rules

sysobjects.schema_versysobjects.schema_ver

Restoring the database or any objects Restoring the database or any objects procedure referencesprocedure references



Running sp_recompile on tableRunning sp_recompile on table

SET OptionsSET OptionsMay affect query resultMay affect query result

ANSI_DEFAULTSANSI_DEFAULTS

ANSI_NULLSANSI_NULLS

ANSI_PADDINGANSI_PADDING

ANSI_WARNINGSANSI_WARNINGS

CONCAT_NULL_YIELDS_NULLCONCAT_NULL_YIELDS_NULL

Certain operations on temporary tablesCertain operations on temporary tables



Interleaving DDL and DML in procedureInterleaving DDL and DML in procedure

Server activity ages plan out of cacheServer activity ages plan out of cache

Table TypeTable Type Empty ConditionEmpty Condition Threshold When EmptyThreshold When Empty Threshold When Not EmptyThreshold When Not Empty

PermanentPermanent < 500 rows< 500 rows Changes >= 500Changes >= 500 Changes >= 500 + (20% of Cardinality)Changes >= 500 + (20% of Cardinality)

TemporaryTemporary < 6 rows< 6 rows Changes >= 6Changes >= 6 Changes >= 500Changes >= 500

VariableVariable Change in Cardinality does not affect Automatic RecompilationChange in Cardinality does not affect Automatic Recompilation


Profiler Trap SP:RecompileProfiler Trap SP:Recompile

CodeCode ReasonReason

11 Schema, bindings, or permissions changed Schema, bindings, or permissions changed between compile or execute between compile or execute

22 Statistics changedStatistics changed

33 Object not found at compile time, deferred Object not found at compile time, deferred check to run timecheck to run time

44 Set option changed in batchSet option changed in batch

55 Temp table schema, binding, or permission Temp table schema, binding, or permission changedchanged

66 Remote rowset schema, binding, or permission Remote rowset schema, binding, or permission changed changed


Avoiding RecompilationAvoiding RecompilationUse sp_executesqlUse sp_executesql

Preferred MethodPreferred Method

Calling Stored Procedure unaffectedCalling Stored Procedure unaffected

Use Sub-Procedures Use Sub-Procedures Only smaller sub-procedure recompiledOnly smaller sub-procedure recompiled

Use KEEP PLAN OptionUse KEEP PLAN Option

SET OptionsSET Options


AgendaAgenda






Query OptimizationQuery Optimization

All input parametersAll input parametersWhether used or notWhether used or not

Suitable indexesSuitable indexes

Data statistics Data statistics

AllAll logic in the procedure logic in the procedureWhether or not the code is Whether or not the code is executedexecuted

Query ComplexityQuery Complexity

IF

IF


Query Optimization in DetailQuery Optimization in Detail



Generated query plan based on parametersGenerated query plan based on parametersProvided by Provided by firstfirst query (arbitrary) query (arbitrary)

Cached and reused for all subsequent useCached and reused for all subsequent useRegardless of suitabilityRegardless of suitability

Some queries Some queries run normally, run normally, others do notothers do not


Managing the CacheManaging the Cache

Flushing the CacheFlushing the CacheDBCC FREEPROCCACHEDBCC FREEPROCCACHE

DBCC DROPCLEANBUFFERSDBCC DROPCLEANBUFFERSTip: Should CHECKPOINT first to ensure all data is Tip: Should CHECKPOINT first to ensure all data is flushed, otherwise “dirty” data pages will remain in the flushed, otherwise “dirty” data pages will remain in the cache.cache.

Restart SQL ServerRestart SQL Server

Power-cycle systemPower-cycle system


SQL Server CacheSQL Server Cache

Least-Frequently-Used strategyLeast-Frequently-Used strategyLeast used objects flushed when cache fillsLeast used objects flushed when cache fills

Unified cache stores data and procedure pagesUnified cache stores data and procedure pagesLazywriter does know the differenceLazywriter does know the difference

Monitor cache with PerfmonMonitor cache with Perfmon


Inserting QP into CacheInserting QP into Cache


Retrieving QP From CacheRetrieving QP From Cache


Lazywriter Cache Processing FlowLazywriter Cache Processing Flow


Managing Query Plan Performance Managing Query Plan Performance

Recompilation might Recompilation might helphelp performance performance When “optimized” query is not optimalWhen “optimized” query is not optimal

Cost of recompile insignificant compared to Cost of recompile insignificant compared to poorly running querypoorly running query

Test all parameter combinationsTest all parameter combinationsCheck for consistent plans, performanceCheck for consistent plans, performance

If query plan varies based on parametersIf query plan varies based on parametersRecompile for each execution?Recompile for each execution?

Best to redesign procedureBest to redesign procedure



Recompiling on DemandRecompiling on DemandCREATE PROCEDURE … WITH RECOMPILECREATE PROCEDURE … WITH RECOMPILE

Compiles QP each time stored procedure is executedCompiles QP each time stored procedure is executed

EXECUTE … WITH RECOMPILEEXECUTE … WITH RECOMPILEWhen parameters are not “typical” When parameters are not “typical”

sp_recompilesp_recompileForces Forces allall plans to be recompiled (very cheap) plans to be recompiled (very cheap)

Point to stored procedure, table…Point to stored procedure, table…

Statement-based recompileStatement-based recompileDynamic string execution (dangerous, but powerful)Dynamic string execution (dangerous, but powerful)

Smaller, more-focused proceduresSmaller, more-focused procedures



Use Query Analyzer to view Query PlanUse Query Analyzer to view Query PlanExecute query with a range of input Execute query with a range of input parametersparameters

Clear procedure and data cacheClear procedure and data cache

View IO StatisticsView IO Statistics

Enable “Show Execution Plan”Enable “Show Execution Plan”






Managing Query Plan Performance StrategyManaging Query Plan Performance Strategy

Flush cache DBCC FREEPROCCACHEFlush cache DBCC FREEPROCCACHE

Force recompile WITH RECOMPILEForce recompile WITH RECOMPILE

Avoid “all-purpose” stored proceduresAvoid “all-purpose” stored procedures

SET OptionsSET Options


Managing Query Plan Performance StrategyManaging Query Plan Performance Strategy

Re-engineer stored proceduresRe-engineer stored proceduresBreak up larger, more complex proceduresBreak up larger, more complex procedures

Each sub-procedure gets its Each sub-procedure gets its own query planown query plan

Design procedures to work Design procedures to work with “typical” parmswith “typical” parms

Build special case proceduresBuild special case procedures

Use templates!Use templates!


PerformancePerformance

Construct Command object

Execute query

Parse, resolve, build QP

Execute query

Return resultsets

Client-side processing

Don’t sweat the small stuff…Don’t sweat the small stuff…

Client-side preparation

Client-side consumption

Server-side execution


AgendaAgenda

Stored Procedure Architecture.Stored Procedure Architecture.






Stored Procedure Architecture.Stored Procedure Architecture.






Seven for the DBASeven for the DBAsp_EnumErrorLogssp_EnumErrorLogs

sp_ReadErrorLogsp_ReadErrorLog

sp_TempDbSpacesp_TempDbSpace

sp_EnumOleDbDatasourcessp_EnumOleDbDatasources

sp_MSForEachDbsp_MSForEachDb

sp_MSLoginMappingssp_MSLoginMappings

sp_MSTableSpacesp_MSTableSpace


sp_EnumErrorLogssp_EnumErrorLogs


sp_ReadErrorLogsp_ReadErrorLog


sp_TempDbSpacesp_TempDbSpace


sp_EnumOleDbDataSourcessp_EnumOleDbDataSources


sp_MSForEachDBsp_MSForEachDB


sp_MSLoginMappingssp_MSLoginMappings


sp_MSTableSpacesp_MSTableSpace



Seven for the DeveloperSeven for the Developersp_GetTypeStringsp_GetTypeString

sp_MSCheck_UId_Owns_Anythingsp_MSCheck_UId_Owns_Anything

sp_MSGet_Qualified_Namesp_MSGet_Qualified_Name

sp_MSForEachTablesp_MSForEachTable

sp_MSIs_PK_Colsp_MSIs_PK_Col

sp_MSTable_Has_Unique_Indexsp_MSTable_Has_Unique_Index

sp_MSTableRefssp_MSTableRefs


sp_GetTypeStringsp_GetTypeString


sp_MSCheck_UId_Owns_Anythingsp_MSCheck_UId_Owns_Anything


sp_MSGet_Qualified_Namesp_MSGet_Qualified_Name


sp_MSForEachTablesp_MSForEachTable


sp_MSIs_PK_Colsp_MSIs_PK_Col


sp_MSTable_Has_Unique_Indexsp_MSTable_Has_Unique_Index


sp_MSTableRefssp_MSTableRefs


SummarySummary

Audience:Audience:““Who cares, let us go home! Please!”Who cares, let us go home! Please!”

Victor: Victor: ““But I need to put up something…”But I need to put up something…”

Understand how SQL Server worksUnderstand how SQL Server works

Helps you help SQL Server toHelps you help SQL Server toexecute “optimal” plansexecute “optimal” plans

““KISS” PrincipleKISS” Principle

Slides are up on www.sqlserver.org.auSlides are up on www.sqlserver.org.au

Documents

SQL Server Stored Procedures Architecture & Performance Victor Isakov MCT, CTT, MSCE, MCDBA [email protected]