Upload
serkan-kiraci
View
218
Download
0
Embed Size (px)
Citation preview
8/8/2019 SEAS2009 Query Optimization Final
http://slidepdf.com/reader/full/seas2009-query-optimization-final 1/65
SQL Server
Enterprise ArchitectSummit
June 09-12, 2009Shanghai, China
SEAS09SEAS09
8/8/2019 SEAS2009 Query Optimization Final
http://slidepdf.com/reader/full/seas2009-query-optimization-final 2/65
Query Optimization and
Query Tuning in SQLServer 2008
Howard YinPrincipal Program Manager
SQL Server Customer Advisory Team
Microsoft Corp.
8/8/2019 SEAS2009 Query Optimization Final
http://slidepdf.com/reader/full/seas2009-query-optimization-final 3/65
3
SEAS09SEAS09
| SQL Server Customer Advisory Team
Agenda
• SQLCAT Introduction• Query Optimizer Architecture
• Query Plan Analysis
• Influencing Plan Selection• Special Considerations
– Parameterized Queries
– Temporary Tables and Table Variables
– Parallel Queries
• References
8/8/2019 SEAS2009 Query Optimization Final
http://slidepdf.com/reader/full/seas2009-query-optimization-final 4/65
4
SEAS09SEAS09
| SQL Server Customer Advisory Team
Agenda
• SQLCAT Introduction• Query Optimizer Architecture
• Statistics Used by Optimizer
• Query Plan Analysis
• Influencing Plan Selection
• Parameterized Queries
• References
8/8/2019 SEAS2009 Query Optimization Final
http://slidepdf.com/reader/full/seas2009-query-optimization-final 5/65
5
SEAS09SEAS09
| SQL Server Customer Advisory Team
SQLCAT Introduction
• SQL Server Customer Advisory Team – Part of SQL Server development group
• SQLCAT Mission – Making customer successful
• Engage with the largest and most challenging SQLdeployments in the world, provide architecturalguidance and reviews, with direct access to productdevelopment engineers
– Improve the product• Through lessons learned, drive new features into the
product, share field experience with product team,
and participate in product planning – Improve the community
• Presentation at SQL PASS, TechEd, TechReady andmany internal and external events
• Whitepapers, Technical white papers and Blogs basedon lesson learned are published at SQLCAT website:www.sqlcat.com
• Lesson learned based training event:SEAS – SQL Server Enterprise Architect
8/8/2019 SEAS2009 Query Optimization Final
http://slidepdf.com/reader/full/seas2009-query-optimization-final 6/65
6
SEAS09SEAS09
| SQL Server Customer Advisory Team
SQLCAT Profile
• SQLCAT structure – Customer Advisory Team including
international and technologyexcellence team
– ISV team – Best practice team
• Small team with worldwide coverage – Senior database architects with many
years of SQL Server and other RDBMSexperience – Extensive customer, project management
experience – Mixed consulting services, product
development background
CustomerTeamQL
IS V
SQLCAT
QL BP
8/8/2019 SEAS2009 Query Optimization Final
http://slidepdf.com/reader/full/seas2009-query-optimization-final 7/65
7
SEAS09SEAS09
| SQL Server Customer Advisory Team
SQL Server Design-winProgram• Identify the most challenging, unique, interesting and
innovative applications running on the Microsoftdatabase platform
• Invest in Large Scale, referenceable SQL Serverprojects across the world
– SQLCAT technical & project experience
– Design & architecture review covering performance,operation, scalability, availability and many aspects
– Priority use of customer lab environment
• Use customer experiences to make a better product• Share the knowledge learned from these challenging
implementations to help the world wide SQLcommunity.
• Reference the customer and make them a case story
8/8/2019 SEAS2009 Query Optimization Final
http://slidepdf.com/reader/full/seas2009-query-optimization-final 8/65
8
SEAS09SEAS09
| SQL Server Customer Advisory Team
Engaging SQLCAT
• Engagement through Design-win program• Coverage/Contacts in your region
– US & worldwide:• Lubor Kollar – [email protected]
– Latin America
• Stuart Ozer – [email protected] – Asia
• Howard Yin (GCR & Japan) – [email protected]
• Peter Carlin (Taiwan & Japan) – [email protected]
• Prem Mehra (India & US) – [email protected]
• Nicholas Dritsas (APAC) – [email protected]
– Europe• Thomas Kejser – [email protected]
• Alexei Khalyako – [email protected]
– US• Michael Thomassy – [email protected]
• Kevin Cox – [email protected]
8/8/2019 SEAS2009 Query Optimization Final
http://slidepdf.com/reader/full/seas2009-query-optimization-final 9/65
9
SEAS09SEAS09
| SQL Server Customer Advisory Team
Agenda
• SQLCAT Introduction
• Query Optimizer Architecture
• Statistics Used by Optimizer• Query Plan Analysis
• Influencing Plan Selection
• Parameterized Queries• References
8/8/2019 SEAS2009 Query Optimization Final
http://slidepdf.com/reader/full/seas2009-query-optimization-final 10/65
Microsoft Confidential 10
SEAS09SEAS09
| SQL Server Customer Advisory Team
SQL Server Relational ServerHigh-Level Architecture
SQL Relat ional Server
Storage Engine (Access Methods, Database PageCache, Locking, Transactions, …)
SQL-OS (Schedulers, Buffer Pool, MemoryManagement, Synchronization Primitives, …)
Query Optimization(Plan Generation, IndexSelection, Join Ordering,
Statistics, Costing) Query Execution(Query Operators,Memory Grants,Parallelism)
Language Processing (Parse/Bind/View Expansion)Utilities( D
B
CC,
Backup
/Restore
,BCP,…)
M
etadat a
,Types ys
tem,
E
xpressionServ
ice
s
Statement/BatchExecution
Plan Cache Management
8/8/2019 SEAS2009 Query Optimization Final
http://slidepdf.com/reader/full/seas2009-query-optimization-final 11/65
Microsoft Confidential 11
SEAS09SEAS09
| SQL Server Customer Advisory Team
SQL Server Relational ServerHigh-Level Architecture
SQL Relat ional Server
Storage Engine (Access Methods, Database PageCache, Locking, Transactions, …)
SQL-OS (Schedulers, Buffer Pool, MemoryManagement, Synchronization Primitives, …)
Query Execution(Query Operators,Memory Grants,Parallelism)
Language Processing (Parse/Bind/View Expansion)Utilities( D
B
CC,
Backup
/Re
store
,BCP,…)
M
etadat a
,Types ys
tem,
E
xpressionServ
ice
s
Statement/BatchExecution
Plan Cache Management
Query Optimization(Plan Generation, IndexSelection, Join Ordering,Statistics, Costing)
8/8/2019 SEAS2009 Query Optimization Final
http://slidepdf.com/reader/full/seas2009-query-optimization-final 12/65
Microsoft Confidential 12
SEAS09SEAS09
| SQL Server Customer Advisory Team
Storage Engine (Access Methods, Database PageCache, Locking, Transactions, …)
SQL-OS (Schedulers, Buffer Pool, MemoryManagement, Synchronization Primitives, …)
Query Optimization(Plan Generation, IndexSelection, Join Ordering,
Statistics, Costing) Query Execution(Query Operators,Memory Grants,Parallelism)
Language Processing (Parse/Bind/View Expansion)Utilities( D
B
CC,
Backup
/Re
store
,BCP,…)
M
etadat a
,Types y
stem,
E
xpressionServ
ice
s
Statement/BatchExecution
Plan Cache Management
SQL Server Relational Server This Talk – Query Optimization
SQL Relat ional Server
Query Opt im izat ion(Plan Generat ion,Index Select ion, JoinOrdering, St at ist ics,
Costing)
8/8/2019 SEAS2009 Query Optimization Final
http://slidepdf.com/reader/full/seas2009-query-optimization-final 13/65
SEAS09SEAS09
| SQL Server Customer Advisory Team
Query Optimization
• Query Optimization is cost based inSQL Server
• Input to Query Optimization is
Operator Tree produced by – Parsing SQL statement (Syntax)
– Semantic analysis and checking (akaAlgebrizing – this includes type
derivation, type checking, bindingvariables, table and column names,etc.)
• Output is Query Plan
8/8/2019 SEAS2009 Query Optimization Final
http://slidepdf.com/reader/full/seas2009-query-optimization-final 14/65
| SQL Server Customer Advisory Team
SEAS09SEAS09
Query Optimization
parse/bindparse/bindQueryQuery
optimizer optimizer
Select *Select *From …From …
JoinJoin
UnionUnion……
…… …… ……
MetadataMetadatalIndicesIndiceslStatisticsStatisticslConstraintsConstraintslUser User
Hash joinHash join
ConcatConcat……
OutputOutput
…… …… ……
lGenerateGeneratealternativesalternatives
l
lEstimate costEstimate cost
InputInputPhysicalPhysical
PropertiesPropertieslMemoryMemorylNumber of Number of
corescores
8/8/2019 SEAS2009 Query Optimization Final
http://slidepdf.com/reader/full/seas2009-query-optimization-final 15/65
SEAS09SEAS09
| SQL Server Customer Advisory Team
Relational Expressions
• SQL is the standard front-endlanguage
• Query processing is based on
relational operator trees»
• Query blockGroupBy T.c, sum(T.a)GroupBy T.c, sum(T.a)
Filter (T.b=R.b and R.c = 5)Filter (T.b=R.b and R.c = 5)
RRTT
Cross productCross product
SELECT SUM(T.a)SELECT SUM(T.a)
FROM T, RFROM T, R
WHERE T.b = R.bWHERE T.b = R.bAND R.c = 5AND R.c = 5
GROUP BY T.cGROUP BY T.c
GroupBy T.c, sum(T.a)GroupBy T.c, sum(T.a)
Filter (R.c = 5)Filter (R.c = 5)
RR
TT
Join (T.b=R.b)Join (T.b=R.b)
8/8/2019 SEAS2009 Query Optimization Final
http://slidepdf.com/reader/full/seas2009-query-optimization-final 16/65
| SQL Server Customer Advisory Team
SEAS09SEAS09
Tree Transformations
Filter (A.x = 5)Filter (A.x = 5)
JoinJoin
BBAA
Filter (A.x = 5)Filter (A.x = 5)
JoinJoin
BB
AA
GroupBy A.x, sum(A.y)GroupBy A.x, sum(A.y)
JoinJoin
BBAA
JoinJoin
BB
AA
GroupBy A.x, sum(A.y)GroupBy A.x, sum(A.y)
JoinJoin
BBAA
Hash-JoinHash-Join
BBAA
SimplificationSimplification ImplementationImplementationExplorationExploration
8/8/2019 SEAS2009 Query Optimization Final
http://slidepdf.com/reader/full/seas2009-query-optimization-final 17/65
| SQL Server Customer Advisory Team
SEAS09SEAS09
Sets Of Transformations
• Join reordering• Outerjoins
• Subqueries
• Aggregation• Union
• Index plans
• Large IN lists
• Empty tablesimplification
• Bitmap filters
• Star and Snowflakequeries
• Join elimination
• Indexed views
• Update plans
• Constraint checking
• Parallel plans
• Remote queries• Partitioned tables
• …
8/8/2019 SEAS2009 Query Optimization Final
http://slidepdf.com/reader/full/seas2009-query-optimization-final 18/65
SEAS09SEAS09
| SQL Server Customer Advisory Team
Costing
• Based on – Estimated intermediate and final result
size
– Detailed model of execution algorithm(I/O, number of CPUs, memory)
• Estimation of result size – Histograms on distribution of values
for base table columns (tablestatistics)
– Derived on each operation, e.g. filter,
join
8/8/2019 SEAS2009 Query Optimization Final
http://slidepdf.com/reader/full/seas2009-query-optimization-final 19/65
SEAS09SEAS09
| SQL Server Customer Advisory Team
Multi-Stage Optimization
• Stages – No-choice queries (trivial plan)
– Transaction processing queries (Phase0)
– Complex query I (Phase 1)
– Complex query II (Phase 2)
• Parameters of a stage
– Entry condition – Transformations allowed
– Termination condition
8/8/2019 SEAS2009 Query Optimization Final
http://slidepdf.com/reader/full/seas2009-query-optimization-final 20/65
| SQL Server Customer Advisory Team
SEAS09SEAS09
SQL Server Cost-based QueryOptimization Process
T0T0simplifysimplify
T1T1 Pool of alternat ivesPool of alternat ives
search(0)search(0) search(1)search(1) search(2)search(2)
T2T2
(input)(input)
(output)(output)
Cost-based optimizationCost-based optimization
use simplificationuse simplification
transformationstransformations
use explorat ion anduse exp loration and
implementationimplementationt ransformations, costt ransformations, cost
alternativesalternatives
SEAS09
8/8/2019 SEAS2009 Query Optimization Final
http://slidepdf.com/reader/full/seas2009-query-optimization-final 21/65
SEAS09SEAS09
| SQL Server Customer Advisory Team
Optimization Phases
• Goal: don’t overspend on optimization – E.g. don’t spend additional 1 second of optimization
time to shave off 0.5 second of run time• Phase 1 is not entered if Phase 0 finds a plan with
cost<0.2
• Phase 2 is not entered if Phase 1 finds a plan with
cost<1.0• Each additional phase has richer transformations and
considers additional plan types – In Phase 0 only NL and Hash joins
– In Phase 1 initially only non-parallel plans areconsidered; if the best plan so far found has cost>=5, Phase 1 is repeated to find a parallel plan andthe “cheaper” of the parallel and non-parallelversions will advance to Phase 2
– Phase 2 is performing for example Outer Joinreordering, Index View matching and Star Queryoptimization
• Optimization may end up in “time-out” and the best sofar if an ! lan is sent for execution
SEAS09SEAS09
8/8/2019 SEAS2009 Query Optimization Final
http://slidepdf.com/reader/full/seas2009-query-optimization-final 22/65
| SQL Server Customer Advisory Team
SEAS09SEAS09
IBM DB2 On NT And UNIX
T0T0simplifysimplify
T1T1 Pool of alternat ivesPool of alternat ives
search(0)search(0) search(1)search(1) search(2)search(2)
T2T2
(input(input
))
(output)(output)
Cost-based optimizationCost-based optimizationQuery rewrit eQuery rewriteheuristically change blocksheuristically change blocks
bott om-up join enumerationbott om-up join enumeration
knobs to lim it searchknobs to lim it search
lNon-uniform query representationNon-uniform query representationlCost-based optimization focused on joinCost-based optimization focused on join
ØDoes not cross “ query block boundary”Does not cross “ query block boundary ”
SEAS09SEAS09
8/8/2019 SEAS2009 Query Optimization Final
http://slidepdf.com/reader/full/seas2009-query-optimization-final 23/65
SEAS09SEAS09
| SQL Server Customer Advisory Team
Oracle Architecture
• Old “rule-based optimizer” – Use heuristics, and follow SQL syntax
– No cost-estimation considered
• New cost-based optimizer – Follow IBM’s (old, System-R kind of
approach)
SEAS09SEAS09
8/8/2019 SEAS2009 Query Optimization Final
http://slidepdf.com/reader/full/seas2009-query-optimization-final 24/65
SEAS09SEAS09
| SQL Server Customer Advisory Team
SQL Server - OptimizerExtensibility
Goal: Support high paced innovation (typicalfor MS products)
• Easy way to add new operators – Logical operators, or execution algorithms
– Cardinality and cost estimation functions
• Adding new tree transformations – Handle new operators
– Add new reordering alternatives
• Apply cost-based optimization uniformly onall operations, beyond joins/SQL queryblock
SEAS09SEAS09
8/8/2019 SEAS2009 Query Optimization Final
http://slidepdf.com/reader/full/seas2009-query-optimization-final 25/65
25
SEAS09SEAS09
| SQL Server Customer Advisory Team
Tools to monitor optimization
• SQL Server 2008 has tools to see “inside”the optimization
• Showplan contains for example(will be covered in depth next)
– Operator cost and subtree cost – Cardinalities of all operators (estimated
number of rows returned by the operator) – Optimizer warnings (e.g. missing join
predicate) – Values of variables used for optimization
• SQL Trace• DMV sys.dm_exec_query_optimizer_info• Perfmon counters in SQL Server: SQL
Statistics
SEAS09SEAS09
8/8/2019 SEAS2009 Query Optimization Final
http://slidepdf.com/reader/full/seas2009-query-optimization-final 26/65
26
SEAS09SEAS09
| SQL Server Customer Advisory Team
Agenda
• SQLCAT Introduction• Query Optimizer Architecture
• Statistics Used by Optimizer
• Query Plan Analysis
• Influencing Plan Selection
• Parameterized Queries
• References
SEAS09SEAS09
8/8/2019 SEAS2009 Query Optimization Final
http://slidepdf.com/reader/full/seas2009-query-optimization-final 27/65
27
SEAS09SEAS09
| SQL Server Customer Advisory Team
What affects optimizerchoices?
• Optimizer Statistics
• Indexes and constraints
•Memory• Logical number of processors
SEAS09SEAS09
8/8/2019 SEAS2009 Query Optimization Final
http://slidepdf.com/reader/full/seas2009-query-optimization-final 28/65
Microsoft Confidential 28
SEAS09SEAS09
| SQL Server Customer Advisory Team
Optimizer Statistics
• Used for costing – Histogram• Captures distribution of values for a column• Max number of steps = 200• Used for estimating cardinality of search predicates• Single-column only
– Density
• Number of occurrences of each unique value in acolumn• Used for estimating cardinality of search predicates,
joins, agg – String index (statistics) – new in SQL Server 2005
• Viewing statistics – DBCC SHOW_STATISTICS
• CREATE / UPDATE STATISTICS – Can specify FULL SCAN or sample size or sample % – Default sample size increases logarithmically with the
number of rows – Sp_createstats, sp_updatestats
• By default SQL Server creates and maintains statistics
automatically – the Best Practice is to use thedefault
SEAS09SEAS09
8/8/2019 SEAS2009 Query Optimization Final
http://slidepdf.com/reader/full/seas2009-query-optimization-final 29/65
Microsoft Confidential 29
SEAS09SEAS09
| SQL Server Customer Advisory Team
Auto Statistics
• Database options – AUTO_CREATE_STATISTICS• Stats on columns created when a query needs
them – AUTO_UPDATE_STATISTICS – when?
• Permanent tables: 500; 500 + 20% * rows in tablewhen compiled
• Temp tables: 6; 500; 500 + 20% * rows in tablewhen compiled• See
http://www.microsoft.com/technet/prodtechnol/sql/2for more details
– Beware AUTO UPDATE uses default samplingeven if created as FULLSCAN => consequenceCREATE INDEX creates statistics with FULLSCANbut subsequent auto update for the same indexwill use sampling
– No auto-create or auto-update stats for tablevariables (and you cannot create statisticseither)
• sp_autostats – Display, change auto-stats setting (ON/OFF)
– Can disable auto-stats per index, table
SEAS09SEAS09
8/8/2019 SEAS2009 Query Optimization Final
http://slidepdf.com/reader/full/seas2009-query-optimization-final 30/65
Microsoft Confidential 30
SEAS09SEAS09
| SQL Server Customer Advisory Team
Statistics Issues
… and Best Practices
• Statistics are used to derive Cardinality Estimate (CE) on eachquery operation
– Filter, Join, GroupBy, … – Warning column in showplan if missing statistics
• Limitations – CE gets progressively less reliable across multiple joins and group-
by’s
– Hidden correlations tough to handle• Data-level correlation: Two columns that are correlated• Predicate-level correlation: Multiple predicates that are not
independent – Data skew impacts estimates, especially with sampled statistics
• Hard to correctly guess density from sample
• Recompilation if number of changes to a column exceedsthresholds
• (Re)Compilations requiring a statistic stall if auto_update stats isexecuting – Consider AUTO_UPDATE_STATISTICS_ASYNC Database option
• Best practices – Reasons to turn off AUTO_UPDATE on a statistic
• If it is causing unnecessary recompilation - lots of updates butthey don’t change nature/distribution of data
• For increased predictability during work-day if can be scheduledduring off time
– Sometimes, FULLSCAN stats necessary – schedule periodic updatestats
SEAS09SEAS09
8/8/2019 SEAS2009 Query Optimization Final
http://slidepdf.com/reader/full/seas2009-query-optimization-final 31/65
Microsoft Confidential 31
SEAS09SEAS09
| SQL Server Customer Advisory Team
Upgrade Considerations
• Upgrade from SS 2000 – All SS 2000 stats are considered
automatically “stale” => autostats willtake care of the “upgrade”
– Best Practice is to update statistics
manually as a part of the upgrade step – If stats are maintained manually they allshould be manually updated duringupgrade
• sp_updatestats @resample ='resample‘
– Details of changes are described inhttp://www.microsoft.com/technet/prodtechnol• Upgrade from SS 2005
– No need to update statistics
SEAS09SEAS09
8/8/2019 SEAS2009 Query Optimization Final
http://slidepdf.com/reader/full/seas2009-query-optimization-final 32/65
32
SEAS09SEAS09
| SQL Server Customer Advisory Team
Agenda
• SQLCAT Introduction• Query Optimizer Architecture
• Statistics Used by Optimizer
• Query Plan Analysis
• Influencing Plan Selection
• Parameterized Queries
• References
SEAS09SEAS09
8/8/2019 SEAS2009 Query Optimization Final
http://slidepdf.com/reader/full/seas2009-query-optimization-final 33/65
33
SEAS09SEAS09
| SQL Server Customer Advisory Team
Query Plan aka Showplan
• 3 forms: textual, XML , graphical• By contents: operators only, with cost and
cardinality estimates and with runtimeinformation
•
•
•
•
••
•
• SQL Trace has 8 different Showplan events in
Performance event category
Text XML Graphical
Operators SET SHOWPLAN_TEXTON
N/A N/A
Operators andestimated costsand card.
SET SHOWPLAN_ALLON
SETSHOWPLAN_XMLON
Display EstimatedExecution Plan inManagement Studio
Run time info SET STATISTICSPROFILE ON
SET STATISTICSXML ON
Include ActualExecution Plan in
Management Studio
SEAS09SEAS09
8/8/2019 SEAS2009 Query Optimization Final
http://slidepdf.com/reader/full/seas2009-query-optimization-final 34/65
34
SEAS09SEAS09
| SQL Server Customer Advisory Team
Demo – Showplan in SSMS
• Estimated Query Plan – Properties of the Index Seek
– Estimated # of rows, # of executions
• Actual Query Plan – Watch for “Actual” number of rows,
estimated number of rows, numberof executions
• SHOWPLAN_TEXT, ALL, XML
• Saving and sharing the query plans
–
SEAS09SEAS09
8/8/2019 SEAS2009 Query Optimization Final
http://slidepdf.com/reader/full/seas2009-query-optimization-final 35/65
35
SEAS09SEAS09
| SQL Server Customer Advisory Team
Actual Query Plan
SEAS09SEAS09
8/8/2019 SEAS2009 Query Optimization Final
http://slidepdf.com/reader/full/seas2009-query-optimization-final 36/65
36
SEAS09S S09
| SQL Server Customer Advisory Team
Demo – Trivial Plan in SSMS
• Trivial plan – By investigating the syntax and
metadata optimizer knows “there areno choices”
– Optimization Level = Trivial
• Non-trivial plan – Optimization Level = FULL
– if early termination happens• Reason For Early Termination has a value
– Good Enough Plan Found OR
– Timeout OR
– Memory Limit Exceeded
–
SEAS09SEAS09
8/8/2019 SEAS2009 Query Optimization Final
http://slidepdf.com/reader/full/seas2009-query-optimization-final 37/65
37
S S09
| SQL Server Customer Advisory Team
Using DMV
• sys.dm_exec_query_optimizer_infocaptures aggregate counters of various compile time events, for
example – How many compilations happened so
far?
– How many finished in which phase(0,1,2,3) of optimization?
– How many Inserts, Deletes, Updates?
– Etc.
SEAS09SEAS09
8/8/2019 SEAS2009 Query Optimization Final
http://slidepdf.com/reader/full/seas2009-query-optimization-final 38/65
38| SQL Server Customer Advisory Team
Demo - Optimizer Warnings
• SHOWPLAN shows warnings• Missing Column Statistics
• Missing Join Predicate
• SQL Trace events – Errors and Warnings Event Category
• Missing Column Statistics
• Missing Join Predicate
SEAS09SEAS09
8/8/2019 SEAS2009 Query Optimization Final
http://slidepdf.com/reader/full/seas2009-query-optimization-final 39/65
39| SQL Server Customer Advisory Team
Demo – Database TuningAdvisor (DTA)
• Input batch file or SQL trace file or table
• GUI interface surfaces only a subset of capabilities of the DTA engine
– Command line version is using XMLdocument conforming toDTASchema.xsd
– SQL Server schemas are published at
http://schemas.microsoft.com/sqlserver/
• Recommended use for smaller sets of
queries (as was the demo)
SEAS09SEAS09
8/8/2019 SEAS2009 Query Optimization Final
http://slidepdf.com/reader/full/seas2009-query-optimization-final 40/65
40| SQL Server Customer Advisory Team
Missing Indexes
• In some cases the Optimizer will recordinformation about “missing index” inShowplan and in several DMVs
– sys.dm_db_missing_index_groups – sys.dm_db_missing_index_group_stats
– sys.dm_db_missing_index_details
– sys.dm_db_missing_index_columns
• The information is opportunistic and notexhaustive
SEAS09SEAS09
8/8/2019 SEAS2009 Query Optimization Final
http://slidepdf.com/reader/full/seas2009-query-optimization-final 41/65
41| SQL Server Customer Advisory Team
Query Plan Debugging – bestpractices• Collect “Actual Execution Plan”• Compare “Estimated” and “Actual” row counts
– Large discrepancies indicate potential problemspots
• Watch for Execution Warnings – BOL describesremedies for each case – Sort Warning
• sort operations do not fit into memory
– Exchange Spill
• buffers in a parallel query plan have beentemporarily written to the tempdb database
– Hash Warning• hash recursion or cessation of hashing (hash
bailout) has occurred during a hashingoperation
– Bitmap Warning• -
SEAS09SEAS09
8/8/2019 SEAS2009 Query Optimization Final
http://slidepdf.com/reader/full/seas2009-query-optimization-final 42/65
42| SQL Server Customer Advisory Team
Agenda
• SQLCAT Introduction• Query Optimizer Architecture
• Statistics Used by Optimizer
• Query Plan Analysis
• Influencing Plan Selection
• Parameterized Queries
• References
SEAS09SEAS09
8/8/2019 SEAS2009 Query Optimization Final
http://slidepdf.com/reader/full/seas2009-query-optimization-final 43/65
Microsoft Confidential 43| SQL Server Customer Advisory Team
Hints
• Types of Query hints – Locking (PAGLOCK, ROWLOCK, NOLOCK,
HOLDLOCK…) – Isolation Level (REPEATABLEREAD,
SERIALIZABLE ,…) – Other (KEEPIDENTITY, KEEPDEFAULTS,
NOWAIT, NOEXPAND,…)
–Optimizer
• Table access• Join• Query
SEAS09SEAS09
8/8/2019 SEAS2009 Query Optimization Final
http://slidepdf.com/reader/full/seas2009-query-optimization-final 44/65
44| SQL Server Customer Advisory Team
Optimizer Hint Classification
SELECT ProductName, Products.ProductID
FROMdbo.[Order Details] WITH (INDEX(0))
INNER LOOP JOINdbo.Products
ON [Order Details].ProductID = Products.ProductID
WHERE Products.UnitPrice > 100
OPTION (RECOMPILE, LOOP JOIN)
TABLEaccessHINT
JOIN HINT
QUERYHINT
SEAS09SEAS09
8/8/2019 SEAS2009 Query Optimization Final
http://slidepdf.com/reader/full/seas2009-query-optimization-final 45/65
Microsoft Confidential 45| SQL Server Customer Advisory Team
Optimizer Hint Examples
• Table access hint: INDEX (n) | (name) – Cannot force seek versus scan until 2008
• Join order, method – FORCE ORDER query hint
• But cannot specify order across sub-queries, orpush aggregations below joins – always QUERYlevel
– LOOP | MERGE | HASH JOIN – both at the querylevel and individual join
• Aggregation – HASH GROUP => Hash Aggregation – ORDER GROUP => Stream Aggregation
• For more details see BOL – TABLE HINT section for Table access hints – Join Hints section for Join hints – OPTION clause section for Query hints
SEAS09SEAS09
8/8/2019 SEAS2009 Query Optimization Final
http://slidepdf.com/reader/full/seas2009-query-optimization-final 46/65
Microsoft Confidential 46| SQL Server Customer Advisory Team
Hints - New in 2005 and 2008
• RECOMPILE hint for individual queries• The following hints provide more control to
“fix” the plan – OPTIMIZE FOR hint to force particular values
of parameters to be used by the optimizer – USE PLAN to enforce particular query plan – PARAMETERIZATION hint
• Database level• Inside a Plan Guide
– Plan Guides• FORCESEEK – new table access hint in SQL
Server 2008
SEAS09SEAS09
8/8/2019 SEAS2009 Query Optimization Final
http://slidepdf.com/reader/full/seas2009-query-optimization-final 47/65
Microsoft Confidential 47| SQL Server Customer Advisory Team
Recompile Hint
• In SS 2000: CREATE PROCEDURE WITHRECOMPILE
• In SS 2005 and 2008, an individual
statement may have OPTION(RECOMPILE) – Better alternative to dynamic SQL when
forcing recompilation for each new set
of variables was suggested in Shiloh – Easier syntax
– No need to grant access to allreferenced objects; execute on the spsuffices
SEAS09SEAS09
8/8/2019 SEAS2009 Query Optimization Final
http://slidepdf.com/reader/full/seas2009-query-optimization-final 48/65
Microsoft Confidential 48| SQL Server Customer Advisory Team
Optimize For Hint
OPTION ( OPTIMIZE FOR( @variable_name = literal_constant [ ,…n ] ) )
• Values are used only for theoptimization, not for execution
• Counteracts parameter detection – mayachieve more predictable behavior
• Cannot be used without dataknowledge
SEAS09SEAS09
8/8/2019 SEAS2009 Query Optimization Final
http://slidepdf.com/reader/full/seas2009-query-optimization-final 49/65
Microsoft Confidential 49| SQL Server Customer Advisory Team
Use Plan Hint
• Plan Stability Feature
• Idea: Use a captured XML Showplanin a USE PLAN query hint to guide
the optimizer to choose the sameplan
SEAS09SEAS09
8/8/2019 SEAS2009 Query Optimization Final
http://slidepdf.com/reader/full/seas2009-query-optimization-final 50/65
Microsoft Confidential 50| SQL Server Customer Advisory Team
Plan Forcing
Usage:
SELECT * FROM t OPTION ( USE PLAN N‘<ShowPlanXML> …
</ShowPlanXML>’ )
Query
Query Hint
String literal
of XML Showplan
SEAS0SEAS0
99
8/8/2019 SEAS2009 Query Optimization Final
http://slidepdf.com/reader/full/seas2009-query-optimization-final 51/65
Microsoft Confidential 51
99
| SQL Server Customer Advisory
Team
What’s Forced by Plan Hint
• Plan topology and order of evaluation
• Execution algorithms
• Index solutions
• Objects referenced in the query
•
Plan Forcing is used to force costbased decisions.
SEAS0SEAS0
99
8/8/2019 SEAS2009 Query Optimization Final
http://slidepdf.com/reader/full/seas2009-query-optimization-final 52/65
Microsoft Confidential 52
99
| SQL Server Customer Advisory
Team
Plan Hint Limitations
• No INSERT/UPDATE/DELETE in 2005, YES in 2008
• No Distributed Query
• No Full-text queries
• Static, Fast_Forward cursors only
•
•
SEAS0SEAS0
99
8/8/2019 SEAS2009 Query Optimization Final
http://slidepdf.com/reader/full/seas2009-query-optimization-final 53/65
Microsoft Confidential 53
99
| SQL Server Customer Advisory
Team
Plan Guides - Idea
Original Query Hinted Version
SELECT … SELECT … OPTION (…)
UPDATE … UPDATE … OPTION (…)
Out-of-band query hinting
Scenario: query is known but cannot bemodified directly in the source code yetsome hinting is desirable
Replace the Original Query by the hintedversion using “Plan Guide” system table
SEAS0SEAS0
99Plan Guides Feature
8/8/2019 SEAS2009 Query Optimization Final
http://slidepdf.com/reader/full/seas2009-query-optimization-final 54/65
Microsoft Confidential 54
99
| SQL Server Customer Advisory
Team
Plan Guides FeatureOverview• Create: sp_create_plan_guide
• Drop/Enable/Disable:sp_control_plan_guide
• Matching: – During module (SP, DML trigger, multi-
statement TVF, scalar UDF)compilation
– During batch compilation – Targets
• original user query
• Simple- or Force-parameterized query
SEAS0SEAS0
99
S i ( ll t i t
8/8/2019 SEAS2009 Query Optimization Final
http://slidepdf.com/reader/full/seas2009-query-optimization-final 55/65
Microsoft Confidential 55
99
| SQL Server Customer Advisory
Team
Scenarios (all pertain to an appyou can’t change)
• Force use of a MERGE JOIN operation for aspecific query in a user-submitted batch
• Force optimizer to always use “worst case”parameter value
• Force compilation sharing for queries with aformat that is the same except constantliterals are different – Specify FORCED PARAMETERIZATION in plan
guide• For use of a specific plan for a manually
parameterized query
SEAS09SEAS09
New around plan guides in
8/8/2019 SEAS2009 Query Optimization Final
http://slidepdf.com/reader/full/seas2009-query-optimization-final 56/65
Microsoft Confidential 56| SQL Server Customer Advisory Team
New around plan guides in2008• Plan Freezing - ability to create a plan from the
plan cache directly – The plan guide is automatically created by
invoking sp_create_plan_guide_from_cache• See documentation for the sp for boilerplate
code• sys.fn_validate_plan_guide function validates if
the plan guide is still valid (after e.g. somemetadata changes)
• SQL Profiler trace events
"Plan_guide_successful" and"Plan_guide_unsuccessful" are located underthe "Performance" category
• Perfmon counters "Guided Plan Executions/sec"and "Misguided Plan Executions/sec“ provide
similar function as the trace events above
SEAS09SEAS09
8/8/2019 SEAS2009 Query Optimization Final
http://slidepdf.com/reader/full/seas2009-query-optimization-final 57/65
57| SQL Server Customer Advisory Team
Agenda
• SQLCAT Introduction• Query Optimizer Architecture
• Query Plan Analysis
• Influencing Plan Selection• Parameterized Queries
• References
SEAS09SEAS09
8/8/2019 SEAS2009 Query Optimization Final
http://slidepdf.com/reader/full/seas2009-query-optimization-final 58/65
Microsoft Confidential 58| SQL Server Customer Advisory Team
Parameterization
• In SQL Server 2000 “auto parameterization” isused only for very simple queries – Referred to as SIMPLE parameterization in SS
2005
•Discoverable by investigating the plan;“parameter” is suddenly there. For example
Query: * =select from T where c 5Plan: -- ( :([ ].[ ].[ ]), :| Table Scan OBJECT tst1 dbo T WHERE
([ ].[ ]=T C Convert([ ])@1 ))
• Advantage: no need to recompile for thesame query with different value instead of “5”
•
SEAS09SEAS09
8/8/2019 SEAS2009 Query Optimization Final
http://slidepdf.com/reader/full/seas2009-query-optimization-final 59/65
Microsoft Confidential 59| SQL Server Customer Advisory Team
Parameterization (contd.)
• In SQL Server 2005 “parameterization”may be forced on a database level forALL queries – Set PARAMETERIZATION option to
FORCED in ALTER DATABASE stmt• Use only when you are sure compilation
is the bottleneck• Some queries may get worse plans if
forcefully parameterized• Works only against databases in 90 and
higher compatibility level
SEAS09SEAS09
8/8/2019 SEAS2009 Query Optimization Final
http://slidepdf.com/reader/full/seas2009-query-optimization-final 60/65
Microsoft Confidential 60| SQL Server Customer Advisory Team
Parameterization (Example)
Query: * = =select from T where c 1 or c 2Plan: -- ( :([ ].[ ].[ ]), :([ ].[ ].[ ].| Table Scan OBJECT tt dbo T WHERE tt dbo T
[ ]=c ( )1 [ ].[ ].[ ].[ ]=OR tt dbo T c ( )2 ))
ALTER DATABASE <dbname> SETPARAMETERIZATION FORCED
Plan: -- ( :([ ].[ ].[ ]), :([ ].[ ].| Table Scan OBJECT tt dbo T WHERE tt dbo[ ].[ ]=T c [ ]@0 [ ].[ ].[ ].[ ]=OR tt dbo T c [ ]@1 ))
ALTER DATABASE <dbname> SETPARAMETERIZATION SIMPLE
SEAS09SEAS09
8/8/2019 SEAS2009 Query Optimization Final
http://slidepdf.com/reader/full/seas2009-query-optimization-final 61/65
61| SQL Server Customer Advisory Team
Agenda
• SQLCAT Introduction• Query Optimizer Architecture
• Query Plan Analysis
• Influencing Plan Selection• Parameterized Queries
• References
SEAS09SEAS09
8/8/2019 SEAS2009 Query Optimization Final
http://slidepdf.com/reader/full/seas2009-query-optimization-final 62/65
62| SQL Server Customer Advisory Team
Optimizer References
• Tips, Tricks, and Advice from the SQLServer Query Processing Teamhttp://blogs.msdn.com/sqlqueryprocessin
• Query Tuning Recommendationshttp://technet.microsoft.com/en-us/librar
• Data Warehouse Query Performancehttp://technet.microsoft.com/en-us/maga
8/8/2019 SEAS2009 Query Optimization Final
http://slidepdf.com/reader/full/seas2009-query-optimization-final 63/65
Microsoft Confidential 63
Some showplan references
• Find Top N costly query plans in adhocbatches ormodules...http://blogs.msdn.com/sqltips/archive/2005/10/05/T
• Troubleshooting Performance Problems in SQLServer 2005http://www.microsoft.com/technet/prodtechnol/sql/2
• Processing XML Showplans Using SQLCLR in SQL
Server 2005http://msdn.microsoft.com/library/default.asp?url=/l
SEAS09SEAS09
8/8/2019 SEAS2009 Query Optimization Final
http://slidepdf.com/reader/full/seas2009-query-optimization-final 64/65
64| SQL Server Customer Advisory Team
QQ AA&&
8/8/2019 SEAS2009 Query Optimization Final
http://slidepdf.com/reader/full/seas2009-query-optimization-final 65/65
© 2008 Microsoft Corporation All rights reserved Microsoft Windows Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U S and/or other countries The