TOP 10 Thinks you shouldn’t do with/in your databaseMargarita NaumovaMicrosoft

30 March 2010

2

Top 10 … on focus today

− Memory Configuration Worst Practices− TempDB configuration worst practices− TempDB Usage worst practices

3

MaxServerMemory Configuration Mistakes

− Set MaxServerMemory to physical Maximum

− Do not set MaxServerMemory At AllAll threads/processes outside SQL Server BufferPool are slow or cannot start at all:

1. Not enough buffers for SSIS Packages, releasing buffer waits2. Linked Server and distributed query issues performance or

execution errors3. Connection/SQL Server Login Time-Outs (Errors 70x)4. Error "WARNING: Failed to reserve contiguous memory

of Size= 65536."

Results:

4

The way it works

Operating System

0xFFFFFFFF

0x00000000

0x80000000

0xC0000000

SQ

L Serv

er Buffer Pool

(8KB buffers)

SQL Server or OS(/3GB switch)

MemToLeave area

Thread stacks

Operating System

5

The way it works

Operating System

0xFFFFFFFF

0x00000000

0x80000000

0xC0000000

SQL Server or OS(/3GB switch)

SQ

L Serv

er

MemToLeave area

Thread stacks

Oth

er

Locks

Qu

ery

Work

sp

ace.

Pla

n C

ach

e

DB

Pag

e C

ach

e

MaxServerMemory

6

Recommendations

− Always define MaxServerMemory!− Take into account MemToLeave area− MemToLeave = MemReserved + (NumThreads* StackSize)

− MemReserved = 256 MB− NumThreads = the total number of worker threads configured (table below)− StackSize = (0.5MB x86, 2MB x64, 4MB IA64)

Example 8Cores, 64Bit: MemToLeave=256+576*2=1,408MB

7

Recommendations

− Calculate worst case scenario− Scenario Examle: 64bit, 8 cores,

32GB RAM− 2GB for the OS− 2 GB for MemToLeave (Linked Srvs,

MultiPage Allocators AND worker threads)− 1GB for backup program− = 5GB− Set Max Server Memory to 27GB

− Special Attention to MultiInstance Cluster Configuration!

8

-3GB/AWE configuration mistakes− Set /3G when there are other

Applications on the server− /AWE and large buffer flushes

The SQL Server needs more memory because it takes all and still works slowly. I set /3G and I set AWE switch but:

1. The OS is hanging, the overall Server is hanging2. I still do not get performance benefit

Results:

9

The way it works - /3GB

Operating System

0xFFFFFFFF

0x00000000

0x80000000

0xC0000000

SQL Server or OS(/3GB switch)

SQ

L Serv

er

MemToLeave area

Thread stacks

Oth

er

Locks

Qu

ery

Work

sp

ace.

Pla

n C

ach

e

DB

Pag

e C

ach

e

10

The way it works - AWE0xFFFFFFFF

0x00000000

0x80000000

0xC0000000

Operating System

SQL Server or OS(/3GB switch)

SQ

L Serv

er

MemToLeave area

Thread stacks

Oth

er

Locks

Qu

ery

Work

sp

ace.

Pla

n C

ach

e

DB

Pag

e C

ach

e

AWE Memory

11

Recommendations

− Use /3GB only in 32bit platforms and only when you have not database page buffer related memory issues

− Use AWE on 32-bit to increase DB Pages BufferPool Size if you have intensive BPool Page Usage

− Watch out Memory\Free System Page Table Entries counter (should be >10 000)

− /USERVA switch in Windows Server 2003− Gives less than 3GB to user mode− Extra space kept in reserve for System PTEs− Recommended values between 2900 and 3030

− Do not use /PAE with /3GB (it increases PTE from 4K to 8K)

12

The LockPagesInMemory Mistakes

− LockPagesInMemory Not set

• The performance of SQL Server decreases suddenly. • A computer that is running SQL Server stops responding for a

short time. • A time-out occurs for applications that connect to SQL Server. • Problems occur when you run even simple commands or use

applications on the system

• New message from 2005 SP2:

“A significant part of sql server process memory has been paged out. This may result in a performance degradation. Duration: 646 seconds. Working set (KB): 901904, committed (KB): 2215752, memory utilization: 40%.”

13

The way it works - LockPagesInMemory− Prevent OS paging of Buffer Pool by Locking Pages

in Memory− Automatic if SQL Server Service Account has

Lock Pages in Memory privilege− Enterprise and Standard Edition Feature− Set using gpedit.msc: http

://msdn2.microsoft.com/en-us/library/aa198229(SQL.80).aspx

− Note: Other parts of SQL may still be paged, along with other applications

14

Recommendations

− On 64bit configuration always set LockPagesInMemory

− Use with MaxServerMemory− SQL Server 2008 SP1 CU2 and SQL Server

2005 SP3 CU 4 for Standard Editions− For the LocalSystem account it is set

automatically

15

DEMO: Get the Memory Statistics

16

The TempDB configuration mistakesSlow Execution of proceduresSlow execution on large queriesDisruption/Prevent procedure from completion1101- insufficient space for session1105 – insufficient space for session3959 – version store is full3967 – version store forced to shrink 3958 – transaction cannot find required version record… • Too much or too less (1) TempDB Database File

• Confuse TempDB Performance and contention

• Extreme TempDB usage for large temp objects, and object generating volumes

• TempDB Size left default

• Temp Tables v/s temp variables v/s … huge user temp objects

Usage Wrong DB File Sizing

Contention Wrong Number of DB Files

Placement Wrong Drive

Issues:

17

Contention defined

− On TempTables Creation

PFS SGAM

tempdb

SGAM

4 GB

latching

18

DEMO: TempDB Contention

User Objects

− User-defined tables and indexes− System tables and indexes− Global temporary tables and indexes− Local temporary tables and indexes− Table variables− Tables returned in table-valued

functions− Scoped to session or stored procedure

Temp Tables v/s temp Variables− Common myth that table variables are

memory-based only− Both end up hitting tempdb− Temp Tables

− Can be indexed− Can have statistics− Colmodctr can cause recompiles

− Table Variables− No indexes or statistics− No recompile threshold

Internal Objects

− Not logged

− Work tables for cursor operations

− Work tables for spool operations

− Temporary large object storage

− Work files for hash join or hash aggregate operations

− Intermediate sort results for creating/rebuilding indexes (if SORT_IN_TEMPDB) or some GROUP BY, ORDER BY or UNION queries

22

− Reason: large sort operation within a query (ORDER BY)

− Sort Operation cannot be done into memory and has to be divided into more steps

− Sort takes multiple passes to TempDB to get the sorted output

Sort Warnings

23

Hash Warnings

− Hash Occursion/hash bailout has occurred during hash operation

− Hash Join or Hash aggregate has run out of memory and been force to split to disk during execution

− Slower query perf and space consumtion increase in TempDB

24

DEMO: TempTables/TempVariables

25

Recommendations

− Minimize use of temporary tables− Avoid Static Cursors usage− Watch out usage of LOB, − Index to avoid worktables/workfiles− Set the size appropriately for production

− Enable autogrow− Use Instant File Initialization− Set Number of files to ½ to ¼ CPUs but <=8− Set equal file sizes

− Place separately− Separate drive for Files and Log− Fast IOPS drive

26

Recommendations (cont)

− Minimize large sort operations trough:− Reducing the sort inputs− Reducing the fields in ORDER BY clause− Creating appropriate index

− Minimize Hash Joins− Make sure that statistics exist on the

columns that are involved in the hashing operation

− Try updating them− Try using different type of JOIN (nested

loop, merge)

27

Top 10 thinks you should do with/in your database (server)Session Summary1. Don’t miss MaxServerMemory2. Don’t use /3GB and /AWE for 32bit platforms just

because you guess that SQL server needs more Memory

3. Don’t miss the LockPagesInMemory on 64bit platform

4. Don’t confuse TempDB performance with contention

5. Don’t use too much/few TempDB Data Files6. Don’t leave TempDB File Size default. Size properly7. Don’t just use TempTables or TempVariables.

Choose properly8. Don’t use Static Cursors9. Don’t do large ORDER BY10.Avoid Large Inputs, groupings causing hash

operations

28

Thank you!Contact me further:

31 March 2010, 16:15 – 17:15, SQL Server Unplugged

Or any time at:e-mail: Margarita.Naumova@microsoft.comMagi_Naumova@hotmail.com

Please Complete The Evaluations! Your Feedback and comments COUNT!

29

Community Booth

− Become part of our technological communities

− Network with colleagues and experts

− Ask the Experts sessions

Exhibition AreaLevel -1

Photo (cc-by-sa) Chris Radcliff @ Flickr