Stephen Archbold

Getting the most from your SAN File and Filegroup design patterns

1Working with SQL Server for 6+ yearsFormer Production DBA for 24/7 High volume operationCurrently SQL Server consultant at ProdataSpecialising in Performance Tuning and ConsolidationBlog at http://blogs.Prodata.ie and http://simplesql.blogspot.comGet me on twitter @StephenArchbold

About me

Data Filegroup/File FundamentalsStorage Design PatternsOLTPData Warehousing Fast Track styleData Warehousing on a SANWhat other go faster buttons have we gotCase Study The unruly fact tableHow do we make the changes

AgendaFile and Filegroup recommendations were always for maintenance, mantra of Dont partition for performance Not so much anymore3General Filegroup Recommended Practices. Separate for:Nothing but system tables in PrimaryI/O patternsDifferent volatilityData Age

If using Multiple Files in a FilegroupFiles must be equally sizedFiles must be equally fullSQL does not redistribute data when adding more files

Filegroup/File FundamentalsProportional fill/Round Robin4Transactional processing is all about speedYou want to get the transaction recorded and the user out as quick as possibleMetric for throughput becomes less about MB/Sec, and more about transactions and I/Os per second

Pattern 1 - OLTPSolid State Disk becoming more commonplaceThese thrive on Random I/OAs the databases can be small, file/filegroup layout can suffer Faster disk brings different challenges

Challenges of OLTPWere not covering log writes becoming the bottleneck in this session6File2.NDFPRIMARYFilegroupFileMyDB.MDFFile1.NDF

TransactionsReferenceRef.NDFVolatileVolatile.NDF

PAGELATCH!Discuss why you separate here, even though maintainability (Piecemeal) would not be usedNeatness, and only have to move hot I/O section to faster disk7Facts and FiguresTest ConditionsData files on Solid State DiskGUID Clustered index to generate random I/O16 threads inserting at onceFull test harness in useful links slide

8Behind the scenesWait Type%SOS_SCHEDULER_YIELD 55 PAGEIOLATCH_EX 17 PAGELATCH_SH 15 ASYNC_IO_COMPLETION 5 PAGELATCH_UP 5 Wait Type%SOS_SCHEDULER_YIELD 66 PAGEIOLATCH_EX 12 PAGELATCH_SH 10 ASYNC_IO_COMPLETION 7 SLEEP_BPOOL_FLUSH 2 Single FileTwo FilesPage Latch contention with lower amount of files - PFS9Resolving in memory contention lies with the file layoutThis is actually nothing new, TempDB has been tuned this way for years!Keep in mind, files are written to in a round robin fashionWhat can we take away from this?Data WarehousingLarge VolumeStar SchemaNeed to optimize for sequential throughputScanning Entire TableNot Shared Storage

Pattern 2 Fast Track ScenarioFilegroupFile / LUNLarge Partitioned Fact TableEnclosure 1MyFact_part1.NDFMyFact_part2.NDFController 1HBA 1Partition 1Partition 2Enclosure 2MyFact_part5.NDFMyFact_part6.NDFMyFact_Part7.NDFMyfact_Part8.NDFController 1Controller 2HBA 2Partition 5Partition 6Partition 7Partition 8CPUCPUCPUCPUCPUCPUMyFact_Part3.NDFMyfact_Part4.NDFController 2Partition 3Partition 4CPUCPUEnclosureMyFact_part9.NDFMyFact_part10.NDFMyFact_Part11.NDFMyfact_Part12.NDFController 1Controller 2HBA 3Partition 9Partition 10Partition 11Partition 12CPUCPUCPUCPUProsEasy to figure out your needsSimple, cheap and fastIn depth guidance available from MicrosoftConsNot recommended for pinpoint queriesOnly really for processing entire data setsNeed VERY understanding Infrastructure team Fast Track Pros and ConsLarge VolumeStar SchemaCannot optimize for sequential throughputShared StorageMore mixed workload

Pattern 3 Datawarehouse on SANWe need Read AheadEnterprise edition is capable of issuing a request for 512KB on a single read ahead request (Standard youre stuck at 64K)It can issue several of these (outstanding I/O) at a time, up to 4MB

But you may not even be close to 512KBGoal Large Request SizeLarger sized IOs when data is in logical orderWill issue regardless of where data is, but if you get sequential pages, you get more bang for your buck!

16Run something like:

And watch this guy:How close are you to the 512k Nirvana

Mention BCP not being suitable as writing to destination can slow requests/generate async_network_io17#1 killer of read aheadRead ahead size will be reduced if pages being requested arent in logical orderBeing a diligent type, you rebuild your indexesBecause SQL is awesome, it does this using parallelism!So whats the catch?If Read Ahead is your goal, MAXDOP 1 to rebuild your indexes!Fragmentation - Party Foul ChampionEnclosure 1PRIMARY

MyDB.MDFFilegroupFile / LUNPartition 1Partition 2Partition 3Partition 4Partition 5Partition 6Partition 7Partition 8Enclosure 1FilegroupFile / LUNMyDB.MDFPrimary

Dimensions

Volatile

Dimensions.NDFStaging.NDFFacts.NDFFacts

Fact.NDFLarge Fact

Partition1.NDFPartition2.NDFPartition3.NDFPartition4.NDFPartition5.NDFPartition6.NDFPartition7.NDFPartition8.NDFHow does Analysis Services pull in data?

Getting data out of your Data Warehouse for Analysis Services

While its a single operation to process a cube, Analysis Services spins up several threadsNow youve moved from a table scan which is single threaded, to multiple range scansLook familiar?

21On read heavy workloads and Enterprise Edition, CompressionIf storing multiple Tables in a Filegroup: -E For Data Warehouses - This allocates 64 extents (4MB) per table, per file, rather than the standard 1 (64K)If using multiple Files in a Filegroup-T1117 For all - This ensures that if auto growth occurs on one file, it occurs on all others. Ensures round robin remains in placeIn General on dedicated SQL serversEvaluate -T834 Requires Lock Pages in memory enabledThis enables large page allocations for the Buffer Pool (2Mb 16Mb)Can cause problems if memory is fragmented by other apps

Do we have any go faster buttons? 834 - Buffer pool allocations are expensive, this reduces the number of themCan cause problems if server is not dedicated or doesnt have max memory set, as it needs to allocate the larger amount and will fail if it cant get it or if memory is fragmented223 TeraByte Data WarehouseTable scan was topping out at 300 mb/secStorage was capable of 1.7 GB/secTable partitioning was in placeAll tables were in a single FilegroupHad to get creative on enhancing throughputCase Study The Unruly Fact Table16 core server, Hyper Threaded to 32 cores128 GB of MemorySQLIO fully sequential, storage gives 2.2 GB/Sec32 range scans started up to simulate workloadPage compression was enabled, the T834 trace flag was enabledMAXDOP of 1 on the server to ensure # of threads were controlled

Test ConditionsFacts and FiguresNot close to peak as it didnt drive outstanding I/O high enoughAll files on a single disk partition25Other metricsScenarioTime SecsAvg IO (K)Avg MB/SecMax MB/SecBaseline 70181575622Post Index Rebuild with MAXDOP 168181615668Single FG for large table(s)565119421196Multiple FG one per partition425051,1501,281Make sure to flag double the throughput didnt half the time Partition size skewHash partitioning can resolve - #of partitions ideally should be equal to # of cores as only one thread can run per scheduler (Rest go waiting)Table scan is single threaded, partition scan is single threaded, but multiple partition scans become parallel scans

26Thankfully easy - Index rebuilds!For non partitioned tables, drop and re-create on the new Filegroup

For partitioned tables Alter the partition scheme to point to the new FileGroupFor heaps, create a Clustered Index on the table on the new filegroup, then drop it!How do we make the changes

File and Filegroup considerations can yield huge gainsKnow your workload and optimise for itIf you have a Hybrid workload, then have a Hybrid architecture!Dont neglect your SQL SettingsCode changes and indexes arent the only way to save the day!

SummaryAnd gains across the board rather than pinpoint problemsHelp resolve in memory bottlenecks28Paul Randal Multi file/filegroup testing on Fusion IOhttp://www.sqlskills.com/blogs/paul/benchmarking-multiple-data-files-on-ssds-plus-the-latest-fusion-io-driver/Fast Track Configuration Guidehttp://msdn.microsoft.com/en-us/library/gg605238.aspxResolving Latch contentionhttp://www.microsoft.com/en-us/download/details.aspx?id=26665Maximizing Table Scan Speed on SQL 2008 R2http://henkvandervalk.com/maximizing-sql-server-2008-r2-table-scan-speed-from-dsi-solid-state-storageSpecifying storage requirements (Find that sweet spot!)http://blogs.prodata.ie/post/How-to-Specify-SQL-Storage-Requirements-to-your-SAN-Dude.aspxFragmentation in Data Warehouseshttp://sqlbits.com/Sessions/Event9/The_Art_of_War-Fast_Track_Data_Warehouse_and_FragmentationPartial Database Availability and Piecemeal restoreshttp://technet.microsoft.com/en-US/sqlserver/gg545009.aspxUseful linksThank you!