Embed Size (px)
Citation preview
Magnificent Database Management
www.rocksolidsql.com
About Me
Martin Catherall - MVP (SQL SERVER)Senior Database Consultant RockSolid SQL
(Melbourne)PASS Regional Mentor (Asia Pacific)
Christchurch SQL SERVER User Group LeadSoftware Developer(1998-2008) C++ / C#
Twitter: @MartyCatherall
Email: [email protected]
Blog: MartinCatherall.com
Introducing RockSolid SQL
The RockSolid PlatformCutting edge operational management and automation software for enterprise clients in banking, FSI, trading, government, retail, mining and manufacturing. ~60,000 SQL Server databases managed by our software
SQL Server Managed ServicesSLA driven monitoring, response, resolution, operational management and operational maintenance. Over 21,000 SQL Server databases managed 24x7 by our SQL Server DBA team
Mission Critical Performance
Performance Security Availability Scalability
Operational analytics
• Insights on operational data• Works with in-memory OLTP
and disk-based OLTP
In-memory OLTP for more applications
Query store
Monitor and optimizequery plans
Native JSON
Expanded support for un-structured JSON data
Temporal database support
Always Encrypted
Sensitive data remains encrypted at all times with ability to query
Row level security
Apply fine-grained access control to table rows
Dynamic Data Masking
Return masked data to non-privileged users (e.g. credit card numbers)
Other enhancements• Audit success/failure of
database operations• TDE support for storage of
In-memory OLTP Tables• Enhanced auditing for OLTP
with ability to track history of record changes
Enhanced AlwaysOn
• 3 synchronous replicas for auto failover across domains
• Round robin load balancing of replicas
• Automatic failover based on DB health
• DTC for transactional integrity across database instances with AlwaysOn
• Support for SSIS with AlwaysOn
Enhanced online operations
Enhanced database caching
Cache data with automatic, multiple TempDB files per instance in multi-core environments
Support for Windows Server v-next
Azure SQL Database
Why look at Azure SQL Database?
Useful, even if you are not planning to adopt ‘The Cloud’ Cloud first strategy from Microsoft.
• New features find their way here first. • Most new features already there – although in preview.• New features go into preview and the General Availability (GA)• New features only tend to get into the ‘box product’ on major release cycles.
The Microsoft Cloud has lots of emerging data technologies.• Blob storage• DocumentDB (JSON document storage)• Datalake.• Azure data warehouse.• ….more…..
SQL Server Management Studio – SSMS
Now on a monthly release cycle. In CTP at the moment – will likely RTM along with SQL Server 2016. After RTM SSMS will continue to get regular monthly updates. Check version from right within the IDE
• Update seamlessly Some corrections need to be made around dialogs in high resolution.
• Work around using TSQL for the moment. Intelli-sence - still being worked on.
Tempdb Enhancements
Set number of datafiles at install time (picks up no of processors)
Trace Flag 1117 and 1118 – on by default - This may ONLY apply to tempdb. Also
Key Issues
Complex Implementation Requires two Servers (CapEx
and OpEx) Data Latency in Analytics More businesses
demand/require real-time Analytics
https://www.youtube.com/watch?v=pED0y-P5afE
Operational Analytics - Traditional(ish) Architecture
SQL Server
Database
Application Tier
Presentation Layer
IIS Server
SQL ServerRelational DW
Database
ETL
BI and analytics
SQL ServerAnalysis Server
Hourly, Daily, Weekly
Operational Analytics - Minimizing Data Latency for Analytics
Benefits No Data Latency No ETL No Separate DW
Challenges Analytics queries are resource
intensive and can cause blocking How to minimize Impact on
Operational workload Sub-optimal execution of
Analytics on relational schema
SQL Server
Database
Application Tier
Presentation Layer
IIS Server
BI and analytics
Add analytics specific indexes
This is OPERATIONAL ANALYTICS
SQL ServerAnalysis Server
Operational Analytics - Columnstore
SQL 2012 feature• NCCI – none updateable – columnstore over rowstore (primary index)• Batch execution mode (limited)
SQL 2014 (~10X compression)• CCI – updateable (great for scanning a large amount of rows)• Delete bitmap.• Delta store
SQL 2016• NCCI – now updateable• Build a B-Tree over the top. (enforce constraints – PK / FK)• B-Tree is great for a small targeted queries. (efficient single row access)• Optimizer will choose which index to use.
https://www.youtube.com/watch?v=QIqZSuERvQ0 Also PASS summit 2015 session (Sunil Agarwal) Question
• How do you see this feature progressing?• Do you currently use it?• Do you plan to use it?
Operational Analytics – In-memory OLTP
SQL 2014 (targeted at specific customers)• Durable Tables <= 256 GB• Garbage collection of unused rows (different concurrency model)• More space taken that actual data Rows marked as logically deleted.• Need an in-memory filegroup.
SQL Server 2016 (getting more useful – more syntax supported.)• Data types (more support – nearly full compatibility)• Durable Tables <= 2 TB• Storage management decoupled from filestream.• De-referencing from Transaction log now faster• Garbage collection more aggressive.• TDE for in-memory.• More online operations supported• Collation restrictions.
https://www.youtube.com/watch?v=jEYrISUfTcw (2014) https://www.youtube.com/watch?v=CikrAKCXFWw (2016) Question
• How do you see this feature progressing?• Do you currently use it?• Do you plan to use it?
TSQL Enhancements
Truncate table – now does partitions (no need to swap out anymore.. unless you want to) DROP object IF EXISTS (column and constraint) Introduction of Temporal tables
• Tended to be done with triggers … now it’s part of the product.• Declare table as Temporal (need additional datetime2 cols)• Create a background ‘History’ of deletes / inserts / updates.• See the state of the table• ….at any point in time• ….over a date range.• Provide some protection from user deletes (or at least a method to reverse without
restoring) https://
www.youtube.com/watch?v=91lO8Pp8Lk4&index=3&list=PLiMfGbYSAUNs-HgxRmcju1SGa8NWTXbhH
JSON • Unable to ‘shred’ JSON like XML.• JSON_VALUE / OPENJSON• USE CHECK• https://
www.youtube.com/watch?v=91lO8Pp8Lk4&index=3&list=PLiMfGbYSAUNs-HgxRmcju1SGa8NWTXbhH
Query Store
Flight data recorder for your queries. Provides DBA’s with additional insight into plan choice and performance. Been in preview in Azure SQL Database for a while. Aggregate statistics of plans
• Estimated plans. Aggregated every 15 mins or so. Helps DBA’s spot plan regressions.
• After SQL updates (SP’s ProvCU’s, version updates) ALTER DATABASE …….. SET QUERY_STORE ON.
Security – Always Encrypted
Always Encrypted – implemented in ADO.NET stack. Client side technology. (server does not know keys – maintains some meta data) Certificate lives on app side. (key management needed-Azure Key Vault) App needs to get certificate from AKV. Certificate exchange on first request. Two types of encryption
• Deterministic (joins work here)• Randomised (different value every time)
Encryption in-memory (over the wire) Decrypted ‘on the client’
• One time handshake. TDE – only encrypts ‘at rest’ (as well as securing backups) https://www.youtube.com/watch?v=BF7fm-U8gsI
Security – Row Level Security.
Row level security (RLS) Fine grained access control. Works via a function and a policy (that YOU write) Traditionally developers have built this in app. But….when access is not through the app then its useless. RLS works at query time – no app changes. Resides on the table (schema bound to table)
Security – Dynamic Data Masking
Dynamic Data Masking (DDM) Obfuscates sensitive data.
• Masked via common functions (or write your own) Masking down after query performed. Privilege user get ‘real data’ Non privileged users get masked data. Question?
• Prod data in dev and use a masking function?
High Availability – AlwaysON
Availability Groups.• 3 synchronous replicas for auto failover.• Round robin load balancing of replicas.• Automatic failover based on database health. (previously based on instance)• Set this at AG level.• SSIS support• Basic Availability Groups (BAGs) -
http://www.brentozar.com/archive/2015/06/how-to-set-up-standard-edition-alwayson-availability-groups-in-sql-server-2016/
https://www.youtube.com/watch?v=4KJsKw3-wk4
Availability – Stretch database
Seamless partitioning (cold data pushed off to azure) By predicate (eg by date) Works via a linked server. On premises db to Azure SQL database (PaaS offering) Wizard to help set this up. https://www.youtube.com/watch?v=YyXOqE9Iips https://
www.youtube.com/watch?v=zeu740z4k20&index=28&list=PLiMfGbYSAUNs-HgxRmcju1SGa8NWTXbhH
Other features
R (analytics) Polybase Compression level increased for backups. Backup encryption supported with compression. Various wizards (eg Memory Optimization Advisor) SELECT INTO (now a parallel operation) Faster and more frequent statistics updates. Online database operations – ALTER / TRUNCATE none blocking. Support for Resilient File System (ReFS) – windows enhancement. https://www.youtube.com/watch?v=Qw2F3x82tmc http://www.techbrothersit.com/2015/06/whats-new-in-sql-server-2016.html
Magnificent Database Management
Magnificent Database Management
Thank You
