Exploring sql server 2016

SQL Server 2016Exploring New features in Database Engine

Pre

sen

ter

Info 1982 I started working with computers

1988 I started my professional career in computers industry.

1996 I started working with SQL Server 6.0

1998 I earned my first certification at Microsoft as Microsoft

Certified Solution Developer (3rd in Greece)

I started my career as Microsoft Certified Trainer (MCT)

with more than 25.000 hours of training until now!

2010 I became for first time Microsoft MVP on SQL Server

I created the SQL School Greece www.sqlschool.gr

2012 I became MCT Regional Lead by Microsoft Learning

Program.

2013 I was certified as MCSE : Data Platform & Business

Intelligence

Antonios ChatzipavlisDatabase Architect,

SQL Server Evangelist

MCT, MCSE, MCITP, MCPD, MCSD, MCDBA, MCSA, MCTS,

MCAD, MCP, OCA, ITIL-F

http://www.sqlschool.gr/

SQ

Lsch

ool.g

rTe

am

Antonios ChatzipavlisSQL Server Evangelist • Trainer

Vassilis IoannidisSQL Server Expert • Trainer

Fivi PanopoulouSystem Engineer • Speaker

Sotiris KarrasSystem Engineer • Speaker

Follo

w u

sin

soci

al m

edia

@antoniosch / @sqlschool

fb/sqlschoolgr

yt/c/SqlschoolGr

SQL School Greece group

Help

needed?

[email protected]

Community News

The Conference for Technical Data Professionals

• 200+ technical sessions

• New and expert industry speakers

• Networking opportunities with thousands of attendees

from around the world

Use Local Chapter Discount Code: LC15CPJ8 for $150 off*

October 25-28

Seattle

*Cannot be applied retroactively or combined with other offers.

Join

and L

earn

Sta

y In

volv

ed

Sign up for a free membership today at sqlpass.org.

Linked In: http://www.sqlpass.org/linkedin

Facebook: http://www.sqlpass.org/facebook

Twitter: @SQLPASS

PASS: http://www.sqlpass.org

http://www.sqlpass.org/linkedin

http://www.sqlpass.org/facebook

http://www.sqlpass.org/

Pre

senta

tion

Conte

nt Microsoft Data Platform

Why SQL Server 2016

SQL Server 2016 Performance

Exploring New Features

Microsoft Data Platform

Evolving data insights and the data platform




146.03K145.84K145.96K146.06K 40.08K38.84K39.99K40.33K



Why SQL Server 2016?

Everything built-in Industry leading TCO Introducing SQL Server 2016 Core Database Platform Enterprise Business Intelligence Cloud Database Solutions SQL Server 2016 Editions SQL Server 2016 Components SQL Server 2016 Compared to Azure SQL Database Upgrading from Previous Versions SQL Server Management Studio Enhancements Enhanced Azure Support

SQL Server 2016: Everything built-in

The above graphics were published by Gartner, Inc. as part of a larger research document and should be evaluated in the context of the entire document. The Gartner document is available upon request from Microsoft. Gartner does not endorse any vendor, product or service depicted in its research

publications, and does not advise technology users to select only those vendors with the highest ratings or other designation. Gartner research publications consist of the opinions of Gartner's research organization and should not be construed as statements of fact. Gartner disclaims all warranties,

expressed or implied, with respect to this research, including any warranties of merchantability or fitness for a particular purpose.

Consistent experience from on-premises to cloud

Microsoft Tableau Oracle

$120

$480

$2,230

Self-service BI per user

In-memory across all workloads

built-inbuilt-in built-in built-in built-in

at massive scale

0 14

0 03

34

29

22

15

5

22

6

43

20

69

18

49

3

0

10

20

30

40

50

60

70

80

2010 2011 2012 2013 2014 2015

SQL Server Oracle MySQL SAP HANA TPC-H non-clustered 10TB

Oracle is #5#2

SQL Server

#1

SQL Server

#3

SQL Server

National Institute of Standards and Technology Comprehensive Vulnerability Database update 10/2015

Ind

ust

ry le

ad

ing

TC

O

BUSINESS INTELLIGENCE

ADVANCED ANALYTICS

DATA WAREHOUSING

OLTP

$648K

+ $120per user for Power BI

SQL Server 2016

ADVANCED ANALYTICS

BUSINESS INTELLIGENCE

OLTP

DATA WAREHOUSING

3.4x

20xper user

$2.2M

+ $2,230 per user for BI

Note: For OLTP and DW scenario, the price comparison

is based on a server with 2 proc, 8 cores each

Built-in with SQL Server vs. expensive add-ons with Oracle

• Complete mobile BI

• In-memory

• End-to-end security

• Advanced Analytics

built-in

built-in

built-in

built-in

•Enterprise Data Platform• Scalability and Performance

• High Availability

• Data Recoverability

•Business Intelligence• Self-Service BI

• Reduced IT workload

• Big Data

•Cloud Infrastructure Solutions• Public cloud

• Private cloud

• Hybrid cloud

Introducing SQL Server 2016

• Scalability and Performance

• Need to support thousands of simultaneous users with near-instantaneous response

• Database layer of the application must be able to optimize its use of OS and hardware resources to provide

concurrency without compromising data integrity

• High Availability

• Business users need continuous access to applications and their data:

• From PCs in the office

• A variety of devices outside of traditional business hours

• From anywhere in the world

• Increased requirement for high availability solutions

• Data Recoverability

• Business information is critical to an organization and they can tolerate little or no data loss

• A data recovery solution must:

• Minimize the time taken to restore data availability

• Ensure that data is recovered right up to the point of failure

Core Database Platform

• Self-Service BI

• Designed to empower users to explore data

• Users can create their own reports

• Supports inclusion of data from a wide range of sources

• Reduced IT Workload

• Business users perform much of their own analysis and reporting

• IT specialists can be freed to focus on high-value activities

• Big Data

• A huge volume of potentially useful information can now be generated from several sources

• SQL Server facilitates big data analysis using familiar tools

• Users are empowered to explore and combine data from a wide range of internal and external

sources

Enterprise Business Intelligence

• Benefits

• Reduced infrastructure management

• On-demand scalability

• Better control of finances

• Public Cloud

• Internet-based infrastructure

• Eliminates the need to install and manage hardware

• Reduces operational overheads

• Reduces data storage costs

• Private Cloud

• Retains internal control for operational, compliance or security requirements

• Uses company-dedicated hardware

• Can reduce database or server provisioning times

• Hybrid Cloud

• Enables an incremental move to the cloud

• Supports data storage across public and private cloud

Cloud Database Solutions

•Enterprise Edition

•Standard Edition

•Web

•Developer

•Express Editions

SQL Server 2016 Editions

•Database Engine

•Analysis Services

•Reporting Services

• Integration Services

•Master Data Services

•Data Quality Services

•SQL Server R Services

•Replication

SQL Server 2016 Components

Not just a database engine

Upgrading from Previous Versions

In-place Upgrade Side-by-side Upgrade

• Easier, mostly automated • More granular control over process

• System data upgraded • Can be used to perform test migration

• No additional hardware • Relatively straightforward rollback

• No change to applications • Can leverage failover/switchover

SQL Server 2008 SP3

SQL Server 2008 R2 SP2

SQL Server 2012 SP1

SQL Server 2014

SQL Server 2016

Run the Upgrade Advisor to check for upgrade issues

•Separate Release Cycle to SQL Server Database Engine

•Updates can be made independently

•Updates to SSMS can be made available as soon as they are ready

•No need to wait for next update to SQL Server

SQL Server Management Studio Enhancements

•Always Encrypted technology• Help you protect your data

• Regardless of where stored or accessed

•Stretch Database technology• Transparently store data in databases stretched between on-premises and Microsoft Azure (cloud)

•Enhanced backup/restore to/from Microsoft Azure• Faster hybrid backups

• High availability

• Disaster recovery scenarios

Enhanced Azure Support

SQL Server 2016 Performance

It Just Runs Faster

• By default provides automatic, soft NUMA configuration• Apply SQL Server 2016 and SQL Server internally leverages SOFT NUMA partitioning to achieve double digit

performance gains.

•DBCC Scales 7x Better• Out of the box DBCC provides you better performance, scale while shrinking your maintenance window

•Native Spatial Implementation(s)• Spatial activities are faster and scale better with Native Spatial Implementation

• SQL Server Parallel Query Placement Decision Logic

• TVPs with Spatial• Apply SQL Server 2016 and TVPs using Spatial columns at 15x faster or better

• Spatial index building is 2 or more times faster

•Out of the box enables the -T1117 and -T1118 behavior for TEMPDB providing better scalability and performance.

• XEvent Linq Reader• The SQL Server 2016 client component processes XEvent files 10x+ faster improving the responsiveness of the

XEvent UI and reader capabilities.

It Just Runs Faster

•Automatic TEMPDB Configuration

•LDF Stamped• SQL Server 16 changes the stamp to 0xC0’s instead of 0x00s.

• Changing the pattern to 0xC0’s avoids common reclamation techniques, improving performance

•Updated Scheduling Algorithms• SQL Server 2016 scheduling algorithms balance the work load better leading to enhanced scalability

•SOS_RWLock Redesign• The SOS_RWLock is a synchronization primitive used in various places throughout the SQL Server code base.

• As the name implies the code can have multiple shared (readers) or single (writer) ownership.

• Apply SQL Server 2016 and code paths leveraging reader/writer locks use less resources and scale better

• Indirect Checkpoint Default• New databases in SQL Server 2016 use indirect checkpoint, improving performance of checkpoint activities

•Larger Data File Writes• SQL Server uses WriteFileGather for the vast majority of data file write requests

• SQL Server 2016 takes advantage of newer hardware scalability by increasing the database file write operations

It Just Runs Faster

•Multiple Log Writer Workers• SQL Server 2016 uses up to 4, log writer workers to service log write activities improving LDF throughput

capabilities

•Column Store Uses Vector Instructions (SSE/AVX)• SQL Server 2016 detects the CPU capabilities for AVX (Advanced Vector Extensions) or SSE (Streaming SIMD

Extensions 4) and leverages the hardware, based vector capabilities to improve scalability and performance.

•BULK INSERT Uses Vector Instructions (SSE/AVX)• SQL Server 2016 takes advantage of CPU vector instructions to improve bulk insert performance

• In-Memory Optimized Database Worker Pool• SQL Server 2016 dynamically adjusts the In-Memory Optimized worker pool to maximize scalability and

performance.

•Leverages On Demand MSDTC Startup• SQL Server 2016 dynamically starts MSDTC as needed allowing resources to be used for other activities until

required

It Just Runs Faster

•AlwaysOn Log Transport Reduced Context Switches• The AlwaysOn log transport uses a SQL Broker based design to send and receive messages between the

primary and secondary replicas.

• SQL Server 2016 improves the log block transportation throughput scaling AlwaysOn by 4x.

•AlwaysOn Parallel Compression / Improved Algorithms• SQL Server 2016 introduces two distinct changes in the AlwaysOn transport, compression design.

• Improved compression algorithms

• Parallel compression of log block data

•AlwaysOn AES-NI Encryption• SQL Server 2016 defaults endpoint creation to AES based encryption allowing hardware based AES-NI

encryption.

• The hardware based capabilities improve Always On log shipping scalability and performance by a significant

factor.

It Just Runs Faster

Throughput MB/s Average CPU utilization (secondary) MB sent on wire/sec

Today 82 17 35

2016 540 36 230

It Just Runs Faster

Simply

upgrading to

SQL 2016 could

bring 25%

performance

improvement

SQL 2016

supports 3X more

physical memory

than previous

versions

The new column

store engine and

query processing

technology could

increase query

performance up

to 100X

The new In-memory

OLTP engine can

process 1.25million

batches/sec on a

single 4 socket

server, 3X of SQL

2014.

SQ

L Serv

er

2016

Exploring New Features

Operational Analytics

Operational Analytics with a Time Delay

What is Real-Time Operational Analytics

SQL Server 2016 Operational Analytics

What are Columnstore Indexes?

How to Use Real-Time Operational Analytics

Operational Analytics with a Time Delay

Key issues

Complex implementation

Requires two servers

(capital expenditures and

operational expenditures)

Data latency in analytics

High demand;

requires real-time analytics

IIS Server

BI analysts

What is Real-Time Operational Analytics

IIS Server

BI analysts

ChallengesAnalytics queries are resource intensive and can cause blocking

Minimizing impact on operational workloads

Sub-optimal execution of analytics on relational schema

BenefitsNo data latency

No ETL

No separate data warehouse

The ability to run analytics queries concurrently with

operational workloads using the same schema

Not a replacement for:• Extreme analytics performance queries possible only using customized schemas (e.g.

Star/Snowflake) and pre-aggregated cubes

• Data coming from non-relational sources

• Data coming from multiple relational sources requiring integrated analytics

Goals:• Minimal impact on operational workloads with concurrent analytics

• Performance analytics for operational schema

SQL Server 2016 Operational Analytics

•Columnstore indexes store data in a column-wise format

•Only one columnstore index can exist per table

• Improves query performance for large datasets, particularly in data

warehousing

•A filter expression can be included when creating the index

What are Columnstore Indexes?

CREATE CLUSTERED COLUMNSTORE INDEX CCIIdxFactOnlineSalesON dbo.FactOnlineSales;

How to Use Real-Time Operational Analytics

CREATE TABLE dbo.Product(

ProductId int IDENTITY(1,1) PRIMARY KEY,ProductCode nvarchar(10),ListPrice money,UnitsInStock int,UnitsSold int

);CREATE NONCLUSTERED COLUMNSTORE INDEX NCCIdx_ProductsON dbo.Product (ProductId, ProductCode, UnitsInStock, UnitsSold);

Columnstore Index

In-Memory OLTP Enhancements

What is In-Memory OLTP?

New In-Memory OLTP Features

Updated In-Memory OLTP Processing

Greater T-SQL coverage

Updated In-Memory Garbage Collection

In-Memory OLTP boosts performance:

•Commonly known by its code name of “Hekaton”

•Tables reside in-memory, and are faster to access for reads and writes, having

no locks or latches

• Increased performance for short-running transactions in an OLTP system

•The CPU can execute natively compiled stored procedures

What is In-Memory OLTP?

In-Memory OLTP enhancements include:

•Up to 2 TB of user data in memory (up from 250 GB)

•Support for system-versioned (temporal) tables

•Support for row-level security (RLS)

•Query Store supports natively-compiled procedures

•Support for all Unicode and collations

•ALTER PROCEDURE supports natively-compiled procedures

•ALTER TABLE supports memory-optimized tables

•Support for Transparent Data Encryption (TDE)

•MARS connections can use memory-optimized tables and natively compiled

procedures

New In-Memory OLTP Features

In-Memory OLTP processing updates include:

•FOREIGN KEY constraints between memory-optimized tables

•NULLable index key columns

•LOB column types and UNIQUE indexes

•CHECK constraints

•UNION and UNION ALL

•SELECT DISTINCT

•OUTER JOIN

•OR, NOT

•Subqueries within SELECT statements, such as EXISTS, IN, and scalar

subqueries

Updated In-Memory OLTP Processing

• CREATE PROCEDURE (Transact-SQL)

• DROP PROCEDURE (Transact-SQL)

• ALTER PROCEDURE (Transact-SQL)

• SELECT (Transact-SQL) and INSERT SELECT statements

• SCHEMABINDING and BEGIN ATOMIC (required for natively compiled stored procedures)

• NATIVE_COMPILATION

• Parameters and variables can be declared as NOT NULL

• Table-valued parameters.

• EXECUTE AS OWNER, SELF, and user.

• GRANT and DENY permissions on tables and procedures.

• Nesting natively compiled stored procedures

• RIGHT OUTER JOIN, LEFT OUTER JOIN, INNER JOIN, and CROSS JOIN in SELECT statements

• NOT, OR, and IN operators in SELECT, UPDATE and DELETE statement

• UNION ALL and UNION

• SELECT DISTINCT

• GROUP BY clause with no aggregate functions in the SELECT clause (<select> list).

• COLUMNSTORE

• COLLATE

Greater T-SQL coverage

• In-Memory Garbage Collector:• Controlled by the main garbage collection thread

• Runs every minute, or

• Runs when number of committed transactions exceeds an internal threshold

• In-Memory Garbage Collector Characteristics:• Non-blocking

• Cooperative

• Efficient

• Responsible

• Scalable

Updated In-Memory Garbage Collection

Memory-Optimized Table

Native JSON

Native JSON Support for SQL Server 2016

JSON in SQL Server 2016

JSON Facts

The FOR JSON Clause

JSON Functions

Indexing JSON Documents

Native JSON Support for SQL Server 2016

A top feature request on MS Connect (1050 votes)

OneDrive Office

Dynamics Bing Yammer

TFS

Service

integration

Exchange information with

various services

Generate

web service

content

Generate JSON that will be

returned to clients

Flexible

database

schema

Make reasonable trade-offs

in database schema design

Analyze

JSON

documents

Parse, query, and analyze

JSON documents

Use

case

s

JSON in SQL Server 2016

[

{

"Number":"SO43659",

"Date":"2011-05-31T00:00:00"

"AccountNumber":"AW29825",

"Price":59.99,

"Quantity":1

},

{

"Number":"SO43661",

"Date":"2011-06-01T00:00:00“

"AccountNumber":"AW73565“,

"Price":24.99,

"Quantity":3

}

]

SO43659 2011-05-31T00:00:00 MSFT 59.99 1

SO43661 2011-06-01T00:00:00 Nokia 24.99 3

Table 2 JSON

Formats result set

as JSON text.

JSON 2 table

Migrates JSON

text to table

Built-in functions

ISJSON

JSON_VALUE

JSON_MODIFY

No custom type or indexStore as NVARCHAR

Does JSON work with X?Does NVARCHAR work with X?

Yes, with in-memory, row-level security, stretch, compression, encryption, and more

Yes, with all client drivers

Different from DocumentDBSQL DB: Relational, easily exchanges data with modern services, unified insights

DocumentDB: Transactional schema-less document store

JSON Facts

•Format data for JSON with control over objects:

•Format data for JSON using automatic formatting:

The FOR JSON Clause

SELECT ProductNumber, Name, ListPriceFROM Production.ProductFOR JSON PATH;

SELECT ProductNumber, Name, ListPriceFROM Production.ProductFOR JSON AUTO;

JSON output:

Table to JSON

The FOR JSON Clause

SO43659 2011-05-31T00:00:00 MSFT 59.99 1

SO43661 2011-06-01T00:00:00 Nokia 24.99 3

SELECT Number AS [Order.Number], Date AS [Order.Date],

Customer AS Account,

Price AS 'Item.UnitPrice', Quantity AS 'Item.Qty'

FROM SalesOrder

FOR JSON PATH

JSON to Table

The FOR JSON Clause

@json:

SELECT *

FROM OPENJSON (@json)

WITH (

Number varchar(200) N'$.Order.Number',Date datetime N'$.Order.Date',Customer varchar(200) N'$.Account',Quantity int N'$.Item.Quantity'

)

SO43659 2011-05-31T00:00:00 Microsoft 1

SO43661 2011-06-01T00:00:00 Nokia 3

•Use JSON functions to convert SQL Server data to JSON formatted data, and

extract data from JSON:• ISJSON tests if a string contains valid JSON

• JSON_VALUE extracts a scalar value from a JSON string

• JSON_QUERY extracts an object or an array from a JSON string

JSON Functions

• JSON data is not a built-in data type in SQL Server 2016

•Create an index on JSON data:• Create a computed column using the expression to be used in search queries

• Create the non-clustered index on the computed column

• Use the same expression in your queries as used to create the computed column, so SQL Server can perform a

seek, rather than a table scan

Indexing JSON Documents

Extract and Work with JSON Data

Temporal Tables

What is Temporal Data?

How to start with temporal

Temporal database support: BETWEEN

How does system-time work?

Application-time temporal

Querying Temporal Tables

Temporal data continuum

In-Memory OLTP and temporal

What is Temporal Data?

Data changes over timeTracking and analyzing changes is often important

Temporal in DBAutomatically tracks history of data changes

Enables easy querying of historical data states

Advantages over workaroundsSimplifies app development and maintenance

Efficiently handles complex logic in DB engine

Time travel Data audit

Slowly changing dimensions

Repair record-level corruptions

How to start with temporal

No change in programming model New Insights

INSERT / BULK INSERT

UPDATE

DELETE

MERGE

DML SELECT * FROM temporal

Querying

CREATE temporalTABLE PERIOD FOR SYSTEM_TIME…

ALTER regular_table TABLE ADD PERIOD…

DDL

FOR SYSTEM_TIME

AS OF

FROM..TO

BETWEEN..AND

CONTAINED IN

Temporal Querying

ANSI 2011 compliant

Provides correct information

about stored facts at any

point in time, or between

two points in time

There are two orthogonal sets of scenarios with regards to temporal data:

System (transaction)-time

Application-time

SELECT * FROM Person.BusinessEntityContact

FOR SYSTEM_TIME BETWEEN @Start AND @End

WHERE ContactTypeID = 17

Temporal database support: BETWEEN

Temporal table (actual data)

Insert / Bulk Insert

* Old versions

Update */ Delete *

History table

Performance


Temporal table (actual data)

Temporal queries *(Time travel, etc.)

History table

Regular queries

(current data)

* Include historical version

Performance


Limits of system-timeTime flows ‘forward only’

System-time ≠ business-time

(sometimes)

Immutable history, future does not

exist

App-time = new scenariosCorrect past records as new info is

available (HR, CRM, insurance,

banking)

Project future events (budgeting,

what-if, loan repayment schedule)

Batch DW loading (with delay)

Performance

CREATE TABLE Employee([EmployeeNumber] int NOT NULL,[Name] nvarchar(100) NOT NULL,[LocationId] int NOT NULL,[Position] varchar(50) NOT NULL,[AnnualSalary] decimal (10,2) NOT NULL,

ValidFrom datetime2 NOT NULL,ValidTo datetime2 NOT NULL,PERIOD FOR VALID_TIME (ValidFrom,ValidTo),

CONSTRAINT PK_EmployeePRIMARY KEY CLUSTERED (EmployeeNumber, VALID_TIME WITHOUT OVERLAPS)

)

ALTER TABLE Employee

ADD CONSTRAINT FK_Employee_DepartmentFOREIGN KEY (LocationId, PERIOD VALID_TIME)REFERENCES Location (LocationId, PERIOD VALID_TIME);

UPDATE Employee

FOR PORTION OF VALID_TIME FROM '2010-01-01' TO '2012-01-01'

SET [Position] = 'CEO'WHERE EmployeeNumber = 1

DELETE FROM Employee

FOR PORTION OF VALID_TIME FROM '2012-01-01' TO '2013-01-01'

WHERE EmployeeNumber = 1

SELECT * FROM Employee

WHERE VALID_TIME CONTAINS '2013-06-30'

SELECT * FROM EmployeeWHERE EmployeeNumber = 1 AND

VALID_TIME OVERLAPS PERIOD ('2013-06-30', '2014-01-01')

/* Temporal join */SELECT * FROM Employee E

JOIN Position D ON E.Position = D.Position ANDD.VALID_TIME CONTAINS PERIOD E.VALID_TIME

Consistency

Temporal edits

Easy time travel

querying

Application-time temporal

•System-versioned tables can be queried using the FOR SYSTEM_TIME clause

and one of the following four sub-clauses:• AS OF <date_time>

• FROM <start_date_time> TO <end_date_time>

• BETWEEN <start_date_time> AND <end_date_time>

• CONTAINED IN (<start_date_time>, <end_date_time>)

•Or use ALL to return everything

Querying Temporal Tables

Temporal data continuum

SQL Database

Performance

Extreme OLTP with

cost-effective data history

Disk-based history table

Super-fast DML and current data querying

Temporal querying in interop mode

In-Memory OLTP and temporal

Fast DML

Internal

data

retention

Performance

Create System-Versioned Tables

Using Always Encrypted

The need for Always Encrypted

Why Use Always Encrypted?

How it works

Always Encrypted Encryption Types

Always Encrypted Keys

Implementing Always Encrypted

The need for Always Encrypted

Prevents data

disclosure

Client-side encryption of

sensitive data using keys that

are never given to the

database system

Queries on

encrypted data

Support for equality

comparison, including join,

group by, and distinct

operators

Application

transparency

Minimal application changes

via server and client library

enhancements

Allows customers to securely store sensitive data outside of their trust boundary.

Data remains protected from high-privileged, yet unauthorized users.

•Always Encrypted is new in SQL Server 2016

•Always Encrypted solves the problem of data being displayed in plain text,

even though it is encrypted on disk

•Always Encrypted encrypts specific columns

•Encryption “at rest” and “in flight”

•Encryption keys are stored on the client, not the server

•Use Always Encrypted together with TDE

•Always Encrypted is already implemented in Azure SQL Database

Why Use Always Encrypted?

How it works

SQL Server or SQL Database

ADO .NET

Name

Wayne Jefferson

Name

0x19ca706fbd9a

Result SetResult Set

Client

Name SSN Country

0x19ca706fbd9a 0x7ff654ae6d USA

dbo.Customers

"SELECT Name FROM Customers WHERE SSN = @SSN","111-22-3333"

Encrypted sensitive data and corresponding keys are never seen in plaintext in SQL Server

trust boundary

ciphertext

"SELECT Name FROM Customers WHERE SSN = @SSN",0x7ff654ae6d

ciphertext

• Randomized encryption

• Encrypt ‘123-456-789’ = Ox573hshg2

• Repeat ‘123-456-789’ = Ox8b3pdi23

• Enables transparent retrieval of encrypted data

• Cannot perform operations with randomized encryption

• Cannot use randomized encryption on indexed columns

• More secure

• Deterministic encryption

• Encrypt ‘123-456-789’ = Ox573hshg2

• Repeat ‘123-456-789’ = Ox573hshg2

• Enables transparent retrieval of encrypted data

• Can perform equality comparisons

• Columns can be indexed

Always Encrypted Encryption Types

•Two keys are required:• Column Master Key

• Column Encryption Key

•The Column Master Key encrypts the Column Encryption Key

•The Column Encryption Key encrypts the data

•Keys are stored client side, not on the server

•New catalog views:SELECT * FROM sys.column_master_key_definitions

SELECT * FROM sys.column_encryption_Keys

Always Encrypted Keys

Security

Officer

1. Generate CEKs and master key

2. Encrypt CEK

3. Store master key securely

4. Upload encrypted CEK to DB

CMK store:

Certificate store

HSM

Azure Key Vault

…

Encrypted

CEK

Column

encryption key

(CEK)

Column

master key

(CMK)

CMK

Database

Encrypted CEK

Security

Key provisioning

Param

Encryption

Type/

Algorithm

Encrypted

CEK Value

CMK Store

Provider

Name CMK Path

@Name Non-DET/

AES 256

CERTIFICATE

_STORE

Current User/

My/f2260…

EXEC sp_execute_sqlN'SELECT * FROM Customers WHERE SSN = @SSN', @params = N'@SSN VARCHAR(11)', @SSN=0x7ff654ae6d

Param

Encryption

Type/

Algorithm

Encrypted

CEK Value

CMK Store

Provider

Name CMK Path

@SSN DET/ AES

256

CERTIFICATE

_STORE

Current User/

My/f2260…

Enhanced

ADO.NET

Plaintext

CEK

Cache

exec sp_describe_parameter_encryption@params = N'@SSN VARCHAR(11)', @tsql = N'SELECT * FROM Customers WHERE SSN = @SSN'

Result set (ciphertext)

Name

Jim Gray

Result set (plaintext)

using (SqlCommand cmd = new SqlCommand("SELECT Name FROM Customers WHERE SSN = @SSN“, conn)){cmd.Parameters.Add(new SqlParameter("@SSN", SqlDbType.VarChar, 11).Value ="111-22-3333");

SqlDataReader reader =cmd.ExecuteReader();

Client (trusted) SQL Server (untrusted)

En

crypti

on

meta

data

Name

0x19ca706fbd9

En

crypti

on

meta

data

CMK store

Security

Example

•Prerequisites: .Net Framework 4.6 and SQL Client

•Using Object Explorer, select your database, then Security• Create a Column Master Key

• Create a Column Encryption Key

• Encrypt individual columns by right-clicking and choosing Encrypt Column from the menu. The Always

Encrypted wizard steps through the process

•Encrypt individual columns using CREATE TABLE or ALTER TABLE

•Remove Always Encrypted by choosing “plain text”

Implementing Always Encrypted

Always Encrypted

High Availability Enhancements

The need for mission-critical availability

Enhanced AlwaysOn Availability Groups

Load balancing in readable secondaries

Distributed Transaction Coordinator (DTC) support

Database-level failover trigger

gMSA support

Domain-independent Availability Groups

The need for mission-critical availability

• Unified, simplified solution

• Easy to deploy, manage, and monitor

• Reuse existing investments

• SAN/DAS environments

• Able to use HA hardware resources

• Fast, transparent failover

• Detects failures reliably

• Able to handle multiple failures

Enhanced AlwaysOn Availability Groups

AG_Listener

New York(Primary)

Asynchronous data Movement

Synchronous data Movement

Unified HA solution

AG

Hong Kong(Secondary)

AG

New Jersey(Secondary)

AG

Greater scalabilityLoad balancing readable secondaries

Increased number of automatic failover targets

Log transport performance

Improved manageabilityDTC support

Database-level health monitoring

Group Managed Service Account


Load balancing in readable secondaries

Computer5

DR site Computer2

Computer3

Computer4

Primary site

Computer1

(Primary)

READ_ONLY_ROUTING_LIST=(('COMPUTER2', 'COMPUTER3','COMPUTER4'), 'COMPUTER5')

In SQL 2014, read-only transactions routed by the Listener went to the first secondary that was available

Read-only Routing (ROR) lists

Now you can configure the ROR lists to round-robin among a specific set of secondaries (for each primary)

•Currently, any distributed transactions touching a database in an

availability group are not allowed• Many customers run unsupported, at risk to their data and reputation

• Many enterprise applications need cross-database transactions

•Fully supported in SQL Server 2016• Joint effort with Windows

• Requires specific patch in order to work cleanly

• Other requirements:

• Availability groups must be running on Windows Server 2016 Technical Preview 2

• Availability groups must be created with the CREATE AVAILABILITY GROUP command and the WITH

DTC_SUPPORT = PER_DB clause. You cannot currently alter an existing availability group

• Learn more: https://msdn.microsoft.com/en-us/library/ms366279.aspx

Distributed Transaction Coordinator (DTC) support

•Currently, Availability Groups only monitor the health of the instance• A database can be offline or corrupt, but will not trigger a failover as long as the instance itself

is healthy

•SQL Server 2016: option to also monitor the health of the databases in the

Availability Group• Databases going offline trigger a change in the health status

Database-level failover trigger

•Group Managed Service Accounts (gMSA)• Automatically set domain scope for Managed Service Accounts

• Automatic password rotation

• Much more secure than regular domain accounts

• Enables cross-system security context

•Why would I want a gMSA?• No need to manually change passwords on all AlwaysOn instances

•How does it work?• Passwords are managed by the domain

•What versions will it be supported in?• Supported in SQL Server 2014 and SQL Server 2016

gMSA support

•New feature in Windows Server 2016

•Environments supported:• Cross domains (with trust)

• Cross domains (no trust)

• No domain at all

•Cluster management via PowerShell only

•SQL management as normal

•Uses certificate-secured endpoints like DBM


Stretch Database

Why Stretch Database?

Stretch SQL Server into Azure

Stretch Database Architecture

How to enable Stretch database

Queries continue working

Advanced security features supported

Backup and Restore

Why Stretch Database

What to do?

Expand server and storage

Move data elsewhere

Delete

Massive tables (hundreds of millions/billions of rows, TBs size)

Users want/need to retain data indefinitely

Cold data infrequently accessed but must be online

Datacenter consolidation

Maintenance challenges

Business SLAs at risk

Stretch SQL Server into Azure

Capability

Stretch large operational tables

from on-premises to Azure with

the ability to query

Benefits

SQL

SERVER

2016

Azure

Stretch Database architecture

How it works• Creates a secure linked server

definition in the on-premises SQL

Server

• Linked server definition has the

remote endpoint as the target

• Provisions remote resources and

begins to migrate eligible data, if

migration is enabled

• Queries against tables run against

both the local database and the

remote endpoint

Remote

endpoint

Remote data

Azure

Inte

rnet B

ou

nd

aryLocal

database

Local data

Eligible data

Linked Servers

How to enable Stretch database

-- Enable local server

EXEC sp_configure 'remote data archive' , '1';

RECONFIGURE;

-- Provide administrator credential to connect to

-- Azure SQL Database

CREATE CREDENTIAL <server_address> WITH

IDENTITY = <administrator_user_name>,

SECRET = <administrator_password>

-- Alter database for remote data archive

ALTER DATABASE <database name>

SET REMOTE_DATA_ARCHIVE = ON (SERVER = server name);

GO

-- Alter table for remote data archive

ALTER TABLE <table name>

ENABLE REMOTE_DATA_ARCHIVE

WITH ( MIGRATION_STATE = ON );

GO;

High-level steps• Configure local server for remote

data archive

• Create a credential with

administrator permission

• Alter specific database for remote

data archive

• Alter table for remote data

archive

Queries continue working

• Business applications continue working without disruption

• DBA scripts and tools work as before. All controls still held in local SQL Server

• Developers continue building or enhancing applications with existing tools and methods

Advanced security features supported

• Data in motion always via secure channels (TLS1.1 / 1.2)

• Always Encrypted supported if enabled by user Encryption key remains on-premises

• Row-level security already working

• SQL Server and SQL Azure audit already working

Backup and Restore

• DBAs only backup/restore local SQL Server hot data

• StretchDB ensures remote data is transactionally consistent with local

• Upon completion of local restore, SQL Server reconciles with remote using metadata operation, not data copy

• Time of restore for remote not dependent on size of data

Enhanced backup

Enhanced backup to Azure

Managed backup

Customized scheduling

Backup to Azure block blobs

Backup to Azure with file snapshots (SQL Server 2016)

Backup to Azure with file snapshots

Point-in-time restore with file snapshots

Summary: enhanced backup

Managed backup

Granular control of the backup

schedule

Local staging support for faster

recovery and resilient to transient

network issues

Support for system databases

Supports simple recovery mode

Backup to Azure block

blobs

Cost savings on storage

Significantly improved restore

performance

More granular control over Azure

Storage

Azure Storage snapshot

backup

Fastest method for creating

backups and running restores

Uses SQL Server database files on

Azure Blob storage

Hybrid solutions

Enhanced backup to Azure

•Support for system databases

•Supports databases in simple recovery mode

•Leveraging backup to block blobs: more granular control

•Allows customized backup schedules: full backup and log backup

Managed backup

105Hybrid solutions

Customized scheduling

Step1: Run the Scheduling SP to configure custom schedulingEXEC Managed_Backup.sp_backup_config_schedule

@database_name = 'testDB'

,@scheduling_option= 'Custom'

,@full_backup_freq_type = 'weekly’

,@days_of_week = 'Saturday'

,@backup_begin_time = '11:00'

,@backup_duration = '02:00'

,@log_backup_freq = '00:05'

Step2: Run the Basic SP to configure Managed Backup EXEC msdb.managed_backup.sp_backup_config_basic

@database_name= 'testDB',

@enable_backup=1,

@container_url='https://storage account

name.blob.core.windows.net/container name',

@retention_days=30

Hybrid solutions

2x cheaper storage

Backup striping and faster restore

Maximum backup size is 12 TB+

Granular access and unified credential story (SAS URIs)

Supports all existing backup/restore features (except append)

Backup to Azure block blobs

CREATE CREDENTIAL [https://<account>.blob.core.windows.net/<container>]

WITH IDENTITY = 'Shared Access Signature',

SECRET = 'sig=mw3K6dpwV%2BWUPj8L4Dq3cyNxCI'

BACKUP DATABASE database TO

URL = N'https://<account>.blob.core.windows.net/<container>/<blob1>',

URL = N'https://<account>.blob.core.windows.net/<container>/<blob2>'

Hybrid solutions

BACKUP DATABASE database TO

URL =

N'https://<account>.blob.core.windows.net/<container>/<backupfileblob1>‘

WITH FILE_SNAPSHOT

Backup to Azure with file snapshots (SQL Server 2016)

Instance

Azure Storage

MDFDatabase

MDF

LDFLDF

BAK

Hybrid solutions

•Available to users whose database files are located in Azure

Storage

•Make a copy of a database using a virtual snapshot within

Azure Storage The database data does not move between the storage system and the server instance, removing the IO

bottleneck

•Uses only a fraction of the space that a traditional backup

would consume


Hybrid solutions

Traditional backupMultiple backup types

Complex point-in-time restore process

Backup to Azure with file snapshots Full backup only once

Point-in-time only needs two adjacent backups

Full Log Log Log Diff Log Log Log Diff Log Log Log

Full . . . . . Log Log Log Log Log Log Log Log Log Log Log

Hybrid solutions

Point-in-time restore with file snapshots


Hybrid solutions

Summary: enhanced backup

Other Enhancements

•The new ALTER DATABASE SCOPED CONFIGURATION statement gives you

control of certain configurations for your particular database.

•The configuration settings affect application behavior.

•The new statement is available in both SQL Server 2016 and SQL Database

V12.

Database Scoped Configurations

•Trace Flags 1117 and 1118 are not required for tempdb anymore. • If there are multiple tempdb database files all files will grow at the same time depending on growth settings.

In addition, all allocations in tempdb will use uniform extents.

•By default, setup adds as many tempdb files as the CPU count or 8, whichever

is lower.

•During setup, you can configure the number of tempdb database files, initial

size, autogrowth and directory placement using the new UI input control on

the Database Engine Configuration - TempDB section of SQL Server

Installation Wizard.

•The default initial size is 8MB and the default autogrowth is 64MB.

•You can specify multiple volumes for tempdb database files. If multiple

directories are specified tempdb data files will be spread across the directories

in a round-robin fashion.

TempDB Database

•A table can reference a maximum of 253 other tables and columns as foreign

keys (outgoing references).

•SQL Server 2016 increases the limit for the number of other table and columns

that can reference columns in a single table (incoming references), from 253 to

10,000.

Foreign Key Relationship Limits

•bcp Utility

•BULK INSERT

•OPENROWSET

Support for UTF-8 code page

•New values for the model database and default values for new databases

(which are based on model).

•The initial size of the data and log files is now 8 MB.

•The default auto-growth of data and log files is now 64MB.

New Default Database Size and Autogrow Values

•Replication of memory-optimized tables are now supported

•Replication is now supported to Azure SQL Databasesee

Replication Enhancements

•ALTER ANY SECURITY POLICY permission • is available as part of the implementation of row level security.

•ALTER ANY MASK and UNMASK permissions • are available as part of the implementation of dynamic data masking.

•ALTER ANY COLUMN ENCRYPTION KEY, VIEW ANY COLUMN ENCRYPTION

KEY, ALTER ANY COLUMN MASTER KEY DEFINITION, and VIEW ANY

COLUMN MASTER KEY DEFINITION permissions • are available as part of the implementation of the Always Encrypted feature.

New Permissions

•ALTER ANY EXTERNAL DATA SOURCE and ALTER ANY EXTERNAL FILE

FORMAT permissions • are visible in SQL Server 2016 but only apply to the Analytics Platform System (SQL Data Warehouse).

•EXECUTE ANY EXTERNAL SCRIPT permissions • are available as part of the support for R scripts.

•ALTER ANY DATABASE SCOPED CONFIGURATION permissions

New Permissions

•The TRUNCATE TABLE statement now permits the truncation of specified

partitions

•ALTER TABLE now allows many alter column actions to be performed while the

table remains available.

•A new query hint NO_PERFORMANCE_SPOOL can prevent a spool operator

from being added to query plans. This can improve performance when many

concurrent queries are running with spool operations

•The FORMATMESSAGE statement is enhances to accept a msg_string

argument.

Additional Enhancements

•The maximum index key size for NONCLUSTERED indexes has been increased

to 1700 bytes.

•New DROP IF syntax is added for drop statements related to • AGGREGATE, ASSEMBLY, COLUMN, CONSTRAINT, DATABASE, DEFAULT, FUNCTION, INDEX, PROCEDURE,

ROLE, RULE, SCHEMA, SECURITY POLICY, SEQUENCE, SYNONYM, TABLE, TRIGGER, TYPE, USER, and VIEW.

•SESSION_CONTEXT can now be set. • Includes the SESSION_CONTEXT function,

• CURRENT_TRANSACTION_ID function,

• and the sp_set_session_context procedure.


•Advanced Analytics Extensions allow users to execute scripts written in a

supported language such as R. • Transact-SQL supports R by introducing the sp_execute_external_script stored procedure, and the external

scripts enabled Server Configuration Option.

• Also to support R, the ability to create an external resource pool

•The COMPRESS and DECOMPRESS functions • convert values into and out of the GZIP algorithm.

•A MAXDOP option has been added to• DBCC CHECKTABLE DBCC CHECKDB and DBCC CHECKFILEGROUP to specify the degree of parallelism.


•The DATEDIFF_BIG and AT TIME ZONE functions and the sys.time_zone_info

view are added to support date and time interactions.

•A credential can now be created at the database level (in addition to the

server level credential that was previously available)

•The input length limit of 8,000 bytes for the HASHBYTES function is removed.

•New string functions STRING_SPLIT and STRING_ESCAPE are added.


•Eight new properties are added to SERVERPROPERTY : • InstanceDefaultDataPath, InstanceDefaultLogPath

• ProductBuild, ProductBuildType,

• ProductMajorVersion, ProductMinorVersion,

• ProductUpdateLevel, ProductUpdateReference.

•Trace flag 1117 is replaced by the AUTOGROW_SINGLE_FILE and

AUTOGROW_ALL_FILES option of ALTER DATABASE, and trace flag 1117 has no

affect.

•For user databases, default allocation for the first 8 pages of an object will

change from using mixed page extents to using uniform extents. • Trace flag 1118 is replaced with the SET MIXED_PAGE_ALLOCATION option of ALTER DATABASE, and trace flag

1118 has no affect


SELECT KNOWLEDGE FROM SQL SERVER

Copyright © 2016 SQLschool.gr. All right reserved. PRESENTER MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION

Data & Analytics

Exploring sql server 2016