Embed Size (px)
Citation preview
SQL Server 2016Exploring New features in Business Inteligence
Pre
sen
ter
Info 1982 I started working with computers
1988 I started my professional career in computers industry.
1996 I started working with SQL Server 6.0
1998 I earned my first certification at Microsoft as Microsoft
Certified Solution Developer (3rd in Greece)
I started my career as Microsoft Certified Trainer (MCT)
with more than 25.000 hours of training until now!
2010 I became for first time Microsoft MVP on SQL Server
I created the SQL School Greece www.sqlschool.gr
2012 I became MCT Regional Lead by Microsoft Learning
Program.
2013 I was certified as MCSE : Data Platform & Business
Intelligence
Antonios ChatzipavlisDatabase Architect,
SQL Server Evangelist
MCT, MCSE, MCITP, MCPD, MCSD, MCDBA, MCSA, MCTS,
MCAD, MCP, OCA, ITIL-F
SQ
Lsch
ool.g
rTe
am
Antonios ChatzipavlisSQL Server Evangelist • Trainer
Vassilis IoannidisSQL Server Expert • Trainer
Fivi PanopoulouSystem Engineer • Speaker
Sotiris KarrasSystem Engineer • Speaker
Follo
w u
sin
soci
al m
edia
@antoniosch / @sqlschool
fb/sqlschoolgr
yt/c/SqlschoolGr
SQL School Greece group
The Conference for Technical Data Professionals
• 200+ technical sessions
• New and expert industry speakers
• Networking opportunities with thousands of attendees
from around the world
Use Local Chapter Discount Code: LC15CPJ8 for $150 off*
October 25-28
Seattle
*Cannot be applied retroactively or combined with other offers.
Join
and L
earn
Sta
y In
volv
ed
Sign up for a free membership today at sqlpass.org.
Linked In: http://www.sqlpass.org/linkedin
Facebook: http://www.sqlpass.org/facebook
Twitter: @SQLPASS
PASS: http://www.sqlpass.org
Pre
senta
tion
Conte
nt
Evolve your data platform
Exploring New Features
Evolve your data platform
DIGITAL
ANALOG
1985 1990 1995 2000 2005 2010 2015 2020
The world’s data
DIGITAL
ANALOG
1985 1990 1995 2000 2005 2010 2015 2020
ANALOG
DATACENTERS (CLOUD)
PC / DEVICE
DIGITAL TAPE
DVD / BLU-RAY
CD
The world’s data
CONNECTED
DIGITAL
ANALOG
1985 1990 1995 2000 2005 2010 2015 2020
DATACENTERS (CLOUD)
PC / DEVICE
DIGITAL TAPE
DVD / BLU-RAY
CD
Connected data
CONNECTED
DIGITAL
ANALOG
1985 1990 1995 2000 2005 2010 2015 2020
CLOUD / IoT
PC / MOBILE
Connected data
CONNECTED
DIGITAL
ANALOG
1985 1990 1995 2000 2005 2010 2015 2020
CLOUD / IoT
MOBILE
Connected data
{ }
Relational
Clo
ud • Disparate systems and processes
• Multiple tools and skillsets
• Siloed insights on disconnected data
• High cost of ownership
Inefficiencies from fragmented architecture
Beyond relational
On
-pre
mis
es
Challenges of the modern data platform
Azure SQL DB
Azure SQL DW
Analytics Platform System
Azure Data Lake
SQL Server 2016
Analytics Platform System
SQL
Relational Beyond relational
On
-pre
mis
es
Clo
ud
Data Management
Power BI
Cortana Analytics
Azure IoT
Business Analytics
Business Analytics & Data Management Platform
Fits your business
Intelligence made relevant to your business
Proven leader
Access any data Scale and manage Powerful insights Advanced analytics
PolyBase
Insights from data across SQL Server and Hadoop with the simplicity of T-SQL
Enhanced SSISDesigner support for previous SSIS versions
Enterprise-grade
Analysis Services Enhanced performance and scalability for Analysis Services
Single SSDT in Visual
Studio 2015 Build richer analytics solutions as
part of your development projects
in Visual Studio
Enhanced MDS Excel add-in 15x faster; moregranular security roles; archival options for transaction logs; and reuse entities across models
Mobile BIBusiness insights for your on-premises data through rich visualization on mobile devices with native apps for Windows, iOS, and Android
Enhanced Reporting
ServicesNew modern reports with rich visualizations
R integrationBringing predictive analytic capabilities to your relational database
Expand your “R” script library with Microsoft Azure Marketplace
Deeper insights across data
SQ
L Serv
er
2016
BI
Exploring New Features
PolyBase
Interest in big data spurs customer demand
Increase in number and
variety of data sources
that generate large
quantities of data
Realization that data is
“too valuable” to delete
Dramatic decline in the
cost of hardware,
especially storage
Adoption of big data technologies like Hadoop
$
PolyBase and queries
Provides a scalable, T-SQL-compatible query processing framework for combining data from both universes
PolyBase View in SQL Server 2016
PolyBase View
• Execute T-SQL queries against relational data in SQL Server and ‘semi-structured’ data in HDFS and/or Azure
• Leverage existing T-SQL skills and BI tools to gain insights from different data stores
• Expand the reach of SQL Server to Hadoop(HDFS)
PolyBase use cases
•PolyBase Engine Service
•PolyBase Data Movement Service (with HDFS Bridge)
•External table constructs
•MR pushdown computation support
Components introduced in SQL Server 2016
How to use PolyBase in SQL Server 2016
Set up a Hadoop Cluster or Azure Storage blob
Install SQL Server
Configure a PolyBasegroup
Choose Hadoop flavor
Attach Hadoop Cluster or Azure Storage
PolyBase T-SQL
queries submitted
here
PolyBase queries can
only refer to tables
here and/or external
tables here
Co
mp
ute
no
des
Head nodes
Access any data
Step 1: Set up a Hadoop Cluster…
Hortonworks or Cloudera DistributionsHadoop 2.0 or above
Linux or Windows
On-premises or in Azure
Access any data
Step 1: …Or set up an Azure Storage blob
Azure Storage blob (ASB) exposes an HDFS layer
PolyBase reads and writes from ASB using Hadoop
RecordReader/RecordWrite
No compute pushdown support for ASB
Azure
Storage
Volume
Azure
Storage
Volume
Azure
Storage
Volume
Azure
Access any data
Step 2: Install SQL Server
PolyBase
DLLs
PolyBase
DLLs
PolyBase
DLLs
PolyBase
DLLs
Install one or more SQL Server instances with PolyBase
PolyBase DLLs (Engine and DMS) are installed and registered
as Windows Services
Prerequisite: User must download and install JRE (Oracle)
Access any data
Step 3: Configure a PolyBase group
PolyBase
Engine
PolyBase
DMS
PolyBase
DMS
PolyBase
DMS
PolyBase
DMS
Use stored procedures and GUI to configure nodes as compute
nodes of a PolyBase group
EXEC sp_join_polybase_group
bob.contoso.local, DemoServer, 1433;
EXEC sp_leave_polybase_group;
Head node
Compute nodes
Access any data
Step 3: Configure a PolyBase group
PolyBase
Engine
PolyBase
DMS
PolyBase
DMS
PolyBase
DMS
PolyBase
DMS
Head node
Compute nodes
PolyBase scale-out group
Head node is the SQL Server
instance to which queries are
submitted
Compute nodes are used for
scale-out query processing for
data in HDFS or Azure
Supported Hadoop distributions Cloudera CHD 5.x on Linux
Hortonworks 2.x on Linux and Windows Server
What happens under the covers? Loading the right client jars to connect to Hadoop distribution
-- different numbers map to various Hadoop flavors-- example: value 4 stands for HDP 2.x on Linux,
value 5 for HDP 2.x on Windows,value 6 for CHD 5.x on Linux
Access any data
Step 4: Choose Hadoop flavor
Step 5: Attach Hadoop Cluster or Azure Storage
PolyBase
Engine
PolyBaseDMS
PolyBaseDMS PolyBaseDMS PolyBaseDMS
Head node
Azure
Storage
Volume
Azure
Storage
Volume
Azure
Storage
Volume
Azure
Access any data
After Setup
Compute nodes are used for scale-out query processing on external tables in HDFS
Tables on compute nodes cannot be referenced by queries submitted to head node
Number of compute nodes can be dynamically adjusted by DBA
Hadoop clusters can be shared between multiple SQL16 PolyBase groups
PolyBase T-SQL
queries submitted
here
PolyBase queries can
only refer to tables
here and/or external
tables here
Co
mp
ute
no
des
Head nodes
Access any data
PolyBase query example #1
-- select on external table (data in HDFS)SELECT * FROM Customer
WHERE c_nationkey = 3 and c_acctbal < 0;
A possible execution plan:
CREATE temp
table TExecute on compute nodes1
IMPORT
FROM HDFSHDFS Customer file read into T
2
EXECUTE
QUERYSelect * from T where
T.c_nationkey =3 and T.c_acctbal < 03
Access any data
PolyBase query example #2
-- select and aggregate on external table (data in HDFS)SELECT AVG(c_acctbal) FROM Customer
WHERE c_acctbal < 0 GROUP BY c_nationkey;
Execution plan:
Run MR Job
on Hadoop
Apply filter and compute
aggregate on Customer. 1
What happens here?Step 1: QO compiles predicate into Java
and generates a MapReduce (MR) job
Step 2: Engine submits MR job to
Hadoop cluster. Output left in hdfsTemp.
hdfsTemp<US, $-975.21>
<FRA, $-119.13>
<UK, $-63.52>
Access any data
PolyBase query example #2
-- select and aggregate on external table (data in HDFS)SELECT AVG(c_acctbal) FROM Customer
WHERE c_acctbal < 0 GROUP BY c_nationkey;
Execution plan: 1. Predicate and aggregate pushed
into Hadoop cluster as a
MapReduce job
2. Query optimizer makes a cost-
based decision on what
operators to push
Run MR Job on
HadoopApply filter and compute
aggregate on Customer.
Output left in hdfsTemp
1
IMPORT
hdfsTEMP Read hdfsTemp into T 3
CREATE temp
table T On DW compute nodes2
RETURN
OPERATION Select * from T4
hdfsTemp<US, $-975.21>
<FRA, $-119.13>
<UK, $-63.52>
Access any data
Polybase
Integration Services
SSIS improvements for SQL Server 2016
AlwaysOn support
Incremental deployment of packages
Improved project upgrade support
Error column name support
Custom log level
Package template
OData V4 support
Designer improvements
One designer multi-version support
AlwaysOn Availability Groups
Secondary for SSISDB
New York(Primary)
New Jersey(Secondary)
SSIS
DB
SSIS
DB
SQL Server 2012SSIS Project XSQL Server 2016SSIS Project X
Improved project upgrade
Access any data
AlwaysOn support for SSISDB
DBA can set up AlwaysOn availability groups
for the SSIS Catalog
Access any data
Deploy packages to Integration Services server
private static void Main(string[] args)
{
// Connection string to SSISDB
var connectionString = "Data Source=.;Initial Catalog=SSISDB;Integrated Security=True;MultipleActiveResultSets=false";
using (var sqlConnection = new SqlConnection(connectionString))
{
sqlConnection.Open();
var sqlCommand = new SqlCommand
{
Connection = sqlConnection,
CommandType = CommandType.StoredProcedure,
CommandText = "[catalog].[deploy_packages]"
};
var packageData = Encoding.UTF8.GetBytes(File.ReadAllText(@"C:\Test\Package.dtsx"));
// DataTable: name is the package name without extension and package_data is byte array of package.
var packageTable = new DataTable();
packageTable.Columns.Add("name", typeof(string));
packageTable.Columns.Add("package_data", typeof(byte[]));
packageTable.Rows.Add("Package", packageData);
// Set the destination project and folder which is named Folder and Project.
sqlCommand.Parameters.Add(new SqlParameter("@folder_name", SqlDbType.NVarChar, ParameterDirection.Input, "Folder", -1));
sqlCommand.Parameters.Add(new SqlParameter("@project_name", SqlDbType.NVarChar, ParameterDirection.Input, "Project", -1));
sqlCommand.Parameters.Add(new SqlParameter("@packages_table", SqlDbType.Structured, ParameterDirection.Input, packageTable, -1));
var result = sqlCommand.Parameters.Add("RetVal", SqlDbType.Int);
result.Direction = ParameterDirection.ReturnValue;
sqlCommand.ExecuteNonQuery();
}}
Deployment options Integration Services Deployment Wizard
SQL Server Management Studio
Deploy packages stored procedure
Object model API
SQL Server Data Tools for Business Intelligence
Access any data
Error column name support
Developer can see the error column name in
both the data viewer and editor
Developer can also see the IntToString
lineage ID mapping in the log
Developer can also programmatically get the
column name using the lineage ID
Custom Log level
Developer can create and use a customized
log level other than the default
Access any data
Package template
Developer can save part of the package as a
template and reuse it in the design of other
packages
Access any data
OData Source support for V4 protocol
Access any data
Capability
Stretch large operational tables
from on-premises to Azure with
the ability to query
Benefits
SQL
Access any data
SSIS improvements for Azure services
Reporting Services & Mobile Report
SQL Server 2016 Reporting Services and mobile reporting
SQL Server 2016
Reporting Services
(Native mode)
Datazen Server now a
component of SSRS
CREATE
SQL Server 2016Reporting Services
(SharePoint integrated mode)
Datazen Windows app
Datazen Android app
Datazen iOS app
Report Server web portal(paginated and mobile reports)
Datazen Publisher
SQL Server Report Builder
Report Designer inSQL Server Data Tools
Datazen phone app
Pin paginated report elements
to Power BI dashboards
SSRS reports rendered as PDF
SharePoint web
MANAGE CONSUME
Render and export reports to PowerPoint® with SQL Server® 2016 Reporting
Services:
• In Report Builder or Report Manager, click Export and choose PowerPoint from
the list
•Pass “PowerPoint” as a parameter in the URL string to render straight to
PowerPoint
PowerPoint Rendering and Export
Print reports to PDF from the report viewer toolbar:
•No need to download an ActiveX control
•Page preview shows how the printed pages will look
•Page settings can be configured
•Can also save to PDF format
•Printing is enabled by default, but can disabled
PDF for Remote Printing
•Reports now render to HTML5:
•Richer and faster user experience
•Targets modern browsers
•Switch between HTML5 and old rendering engine
HTML5 Rendering Engine
•New charts in SQL Server 2016 Reporting Services include:
New Chart Types
Tree Map Chart Sunburst Chart
Reporting Services subscriptions have been enhanced with the following
features:
•Native Mode:• Quickly enable and disable subscriptions. Disabled subscriptions retain configuration properties, so they are
easy to re-enable
• Shared credentials can be used for file share subscriptions. Can be alongside file share with individual
credentials
•SharePoint and Native Mode:• Include a report description when creating a subscription
• Change the owner of a subscription using the new interface – administrators can also do this using a script
Subscription Enhancements
SQL Server 2016 Reporting Services additional enhancements include:
•Reporting Services web portal gives access to users on role-based security
•Use Mobile Report Publisher to publish reports to the web portal for viewing
on mobile devices
•Pin report items to a Power BI dashboard
•Customize the layout of parameters with the new grid-style design surface
•Support for Microsoft .NET Framework
•Report Builder supports High DPI
Other Reporting Services Updates
•SQL Server 2016 Mobile Report
Publisher:• Develop and publish highly visual mobile
reports
• Reports scale to size on tablets and phones
• Data sources include on-premises SQL Server,
Azure SQL Databases, and Excel
• Reports can be viewed in Power BI for iOS—
Power BI must be enabled on the reporting
server
• Compatible with Windows 7 and later
• Stand-alone download from Microsoft
website
• Create and manage KPIs and data sources in
the new Reporting Services
• Find the new Reporting Services web portal
at: http://<server_name>/reports_preview
What are Mobile Reports?
•SQL Server Mobile Report Publisher:• Mobile reports are published to the new SQL Server 2016 Reporting Services
• Both paginated and mobile reports can be rendered and viewed in the new Reporting Services
• Mobile reports can be viewed on an iOS mobile device:
• iPhone and iPad must be iOS 8.0 or later
• iPhone must be iPhone 5 or later
• KPIs can be viewed and created
Publishing Mobile Reports
Analysis Services
•Create larger data models—data is loaded more efficiently. Improvements to
MDS Add-in for Excel support this increase in capability
•Data compression on the entity level, using row level compression
•Transaction log maintenance and scheduling
•Super User function for easier security management
•Custom indexes
•Manage business rules using the MDS Excel Add-in
•Manage revision history
Enhanced Master Data Services
•Perform real-time advanced analytics on operational and analytic data
•R statistical programming language has been integrated into Transact-SQL
•Data scientists can create predictive applications in R and deploy to a SQL
Server 2016 production environment
•R can use SQL Server features including in-memory, columnstore indexes, and
parallel processing
•Execute Transact-SQL procedure with R code from any application that can
connect to SQL Server
•Requires Advanced Analytics Extensions, R Open, R Enterprise, post-install
configurations, and script
Advanced Analytics with R
•Set up environments using Configuration Manager
•Connect to any database from your history
•Pin favorite connections for fast access
•Browse SQL Azure databases
•System-versioned temporal tables in database projects
•Multiple languages
•Row level security
•New columnstore index enhancements
•SSIS control flow templates
•SSIS Hadoop connector supports Avro and Kerberos
•Max buffer size in SSIS data flow task increased
•SSAS calculated tables can be created
SSDT in Visual Studio 2015
•Tabular Model Scripting Language (TMSL) to automate administrative tasks
•Upgrade to tabular 1200 models, add roles, and upgrade metadata in SSDT
•DirectQuery now available for tabular models
•Partition management for tabular models
•Bi-Directional Cross Filters for tabular models
•Calculated tables in SSDT
•DBCC for Analysis Services
•New assembly to extend AMO
What’s New in SQL Server Analysis Services
•More than 50 new DAX functions in SQL Server 2016, including but not limited
to:• Date and Time Functions—DATEDIFF and CALENDAR
• Filter Functions—ADDMISSINGITEMS
• Information Functions—ISONORAFTER
• Math and Trig Functions—EVEN, PI, SIN, and TAN
• Text Functions—CONCATENATEX
• Other Functions—GROUPBY, INTERSECT, and ISEMPTY
•Named variables are now supported in DAX
•Save incomplete DAX measures
• Improved DAX formatting in the formula bar
Enhanced DAX Functionality
•Extended Events for Analysis Services
•Add computer accounts as administrators in SSMS
•Project templates added for tabular 1200 models in SSDT
•New data sources for DirectQuery mode include Teradata, Oracle, and
Microsoft Analytics Platform
•Formula fixup automatically updates measures referencing a table or column
that is renamed (tabular 1200 models only)
Interface Enhancements
• Improved performance for generating tabular models in the 1120 compatibility
level up to three times faster
•Tabular model parallel processing functionality for tables with two or more
partitions
•Simpler query generation for DirectQuery delivers better performance
Performance Improvements
SELECT KNOWLEDGE FROM SQL SERVER
Copyright © 2016 SQLschool.gr. All right reserved. PRESENTER MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION
