Upload
james-serra
View
2.907
Download
6
Embed Size (px)
Citation preview
James Serra
Implement SQL Server on an Azure VM
About Me Microsoft, Big Data Evangelist In IT for 30 years, worked on many BI and DW projects Worked as desktop/web/database developer, DBA, BI and DW architect and
developer, MDM architect, PDW/APS developer Been perm employee, contractor, consultant, business owner Presenter at PASS Business Analytics Conference, PASS Summit, Enterprise Data
World conference Certifications: MCSE: Data Platform, Business Intelligence; MS: Architecting
Microsoft Azure Solutions, Design and Implement Big Data Analytics Solutions, Design and Implement Cloud Data Platform Solutions
Blog at JamesSerra.com Former SQL Server MVP Author of book “Reporting with Microsoft SQL Server 2012”
Agenda Azure VMs Migrating data Scaling VMs SQL Server VM features VM storage HA/DR architectures Best practices
Who manages what?Infrastructure
as a Service
Storage
Servers
Networking
O/S
Middleware
Virtualization
Data
Applications
Runtime
Managed by M
icrosoft
You
scal
e, m
ake
resil
ient
& m
anag
e
Platformas a Service
Scale, Resilience and m
anagement by M
icrosoft
You
man
age
Storage
Servers
Networking
O/S
Middleware
Virtualization
Applications
Runtime
Data
On PremisesPhysical / Virtual
You
scal
e, m
ake
resil
ient
and
man
age
Storage
Servers
Networking
O/S
Middleware
Virtualization
Data
Applications
Runtime
Softwareas a Service
Storage
Servers
Networking
O/S
Middleware
Virtualization
Applications
Runtime
Data
Scale, Resilience and m
anagement by M
icrosoft
Windows AzureVirtual Machines
Windows AzureCloud Services
Virtual Machine components
Compute Storage Networking
Flexible, Scalable CPU and memory
Standard and Premium Storage tiers
Internal/External Networking
Azure VM VM hosted on Microsoft Azure Infrastructure (“IaaS”)
• From Microsoft images (gallery) or your own images (custom) SQL 2008R2 / 2012 / 2014 / 2016 Web / Standard / Enterprise
Images refreshed with latest version, SP, CU• Fast provisioning (~10 minutes). Provision groups of servers with resource templates• Accessible via RDP and Powershell• Full compatability with SQL Server “Box” software
Pay per use• Per minute (only when running)• Cost depends on size and licensing• EA customers can use existing SQL licenses (BYOL)
• Network: only outgoing (not incoming)• Storage: only used (not allocated)
Elasticity• 1 core / 2 GB mem / 1 TB 32 cores / 448 GB mem / 64 TB
Windows Azure virtual machine tiers
Basic Standard
A0 – A41 – 8 CPU cores768 MB – 14 GB RAMMax 16 datadisks w/300 IOPS per disk
For dev/test workloads or applications that don’t require load-balancing, auto-scaling, or memory-intensive VM’s.
G series (G1 – G5)2 – 32 CPU cores28 GB – 448 GB RAMUp to 64 datadisks with 500 IOPS/disk
GS Series (GS1 – GS5)2 – 32 CPU cores28 GB – 448 GB RAMUp to 64 datadisks with 5000 – 80000 IOPS/disk
F SeriesWeb/Application servers
H SeriesModeling/Simulation servers
N SeriesGraphics workloads
https://azure.microsoft.com/en-us/documentation/articles/virtual-machines-size-specs/
A0 – A111 – 16 CPU cores768 MB – 112 GB RAMMax 16 datadisks with 500 IOPS/disk
D1 – D14 & D1_v2 – D15_v21 – 20 CPU cores768 MB – 140 GB RAMMax 16 datadisks with 500 IOPS/disk
DS1 – DS14Up to 50,000 IOPS32 – 512 MB second
Compare compute performance
ACU = Azure Compute Unit
Microsoft Azure VMs - Performance Different VM Options
A-Series: Slowest CPU, least memory (A8-A11 compute intensive) D-Series: Faster CPU, more memory. Non-persistent SSD drive (good for TempDB) Dv2-Series: 35% faster CPU than D-Series DS-Series: Same CPU and memory as D-Series. Support Premium Storage (good for Data, Log,
and TempDB!!) G-Series: Fastest CPU, most memory. GS-Series: Fastest CPU, most memory. Support Premium Storage
G5: 32 cores, 448 GB mem, 6 TB SSD, 64 1TB disks - Biggest VM in the market
Different Storage Options Standard Storage: Low throughput (max 500 IOPs p/disk), High latency (avg 40ms), pay for
used space Premium Storage: High throughput (max 5000 IOPs p/disk), Low latency (avg 4ms), pay for
allocated spaceAzure calculator: https://azure.microsoft.com/en-us/pricing/calculator/
SQL Server in Azure VMs – Performance• Premium Storage
High throughput (max 5000 IOPs p/disk) Low latency (avg 4ms) SSD disks in the backend Uses local SSD disk as read cache
Avg 1ms read latency Cache hits not part of max IOPs/bandwidth
VM Size
CPU cores
Memory
Cache size Max. IOPS
Max. Storage Bandwidth
DS1 1 3.5 GB 43 GB 3,200 32 MB/sDS2 2 7 GB 86 GB 6,400 64 MB/sDS3 4 14 GB 172 GB 12,800 128 MB/sDS4 8 28 GB 344 GB 25,600 256 MB/sDS11 2 14 GB 72 GB 6,400 64 MB/sDS12 4 28 GB 144 GB 12,800 128 MB/sDS13 8 56 GB 288 GB 25,600 256 MB/sDS14 16 112 GB 576 GB 50,000 512 MB/s
Disk Type
Disk Size
Storage IOPS
Storage Bandwidth (MB/s)
P10 128 GB 500 100
P20 512 GB 2300 150
P30 1024 GB 5000 200
GS5 32 448 GB 4,608 GB 100,000 2048 MB/s
VM Gallery Images via Azure MarketplaceCertified pre-configured software images (1087)
Azure Quickstart TemplatesFree community contributed templates (417)
Storage configuration
Automatically creates one Windows storage space (virtual drive) across all disks
Services by region
Hyper scale Infrastructure is the enabler34 Regions Worldwide, 30 Generally Available…
100+ datacenters Top 3 networks in the world 2.5x AWS, 7x Google DC Regions G Series – Largest VM in World, 32 cores, 448GB Ram, SSD…
OperationalAnnounced/Not Operational
Central US
Iowa
West USCaliforni
a
East USVirginia
US GovVirginia
North Central US
Illinois
US GovIowa
South Central US
Texas
Brazil SouthSao Paulo
State
West Europe
Netherlands
China North *
BeijingChina
South *Shanghai
Japan EastTokyo,
Saitama
Japan West
OsakaIndia South
Chennai East AsiaHong Kong
SE AsiaSingapo
re
Australia South East
Victoria
Australia EastNew South
Wales
India CentralPune
Canada EastQuebec City
Canada CentralToronto
India West
Mumbai
Germany North East **
Magdeburg
Germany Central **Frankfurt
North EuropeIreland
East US 2
Virginia
United KingdomRegions (south &
west)
US DoD EastTBD
US DoD WestTBD
* Operated by 21Vianet ** Data Stewardship by Deutsche Telekom
Korea (Seoul
& South)
West US 2
West Central
US
Migrating DataMigrate from on-prem SQL server to Azure VM IaaS: • Use the Deploy a SQL Server Database to a Microsoft Azure VM wizard. Recommended method for migrating an on-
premises user database when the compressed database backup file is less than 1 TB. Use on SQL Server 2005 or greater to SQL Server 2014 or greater
• Perform on-premises backup using compression and manually copy the backup file into the Azure virtual machine and then do a restore (only if you cannot use the above wizard or the database backup size is larger than 1 TB). Use on SQL Server 2005 or greater to SQL Server 2005 or greater
• Perform a backup to URL and restore into the Azure virtual machine from the URL. Use on SQL Server 2012 SP1 CU2 or greater to SQL Server 2012 SP1 CU2 or greater
• Detach and then copy the data and log files to Azure blob storage and then attach to SQL Server in Azure VM from URL. Use on SQL Server 2005 or greater to SQL Server 2014 or greater
• Convert on-premises physical machine to Hyper-V VHD, upload to Azure Blob storage, and then deploy as new VM using uploaded VHD. Use when bringing your own SQL Server license, when migrating a database that you will run on an older version of SQL Server, or when migrating system and user databases together as part of the migration of database dependent on other user databases and/or system databases. Use on SQL Server 2005 or greater to SQL Server 2005 or greater
• Ship hard drive using Windows Import/Export Service. Use when manual copy method is too slow, such as with very large databases. Use on SQL Server 2005 or greater to SQL Server 2005 or greater
• If you have an AlwaysOn deployment on-premises and want to minimize downtime, use the Add Azure Replica Wizard to create a replica in Azure and then failover, pointing users to the Azure database instance. Use on SQL Server 2012 or greater to SQL Server 2012 or greater
• If you do not have an AlwaysOn deployment on-premises and want to minimize downtime, use SQL Server transactional replication to configure the Azure SQL Server instance as a subscriber and then disable replication, pointing users to the Azure database instance. Use on SQL Server 2005 or greater to SQL Server 2005 or greater
• Others: data-tier application, transact-SQL scripts, sql server import and export wizard, SSIS, copy database wizard
Scale VMs
Scale VMs• Be aware a region may not support the VM
size• Resize requires just a VM reboot if in same
family or if Azure hardware cluster supports new VM size
• If hardware cluster does not support new VM size: • If using Resource Manager (ARM)
deployment model you can resize VMs by first stopping your VM, selecting a new VM size and then restarting the VM
• If using Classic (ASM) deployment model, VMs must be deleted and then recreated using the same OS and data disks. See PowerShell script
Microsoft Azure VMs – ManagementAutomated Patching• Predictable solution for patching (Windows & SQL)
• Simple: just specify a time window• Uses SQL Agent Extension and MS Update• Portal and Powershell• It relies on the Windows Update and the
Microsoft Update infrastructure and installs any update that matches the ‘Important’ category for the machine
Microsoft Azure VMs – ManagementAutomated Backup• For all DBs in the SQL instance• Simple: just specify a retention period• Supports Compression and Encryption• Portal and Powershell• Full database and transaction log backups• Configure at database level or SQL Server instance
level
In SQL Server 2016:• Full, bulk-logged and simple recovery models are
all supported• System databases can be configured for backups• Backup striping can be used to support backup
sizes of up to 12 TB• Customer backup schedules can be specified to
ensure your backups are created when it is best for your workload
Licensing/bring your own license
Obtain SQL image from Azure VM
gallery
pay per use
Install or upload your
own SQL Server Image
Connecting to Azure VMs
Virtual Network
VPN GW
Frontend10.1/16
Mid-tier10.2/16
Backend10.3/16
Internet
On Premises10.0/16
VPN &ExpressRoute
AD / DNS
Azure
Direct InternetConnectivity
Virtual Machine networking• Create subnets with private or public IP addresses• Bring your own DNS or use Azure-provided DNS• Secure with Network Security Groups ACLs• Control traffic with user-defined routes
Connectivity options and hybrid offeringsCloud Customer Segment and
workloads
Secure site-to-site VPN connectivity
• SMB, Enterprises• Connect to Azure
compute
Secure point-to-site connectivity
• Developers• POC Efforts• Small scale
deployments• Connect from
anywhere
ExpressRoute private connectivity
• SMB, Enterprises• Mission critical workloads• Backup/DR, media, HPC• Connect to Microsoft
services
Internet connectivity
• Consumers• Access over public IP• DNS resolution• Connect from anywhere
Virtual Machine storage architecture
C:\OS disk (127 GB)
Usually 115 GB free
E:\, F:\, etc.Data disks (1 TB)Attach SSD/HDD up to 1TB.
These are .vhd files
D:\Temporary disk
(Contents can be lost)
SSD/HDD and size depends on VM chosenDisk Cache
Azure Blob storage
Azure Default Blob Storage Azure Storage Page Blobs, 3 copies Storage high durability built-in
(like have RAID) VHD disks, up to 1 TB per disk (64 TB
total)
Geo-storage replication 3 copies locally, another 3 copies in
different region Disable for SQL Server VM disk
(consistent write order across multiple disks is not guaranteed). Instead use DR techniques in this deck
Defend against regional disasters
Geo replication
Blog Storage Replication Options
LRS = Locally Redundant Storage (synchronous, same facility)ZRS = Zone-Redundant Storage (synchronous,, multiple facilities)GRS = Geo-Redundant Storage (asynchronous, multiple regions – secondary region chosen for you)RA-GRS = Read-Access Geo-Redundant Storage (readable)
HA/DR pieces• Virtual Machine• Application• Storage• Network• SQL Server (main focus of this deck)
HA/DR deployment architecturesAlwaysOn Failover Cluster
Instances (FCI)
AlwaysOn Availability
Groups
Database Mirroring
Log Shipping Backup to Azure (blob
storage)
Azure Site Recovery
Azure Only Availability replicas running across multiple datacenters in Azure VMs for disaster recovery.Cross-region solution protects against complete site outage.Hybrid Some availability replicas running in Azure VMs and other replicas running on-premises for cross-site disaster recovery.
HA only, not DR FCI on a two-node WSFC running in Azure VMs with storage supported by a third-party clustering solution.FCI on a two-node WSFC running in Azure VMs with remote iSCSI Target shared block storage via ExpressRoute.
Azure OnlyPrincipal and mirror and servers running in different datacenters for disaster recovery. Principal, Mirror, and Witness run within same Azure data center, deployed using a DC or server certificates for HA.HybridOne partner running in an Azure VM and the other running on-premises for cross-site disaster recovery using server certificates.
For DR only / Hybrid onlyOne server running in an Azure VM and the other running on-premises for cross-site disaster recovery.Log shipping depends on Windows file sharing, so a VPN connection between the Azure virtual network and the on-premises network is required.Requires AD deployment on DR site.
On-prem or Azure production databases backed up directly to Azure blob storage for disaster recovery.
SQL 2016: Backup to Azure with file snapshots
Simpler BCDR storySite Recovery makes it easy to handle replication, failover and recovery for your on-premises workloads and applications (not data!).Flexible replication You can replicate on-premises servers, Hyper-V virtual machines, and VMware virtual machines. Eliminate the need for secondary
SQL Server data files in
AzureNative support for SQL Server data files stored as Azure blobs
RPO/RTORTO – Recover Time Objective. How much time after a failure until we have to be up and running again?RPO – Recover Point Objective. How much data can we lose?• HA – High Availability• RTO: seconds to minutes• RPO: Zero to seconds• Automatic failover• Well tested (maybe with each patch or
release)• DR – Disaster Recovery• RTO: minutes to hours• RPO: seconds to minutes • Manual failover into prepared
environment• Tested from time to time
How long does it take to fail over:• Backup-Restore: Hours• Log Shipping: Minutes• AlwaysOn FCI: Seconds to
minutes• AlwaysOn AG/Mirroring:
Seconds
AlwaysOn Availability GroupsAlwaysOn Availability
GroupsAzure Only Availability replicas running across multiple datacenters in Azure VMs for disaster recovery.Cross-region solution protects against complete site outage.
HybridSome availability replicas running in Azure VMs and other replicas running on-premises for cross-site disaster recovery.
Availability replicas running across multiple datacenters in Azure VMs for disaster recovery. This cross-region solution protects against complete site outage. Within a region, all replicas should be within the same cloud service and the same VNet. Because each region will have a separate VNet, these solutions require VNet to VNet connectivity. For more information, see Configure a Site-to-Site VPN in the Azure classic portal.
All availability replicas running in Azure VMs for high availability within the same region. You need to configure a domain controller VM, because Windows Server Failover Clustering (WSFC) requires an Active Directory domain.For more information, see Configure AlwaysOn Availability Groups in Azure (GUI).
Microsoft Azure VMs – HA & DRAlwaysOn between Azure Regions• Configure AlwaysOn between VMs in different geographic regions (asynchronous)• Over secure tunnel
• Manual Failover (~15 seconds) in case of a regional failure• Test it at any time
• Use closest secondary for read workloads• Region 1: AG used instead
of FCI (synchronous)
AlwaysOn Failover Cluster Instances (FCI)
An FCI on a two-node WSFC running in Azure VMs with remote iSCSI Target shared block storage via ExpressRoute. For example, NetApp Private Storage (NPS) exposes an iSCSI target via ExpressRoute with Equinix to Azure VMs.
For third-party shared storage and data replication solutions, you should contact the vendor for any issues related to accessing data on failover.
Note that using FCI on top of Azure File storage is not supported yet, because this solution does not utilize Premium Storage. We are working to support this soon.
Always On Failover Cluster
Instances (FCI)HA only, not DR
FCI on a two-node WSFC running in Azure VMs with storage supported by a third-party clustering solution.
FCI on a two-node WSFC running in Azure VMs with remote iSCSI Target shared block storage via ExpressRoute.
You can use FCI to host an availability replica for an availability group
FCI on a two-node WSFC running in Azure VMs with storage supported by a third-party clustering solution.
AlwaysOn FCI vs AlwaysOn AG
Database MirroringDatabase Mirroring
Azure OnlyPrincipal and mirror and servers running in different datacenters for disaster recovery. Principal, Mirror, and Witness run within same Azure data center, deployed using a DC or server certificates for HA.HybridOne partner running in an Azure VM and the other running on-premises for cross-site disaster recovery using server certificates.
Principal and mirror and servers running in different datacenters for disaster recovery. You must deploy using server certificates because an Active Directory domain cannot span multiple datacenters.
Principal, mirror, and witness servers all running in the same Azure datacenter for high availability. You can deploy using a domain controller.
You can also deploy the same database mirroring configuration without a domain controller by using server certificates instead.
Block blobsReduced storage costsSignificantly improved restore performanceMore granular control over Azure Storage
Azure Storage snapshot backupFastest method for creating backups and running restoresSupport of SQL Server database files on Azure Blob Storage
Backup to AzureBackup to
Azure (blob storage)
Managed backupOn-prem to AzureGranular control of the backup scheduleLocal staging for faster recovery and greater network resiliencySystem database supportSimple recovery mode support
On-prem or Azure production databases backed up directly to Azure blob storage for disaster recovery.
SQL 2016: Backup to Azure with file snapshots
Production databases backed up directly to blob storage in a different datacenter for disaster recovery
On-premises production databases backed up directly to Azure blob storage for disaster recovery.
SQL Server in Azure VM Best Practices
https://azure.microsoft.com/en-us/documentation/articles/virtual-machines-sql-server-performance-best-practices/
VM size• DS3 or higher for SQL Enterprise edition• DS2 or higher for SQL Standard and Web editionStorage• Use Premium Storage• Keep the storage account and SQL Server VM in the same region• Disable Azure geo-redundant storage (geo-replication) on the storage account (consistent write order across multiple disks is not guaranteed)Disks• Use a minimum of 2 P30 disks (1 for log files; 1 for data files and TempDB)• Avoid using operating system or temporary disks for database storage or logging• Enable read caching on the disk(s) hosting the data files and TempDB• Do not enable caching on disk(s) hosting the log file• Stripe multiple Azure data disks to get increased IO throughput• Format with documented allocation sizesI/O• Enable database page compression• Enable instant file initialization for data files• Limit or disable autogrow on the database• Disable autoshrink on the database• Move all databases to data disks, including system databases• Move SQL Server error log and trace file directories to data disks• Setup default backup and database file locations• Enable locked pages• Apply SQL Server performance fixes
© 2016 Microsoft Corporation. All rights reserved.