Azure上のデータベース機能の選び方。KVSからDWHまで

第36回 PaaS勉強会

Microsoft アカウント

Microsoft Azure 無料サブスクリプション

Visual Studio 無料サブスクリプション

Python SDK Windows版のインストール

Python SDK Mac のインストール

Microsoft Developer Network

Azure コマンドラインインタフェースのインストール

AzCopy コマンドラインツール (Azure Storage 用)

ストレージエクスプローラーWindows版のインストール

ストレージエクスプローラーMac版のインストール

Platform Services

Infrastructure Services

Web Apps

MobileApps

APIManagement

API Apps

Logic Apps

Notification Hubs

Content DeliveryNetwork (CDN)

Media Services

BizTalkServices

HybridConnections

Service Bus

StorageQueues

HybridOperations

Backup

StorSimple

Azure SiteRecovery

ImportExport

SQL Database

DocumentDB

RedisCache

AzureSearch

StorageTables

DataWarehouse

Azure AD Health Monitoring

AD PrivilegedIdentity Management

OperationalAnalytics

Cloud Services

BatchRemoteApp

ServiceFabric

Visual Studio

AppInsights

Azure SDK

VS Online

Domain Services

HDInsight MachineLearning

StreamAnalytics

Data Factory

EventHubs

MobileEngagement

Data Lake

IoT Hub

Data Catalog

Security amp Management

Azure ActiveDirectory

Multi-FactorAuthentication

Automation

Portal

Key Vault

StoreMarketplace

VM Image Galleryamp VM Depot

Azure ADB2C

Scheduler

The Azure Platform

様々なアプリ開発が行われています

Web amp mobile Business apps Microservice apps

Development amp test Big data amp analytics Internet of Things

Backup recovery

amp archive

High Performance

ComputingDigital media

Identity amp security

Mediaamp CDN

Integration Developerservices

Data

Analytics amp IoT

MY APP

とマイクロソフトさんはいうけれど

デバイス

MachineLearning

StreamAnalytics

SQL Database

Azure Storage

HDInsight(Hadoop)

Event Hubs

BIツール（Power BI など）

機器

制御装置

Stream AnalyticsData Factory

Data LakeStore

SQL

Data

Warehouse

業務システム

Machine Learning

API

IoT Hub

DocumentDB

Data Lake

Analytics service

Revolution R

Enterprise

Recommendations

customer churn

forecasting etc

Face vision Speech text

Cognitive Services

デバイス

MachineLearning

StreamAnalytics

SQL Database

Azure Storage

HDInsight(Hadoop)

Event Hubs


機器

制御装置


Data LakeStore

SQL

Data

Warehouse

業務システム

Machine Learning

API

IoT Hub

DocumentDB

Data Lake

Analytics service

Revolution R

Enterprise

Recommendations

customer churn

forecasting etc


Cognitive Services

①大量データの受け入れ

②リアルタイム処理データの集約

③データの蓄積

構造化非構造化文書など様々な形式での保存

④データの加工移行

⑥機械学習

⑦Hadoop解析

⑧マイクロソフト技術を用いた分散解析

⑨通常のRMSが機能を追加した企業向けRを使用した解析

⑪クラウドベースの Self-Service BI 機能

⑫機械学習の予測モデルを業務システムなどから呼び出し可能にするサービス

外部クラウドなどからのデータ取り込み

②リアルタイム処理データの加工

⑤ディープラーニング

⑩Excelを用いた可視化

Azure Storage様々な文書Blobデータテーブル形式のデータが保存可能なストレージ

SQL DatabaseSQL Server 2016 と同等の機能を提供する PaaS 型データベースサービス

Azure Database for MySQLPostgreSQLPaaS 型データベースサービスのMySQLPostgreSQL版

Azure SQL Data Warehouseペタバイト単位のストレージを有した PaaS 型データウェアハウスサービス

Cosmos DBマルチデータモデルのグローバル分散DB

Azure Data Lake Storeビッグデータの分析ワークロードに対応する大規模データ格納エリア

補足資料各種データストアの選び方（英語）

本日のご紹介内容

IoT Hub

Event Hubs

Data Warehouse

Data Factory

App Insights Log Analytics

Monitor

CDN

Search Power BI

Blob Storage Pillars

Open amp

Interoperable

Manageable amp

Cost Efficient

Scalable amp

Performant

Secure amp

Compliant

Durable amp

Available

Machine Learning

Stream Analytics

Batch

Functions

Data Lake Analytics

Azure HDInsight

Big Data Use Cases

Ingest amp ETL Streaming Analytics amp Machine Learning Data Aggregation Presentation

QueuesReliable queues at

scale for cloud

services

ファイルを保存する際にはこの3つの選択肢があります

httpsdocsmicrosoftcomja-jpazurestoragestorage-decide-blobs-files-disks

URL記述 httpltstorage accountgttablecorewindowsnetlttablegt

フルマネージドの RDBMS

データベース

RDBMS

OS

Azure が管理

利用者が管理

VM上の RDBMS

ホスト OS

仮想化

フルコントロール

データベース

RDBMS

OS

仮想化

ホスト OS

容易な管理

Azure Database

SQL Server物理マシン仮想マシン

SQL Server on Azure VM仮想マシン

SQL Database Data WarehousePaaS

物理マシン仮想マシン

VM Template Market place仮想マシン

Azure Database for MySQL PostgreSQL

Azure Database for MySQLPostgreSQL 登場

bull High Availability

bull 使用したリソースに応じた課金（Pay-as-you-go）

bull 無停止でのスケール変更

bull セキュリティ

bull 自動バックアップ + Point-In-Time-Restore （最大35日）

bull Azure Web Apps との統合

パブリックプレビュー開始

論理データベース

Write

Write Ack

Ack

Read

value write

Ack

特徴

無停止フェールオーバー

無停止スケールアップ(プラン変更)

自動パフォーマンスチューニング

スループット指定によるプロビジョニング

データベーストランザクションユニット (DTU) とエラスティックデータベーストランザクションユニット (eDTU) の説明

httpsdocsmicrosoftcomja-jpazuresql-databasesql-database-what-is-a-dtu

Azure SQL Database Query Performance Insight

httpsdocsmicrosoftcomja-jpazuresql-databasesql-database-query-performance

Query Store is turned on by default for Azure SQL Database

httpsazuremicrosoftcomja-jpupdatesquery-store-on-is-the-new-default-for-azure-sql-database

Queries

SQL Database

Deeper Insight

Query

Store

SQL Database Advisor

httpsdocsmicrosoftcomja-jpazuresql-databasesql-database-advisor

Azure Portal を使用した SQL Database Advisor

httpsdocsmicrosoftcomja-jpazuresql-databasesql-database-advisor-portal

領域改善後

新しいインデックス提案が作成されるまで 18時間以内

T-SQL文が実行されるまでのタイムラグ即時（数分以内）

「悪い」チューニング内容に対しての検知時間 1時間以内

次のインデックス提案が行われるまでの待ち時

間即時（数分以内）

導入までの総時間 1日以内

SQL Database の脅威の検出

httpsdocsmicrosoftcomja-jpazuresql-databasesql-database-threat-detection

Flexible

PostgreSQL PREVIEWMySQL PREVIEWP

ow

er

BI A

pp

Serv

ices

Data

Fact

ory

A

naly

tics

ML

Co

gn

itiv

e B

othellip

Global Azure with 38 Regions

Azure Compute

SQL Data

Warehouse

Azure Storage

SQL Database

Flexible On-demand scaling Resource governance

Trusted HADR BackupRestore Security Audit Isolation

Intelligent Advisors Tuning Monitoring

Azure Database

Services Platform

コントロールノード

SQLDB

Azure Blob Storage(WASB)

100 DWU lt gt 2000 DWUコントロールノード

ユーザーからの接続コンピュートとストレージの調整

コンピュートノード




SQLDB

SQLDB

SQLDB

SQLDB



SQLDB コントロールノード

D51

D52

D53

D60

hellip

D41

D42

D43

D50

hellip


D31

D32

D33

D40hellip

D21

D22

D23

D30

hellip

D11

D12

D13

D20

hellip

D1

D2

D3

D10

hellip

SQL DB





D51

D52

D53

D60

hellipコンピュートノード

D41

D42

D43

D50

hellip


D31

D32

D33

D40hellip


D21

D22

D23

D30

hellip


D11

D12

D13

D20

hellip


D1

D2

D3

D10

hellip

SQL DB

SQL DB

SQL DB

SQL DB

SQL DB

SQL DB

Azure Cosmos DBMicrosoft のグローバル分散データベースサービス

bull

bull

bull

bull

bull

ターンキー方式のグローバル分散

ストレージとスループット水平スケールアウトしあらゆるニーズに対応

数百万トランザクション秒

PB 級のデータ

数百トランザクション秒

GB 級のデータ

Cosmos DB

9 PM PST

少ないスループット

多くのスループット



11 PM PST11 PM PST

Read lt 2 ms

Writes lt 6 ms

Read lt 10 ms

Writes lt 15 ms

9950

キーバリュー

ドキュメント

グラフ

SQL (DocumentDB)

MongoDB

Azure Table Storage

Gremlin グラフ

Cosmos DB はマルチモデル型データベース従来の

MySQL

PostgreSQL（LAMP）

ドキュメント指向DB 型（MongoDB互換）

グラフ型（Gremlin互換）

SQL

ドキュメント指向DB 型Column Family（Document DB）

キーバリューストア（Azure Table）

Azure Storage

Standard Table API

01

Strong

Bounded

Staleness

Session

Consistent

Prefix

Eventual

アプリ

パフォーマンス高可用性

Navigating CAP theorem

Consistent data worldwide

Strong Bounded-stateless Session Consistent prefix Eventual

45

データ無制限のストレージと分析プラットフォーム

無制限のストレージ（エクサバイト超）

ネイティブ HDFS に基づく実装

HDInsight Hortonworks Cloudera との統合

全ての HDFS 準拠プロジェクトとの親和性

(Spark Storm Flume Sqoop Kafka R etc)

Azure Data Lake Store- Hadoop Distributed File System (HDFS) For The Cloud

構造化データと非構造化データに対するデータ処理

宣言型の SQL 文とカスタムコード (UDF UDT UDO hellip )

ローカルとリモートに対するクエリ

バッチ型インタラクティブ型ストリーミング型(in the future)

Azure Data Lake Analytics- New Query Language U-SQL

ETL パイプライン

スキーマの定義Relational

LOB

Applications

Schema-on-Write

直ちに必要でないデータは全て捨てられるかアーカイブされている

無限に保存分析結果の参照

あらゆるデータ

ソースからの

データ収集

Schema-on-Read

全データ取込要件にかかわらず

全データ格納そのままのフォーマットでスキーマ定義なし

分析実行Hadoopのような分析エンジンの利用

インタラクティブクエリ

バッチ型クエリ

機械学習

データウェアハウス

リアルタイム分析

リレーショナル

デバイス

観察仮説集まったデータから仮説をたてる

49

無制限のサイズ

ギガバイトからペタバイトのサイズの単一ファイル

巨大なスループットを必要とする大規模な分析システムを実現

あらゆるスループットに自動最適化

ペタバイト超のデータの並列コンピューティング

同一リージョン内に３つのコピーを自動的に作成

インストールやチューニングは不要

数分で稼働開始

必要に応じてどのようなサイズにもスケールアウト可能

データウェアハウスデータレイクストア

非構造Rawデータ（ログ文書バイナリ）

Tier 1

MachineLearning

Data LakeAnalytics HDInsight

目的に合わせた

データウェハウス分析処理

（売上分析経営分析）

データウェハウスに格納しきれなかった

過去データ関連データを連結して分析

ハードウェアの性能限界によって行えなかった処理量の分析を実現

ディスクコンピューティング

非構造データの

分析処理

（画像音声映像など）

広範囲のデータを利用した

機械学習統計解析処理

などによる高度分析

SQL Data Warehouse


Data LakeAnalytics

構造化データ半構造データ

分析用に加工(csv等)

Tier 2

アーカイブデータTier 2 DWHのアーカイブ

Tier 3

Analytics

Storage

HDInsight(ldquomanaged clustersrdquo)

Azure Data Lake Analytics

Azure Data Lake Store

Azure Data Lake

LOB Applications

SocialDevices

Clickstream

Sensors

Video

Web

Relational

HDInsight

Data Lake Analytics

Machine Learning

Spark

R



Azure Storage Blob

外部表Azure SQLData Warehouse

HDInsight

開発者は OSS に馴染んでいる Java Eclipse Hive etc

マネージド Hadoop クラスタにおいてカスタマイズ管理が可能でフレキシビリティがある


C SQL amp PowerShell のスキル

経験を活かせる

利便性効率性自動スケール

ジョブ実行の仕組みが提供される

(New)

(New)

一般的なパターン

処理読み込み保存

INSERT

OUTPUT

OUTPUT

SELECThellip

FROMhellip

WHEREhellip

+

PythonR

EXTRACT

EXTRACT

SELECT

SELECT

Azure

Data

Lake

Azure

Data

Lake

Azure

SQL

DB

Azure

Storage

Blobs

Azure

Storage

Blobs

RowSet RowSet

Cognitive Services

REFERENCE ASSEMBLY WebLogExtASM

rs =

EXTRACT

UserID string

Start DateTime

End DateTime

Region string

SitesVisited string

PagesVisited string

FROM ldquoLogsWebLogRecordstxtrdquo

USING WebLogExtractor ()

result = SELECT UserID

(EndSubtract(Start))TotalSeconds AS Duration

FROM rs ORDER BY Duration DESC FETCH 10

OUTPUT result TO ldquoLogsResultstop10tsv

USING OutputterTsv()

bull 型定義は C の型定義と同じ

bull データをファイルから抽出読み込み

するときにスキーマが必要

Data Lake Store 内のファイル

独自形式を解析するカスタム関数

C の関数

行セット（中間テーブルの概念

に近い）

TSV形式で書き込む関数

U-SQL ジョブのサブミットAzure PortalやVisual StudioのAzure Data Lake ToolsからU-SQLジョブをサブミット可能

58

進捗

統計情報

従来型の処理分析 Azure Data Lake を中心とした処理分析

Business apps

Custom apps

Sensors and devices

ADL StorePeople

非構造化データも含めてあらゆるデータを格納

Azure SQL DW

Azure AD

Power BI

ADF

ADL Analytics

bull 処理分析業務の大半はデータ準備作業が占めるbull 処理分析業務に手間時間が必要

Business apps

Custom apps

Sensors and devices

HDInsight

ユーザー管理認証

データの連携

Power BI

File System

Database

Database

Hadoop

DWH

Data Mart


Microsoft data platform solutions

Product Category Description More Info

SQL Server 2016 RDBMS Earned top spot in Gartnerrsquos Operational Database magic

quadrant JSON support Linux TBD

httpswwwmicrosoftcomen-usserver-

cloudproductssql-server-2016

SQL Database RDBMSDBaaS Cloud-based service that is provisioned and scaled quickly

Has built-in high availability and disaster recovery JSON

support

httpsazuremicrosoftcomen-

usservicessql-database

SQL Data Warehouse MPP RDBMSDBaaS Cloud-based service that handles relational big data

Provision and scale quickly Can pause service to reduce cost


usservicessql-data-warehouse

Analytics Platform System (APS) MPP RDBMS Big data analytics appliance for high performance and

seamless integration of all your data


cloudproductsanalytics-platform-system

Azure Data Lake Store Hadoop storage Removes the complexities of ingesting and storing all of your

data while making it faster to get up and running with batch

streaming and interactive analytics


usservicesdata-lake-store

Azure Data Lake Analytics On-demand analytics job

serviceBig Data-as-a-

service

Cloud-based service that dynamically provisions resources so

you can run queries on exabytes of data Includes U-SQL a

new big data query language


usservicesdata-lake-analytics

HDInsight PaaS Hadoop

computeHadoop

clusters-as-a-service

A managed Apache Hadoop Spark R HBase Kafka and

Storm cloud service made easy


usserviceshdinsight

DocumentDB PaaS NoSQL Document

Store

Get your apps up and running in hours with a fully managed

NoSQL database service that indexes stores and queries

data using familiar SQL syntax


usservicesdocumentdb

Azure Table Storage PaaS NoSQL Key-value

Store

Store large amount of semi-structured data in the cloud httpsazuremicrosoftcomen-

usservicesstoragetables

Microsoft Big Data Portfolio

SQL Server Stretch

Business intelligence

Machine learning analytics

Insights

Azure SQL Database

SQL Server 2016

SQL Server 2016 Fast Track

Azure SQL DW

ADLS amp ADLA

DocumentDB

HDInsight

Hadoop

Analytics Platform System

Sequential Scale Out + AcrossScale Up

Key

Relational Non-relational

On

-pre

mis

es

Clo

ud

Microsoft has solutions covering

and connecting all four

quadrants ndash thatrsquos why SQL

Server is one of the most utilized

databases in the world

Azure

Data Lake Store

Azure

Blob StoragePurpose Optimized for big data analytics General purpose bulk storage

Use Cases Batch Interactive Streaming App backend backup data media storage for

streaming

Units of Storage Accounts Folders Files Accounts Containers Blobs

Structure Hierarchical File System Flat namespace

WebHDFS Implements WebHDFS No (WASB)

Security AD SAS keys

Storage Auto SharedFiles chunked Manually manage expansionFiles intact

Service State Generally Available PolyBase

just supported

Generally Available

Billing Pay for data stored and for IO Pay for data stored and for IO

Region Availability Two US regions (Other regions coming soon) All Azure Regions

ADL Store vs Blob Store

httpsdocsmicrosoftcomen-usazuredata-lake-storedata-lake-store-comparison-with-blob-storage

Want Hadoop

Need exact same on-prem

Need interactive streaming

MandatoryNo strong opinion

Azure Marketplace (IaaS)

bull Need all workloads exactly like on-

premises

bull Need 100 HortonworksClouderaMapR

Azure HDInsight

bull Most Hadoop workloads

bull Fully managed by Microsoft

bull Sell HDI + ADLS

bull Stickier to Microsoft than VMs

bull Can do interactive (Spark) and streaming

(StormSpark)


bull Easiest experience for admin no sense of

clusters instant scale per job

bull Easiest experience for developers Visual

StudioU-SQL (C+SQL)

bull Sell ADLA + ADLS

bull Batch workloads only

Need everything exactly like on-prem

Need core projects Yes Batch is OK

Always present ADLA if NET or Visual Studio Shop

If NET or VS shop

Azure SQL DW HDInsight Hive HDInsight Spark Azure Data Lake SQL Server (IaaS)

Volume Petabytes Petabytes Petabytes Petabytes Terabytes

Security Encryption TD

Audit

ADLS Apache

Ranger

ADLS AAD Security

Groups (data)

Encryption TD

Audit

Languages T-SQL HiveQL SparkSQL HiveQL

Scala Java Python

R

U-SQL T-SQL

Extensibility No Yes NETSerDe Yes Packages Yes NET Yes NET CLR

External File

Types

ORC TXT

Parquet RCFile

ORC CSV Parquet

+ others

Parquet JSON

Hive + others

Many ORC TXT Parquet

RCFile

Admin Low-Medium Medium-High Medium-High Low High

Cost Model DWU Nodes amp VM Nodes amp VM UnitsJobs VM

Schema

Definition

Schema on Write

Polybase

Schema on Read Schema on Read Schema on Read Schema on Write

Polybase

Max DB Size 240TB Comp (5X

= 1PB)

Unlimited 64TB (64 1TB drives)

Big Data Solutions Decision Tree

httpsbiz-excellencecom20160830big-data-dt

Machine Learning Solutions Decision Tree

httpsbiz-excellencecom20160913machine-

learning-dt

Microsoft アカウント

Microsoft Azure 無料サブスクリプション

Visual Studio 無料サブスクリプション

Python SDK Windows版のインストール

Python SDK Mac のインストール

Microsoft Developer Network

Azure コマンドラインインタフェースのインストール

AzCopy コマンドラインツール (Azure Storage 用)

ストレージエクスプローラーWindows版のインストール

ストレージエクスプローラーMac版のインストール

Platform Services


Web Apps

MobileApps

APIManagement

API Apps

Logic Apps

Notification Hubs


Media Services

BizTalkServices

HybridConnections

Service Bus

StorageQueues

HybridOperations

Backup

StorSimple

Azure SiteRecovery

ImportExport

SQL Database

DocumentDB

RedisCache

AzureSearch

StorageTables

DataWarehouse




Cloud Services

BatchRemoteApp

ServiceFabric

Visual Studio

AppInsights

Azure SDK

VS Online

Domain Services


StreamAnalytics

Data Factory

EventHubs

MobileEngagement

Data Lake

IoT Hub

Data Catalog




Automation

Portal

Key Vault

StoreMarketplace


Azure ADB2C

Scheduler

The Azure Platform




Backup recovery

amp archive

High Performance



Mediaamp CDN


Data

Analytics amp IoT

MY APP


デバイス

MachineLearning

StreamAnalytics

SQL Database

Azure Storage

HDInsight(Hadoop)

Event Hubs


機器

制御装置


Data LakeStore

SQL

Data

Warehouse

業務システム

Machine Learning

API

IoT Hub

DocumentDB

Data Lake

Analytics service

Revolution R

Enterprise

Recommendations

customer churn

forecasting etc


Cognitive Services

デバイス

MachineLearning

StreamAnalytics

SQL Database

Azure Storage

HDInsight(Hadoop)

Event Hubs


機器

制御装置


Data LakeStore

SQL

Data

Warehouse

業務システム

Machine Learning

API

IoT Hub

DocumentDB

Data Lake

Analytics service

Revolution R

Enterprise

Recommendations

customer churn

forecasting etc


Cognitive Services






⑥機械学習

⑦Hadoop解析

















IoT Hub

Event Hubs

Data Warehouse

Data Factory


Monitor

CDN

Search Power BI


Open amp

Interoperable

Manageable amp

Cost Efficient

Scalable amp

Performant

Secure amp

Compliant

Durable amp

Available

Machine Learning

Stream Analytics

Batch

Functions

Data Lake Analytics

Azure HDInsight

Big Data Use Cases



scale for cloud

services





データベース

RDBMS

OS

Azure が管理

利用者が管理

VM上の RDBMS

ホスト OS

仮想化


データベース

RDBMS

OS

仮想化

ホスト OS

容易な管理

Azure Database
















Write

Write Ack

Ack

Read

value write

Ack

特徴











Queries

SQL Database

Deeper Insight

Query

Store





領域改善後









Flexible


ow

er

BI A

pp

Serv

ices

Data

Fact

ory

A

naly

tics

ML

Co

gn

itiv

e B

othellip


Azure Compute

SQL Data

Warehouse

Azure Storage

SQL Database




Azure Database

Services Platform


SQLDB








SQLDB

SQLDB

SQLDB

SQLDB




D51

D52

D53

D60

hellip

D41

D42

D43

D50

hellip


D31

D32

D33

D40hellip

D21

D22

D23

D30

hellip

D11

D12

D13

D20

hellip

D1

D2

D3

D10

hellip

SQL DB





D51

D52

D53

D60


D41

D42

D43

D50

hellip


D31

D32

D33

D40hellip


D21

D22

D23

D30

hellip


D11

D12

D13

D20

hellip


D1

D2

D3

D10

hellip

SQL DB

SQL DB

SQL DB

SQL DB

SQL DB

SQL DB


bull

bull

bull

bull

bull




PB 級のデータ


GB 級のデータ

Cosmos DB

9 PM PST





11 PM PST11 PM PST

Read lt 2 ms

Writes lt 6 ms

Read lt 10 ms

Writes lt 15 ms

9950

キーバリュー

ドキュメント

グラフ

SQL (DocumentDB)

MongoDB

Azure Table Storage

Gremlin グラフ


MySQL




SQL



Azure Storage

Standard Table API

01

Strong

Bounded

Staleness

Session

Consistent

Prefix

Eventual

アプリ





45















LOB

Applications

Schema-on-Write




ソースからの

データ収集

Schema-on-Read






機械学習




デバイス


49












Tier 1

MachineLearning










分析処理





SQL Data Warehouse


Data LakeAnalytics



Tier 2


Tier 3

Analytics

Storage




Azure Data Lake

LOB Applications

SocialDevices

Clickstream

Sensors

Video

Web

Relational

HDInsight

Data Lake Analytics

Machine Learning

Spark

R



Azure Storage Blob


HDInsight








(New)

(New)



INSERT

OUTPUT

OUTPUT

SELECThellip

FROMhellip

WHEREhellip

+

PythonR

EXTRACT

EXTRACT

SELECT

SELECT

Azure

Data

Lake

Azure

Data

Lake

Azure

SQL

DB

Azure

Storage

Blobs

Azure

Storage

Blobs

RowSet RowSet

Cognitive Services


rs =

EXTRACT

UserID string

Start DateTime

End DateTime

Region string

SitesVisited string

PagesVisited string













C の関数


に近い）



58

進捗

統計情報


Business apps

Custom apps

Sensors and devices

ADL StorePeople


Azure SQL DW

Azure AD

Power BI

ADF

ADL Analytics


Business apps

Custom apps

Sensors and devices

HDInsight


データの連携

Power BI

File System

Database

Database

Hadoop

DWH

Data Mart










support


















service







computeHadoop





usserviceshdinsight


Store







Store




SQL Server Stretch



Insights

Azure SQL Database

SQL Server 2016


Azure SQL DW

ADLS amp ADLA

DocumentDB

HDInsight

Hadoop



Key


On

-pre

mis

es

Clo

ud






Azure

Data Lake Store

Azure



streaming







just supported

Generally Available





Want Hadoop






premises


Azure HDInsight






(StormSpark)





StudioU-SQL (C+SQL)






If NET or VS shop




Audit

ADLS Apache

Ranger

ADLS AAD Security

Groups (data)

Encryption TD

Audit


Scala Java Python

R

U-SQL T-SQL


External File

Types

ORC TXT

Parquet RCFile

ORC CSV Parquet

+ others

Parquet JSON

Hive + others


RCFile



Schema

Definition

Schema on Write

Polybase


Polybase


= 1PB)






learning-dt

Platform Services


Web Apps

MobileApps

APIManagement

API Apps

Logic Apps

Notification Hubs


Media Services

BizTalkServices

HybridConnections

Service Bus

StorageQueues

HybridOperations

Backup

StorSimple

Azure SiteRecovery

ImportExport

SQL Database

DocumentDB

RedisCache

AzureSearch

StorageTables

DataWarehouse




Cloud Services

BatchRemoteApp

ServiceFabric

Visual Studio

AppInsights

Azure SDK

VS Online

Domain Services


StreamAnalytics

Data Factory

EventHubs

MobileEngagement

Data Lake

IoT Hub

Data Catalog




Automation

Portal

Key Vault

StoreMarketplace


Azure ADB2C

Scheduler

The Azure Platform




Backup recovery

amp archive

High Performance



Mediaamp CDN


Data

Analytics amp IoT

MY APP


デバイス

MachineLearning

StreamAnalytics

SQL Database

Azure Storage

HDInsight(Hadoop)

Event Hubs


機器

制御装置


Data LakeStore

SQL

Data

Warehouse

業務システム

Machine Learning

API

IoT Hub

DocumentDB

Data Lake

Analytics service

Revolution R

Enterprise

Recommendations

customer churn

forecasting etc


Cognitive Services

デバイス

MachineLearning

StreamAnalytics

SQL Database

Azure Storage

HDInsight(Hadoop)

Event Hubs


機器

制御装置


Data LakeStore

SQL

Data

Warehouse

業務システム

Machine Learning

API

IoT Hub

DocumentDB

Data Lake

Analytics service

Revolution R

Enterprise

Recommendations

customer churn

forecasting etc


Cognitive Services






⑥機械学習

⑦Hadoop解析

















IoT Hub

Event Hubs

Data Warehouse

Data Factory


Monitor

CDN

Search Power BI


Open amp

Interoperable

Manageable amp

Cost Efficient

Scalable amp

Performant

Secure amp

Compliant

Durable amp

Available

Machine Learning

Stream Analytics

Batch

Functions

Data Lake Analytics

Azure HDInsight

Big Data Use Cases



scale for cloud

services





データベース

RDBMS

OS

Azure が管理

利用者が管理

VM上の RDBMS

ホスト OS

仮想化


データベース

RDBMS

OS

仮想化

ホスト OS

容易な管理

Azure Database
















Write

Write Ack

Ack

Read

value write

Ack

特徴











Queries

SQL Database

Deeper Insight

Query

Store





領域改善後









Flexible


ow

er

BI A

pp

Serv

ices

Data

Fact

ory

A

naly

tics

ML

Co

gn

itiv

e B

othellip


Azure Compute

SQL Data

Warehouse

Azure Storage

SQL Database




Azure Database

Services Platform


SQLDB








SQLDB

SQLDB

SQLDB

SQLDB




D51

D52

D53

D60

hellip

D41

D42

D43

D50

hellip


D31

D32

D33

D40hellip

D21

D22

D23

D30

hellip

D11

D12

D13

D20

hellip

D1

D2

D3

D10

hellip

SQL DB





D51

D52

D53

D60


D41

D42

D43

D50

hellip


D31

D32

D33

D40hellip


D21

D22

D23

D30

hellip


D11

D12

D13

D20

hellip


D1

D2

D3

D10

hellip

SQL DB

SQL DB

SQL DB

SQL DB

SQL DB

SQL DB


bull

bull

bull

bull

bull




PB 級のデータ


GB 級のデータ

Cosmos DB

9 PM PST





11 PM PST11 PM PST

Read lt 2 ms

Writes lt 6 ms

Read lt 10 ms

Writes lt 15 ms

9950

キーバリュー

ドキュメント

グラフ

SQL (DocumentDB)

MongoDB

Azure Table Storage

Gremlin グラフ


MySQL




SQL



Azure Storage

Standard Table API

01

Strong

Bounded

Staleness

Session

Consistent

Prefix

Eventual

アプリ





45















LOB

Applications

Schema-on-Write




ソースからの

データ収集

Schema-on-Read






機械学習




デバイス


49












Tier 1

MachineLearning










分析処理





SQL Data Warehouse


Data LakeAnalytics



Tier 2


Tier 3

Analytics

Storage




Azure Data Lake

LOB Applications

SocialDevices

Clickstream

Sensors

Video

Web

Relational

HDInsight

Data Lake Analytics

Machine Learning

Spark

R



Azure Storage Blob


HDInsight








(New)

(New)



INSERT

OUTPUT

OUTPUT

SELECThellip

FROMhellip

WHEREhellip

+

PythonR

EXTRACT

EXTRACT

SELECT

SELECT

Azure

Data

Lake

Azure

Data

Lake

Azure

SQL

DB

Azure

Storage

Blobs

Azure

Storage

Blobs

RowSet RowSet

Cognitive Services


rs =

EXTRACT

UserID string

Start DateTime

End DateTime

Region string

SitesVisited string

PagesVisited string













C の関数


に近い）



58

進捗

統計情報


Business apps

Custom apps

Sensors and devices

ADL StorePeople


Azure SQL DW

Azure AD

Power BI

ADF

ADL Analytics


Business apps

Custom apps

Sensors and devices

HDInsight


データの連携

Power BI

File System

Database

Database

Hadoop

DWH

Data Mart










support


















service







computeHadoop





usserviceshdinsight


Store







Store




SQL Server Stretch



Insights

Azure SQL Database

SQL Server 2016


Azure SQL DW

ADLS amp ADLA

DocumentDB

HDInsight

Hadoop



Key


On

-pre

mis

es

Clo

ud






Azure

Data Lake Store

Azure



streaming







just supported

Generally Available





Want Hadoop






premises


Azure HDInsight






(StormSpark)





StudioU-SQL (C+SQL)






If NET or VS shop




Audit

ADLS Apache

Ranger

ADLS AAD Security

Groups (data)

Encryption TD

Audit


Scala Java Python

R

U-SQL T-SQL


External File

Types

ORC TXT

Parquet RCFile

ORC CSV Parquet

+ others

Parquet JSON

Hive + others


RCFile



Schema

Definition

Schema on Write

Polybase


Polybase


= 1PB)






learning-dt




Backup recovery

amp archive

High Performance



Mediaamp CDN


Data

Analytics amp IoT

MY APP


デバイス

MachineLearning

StreamAnalytics

SQL Database

Azure Storage

HDInsight(Hadoop)

Event Hubs


機器

制御装置


Data LakeStore

SQL

Data

Warehouse

業務システム

Machine Learning

API

IoT Hub

DocumentDB

Data Lake

Analytics service

Revolution R

Enterprise

Recommendations

customer churn

forecasting etc


Cognitive Services

デバイス

MachineLearning

StreamAnalytics

SQL Database

Azure Storage

HDInsight(Hadoop)

Event Hubs


機器

制御装置


Data LakeStore

SQL

Data

Warehouse

業務システム

Machine Learning

API

IoT Hub

DocumentDB

Data Lake

Analytics service

Revolution R

Enterprise

Recommendations

customer churn

forecasting etc


Cognitive Services






⑥機械学習

⑦Hadoop解析

















IoT Hub

Event Hubs

Data Warehouse

Data Factory


Monitor

CDN

Search Power BI


Open amp

Interoperable

Manageable amp

Cost Efficient

Scalable amp

Performant

Secure amp

Compliant

Durable amp

Available

Machine Learning

Stream Analytics

Batch

Functions

Data Lake Analytics

Azure HDInsight

Big Data Use Cases



scale for cloud

services





データベース

RDBMS

OS

Azure が管理

利用者が管理

VM上の RDBMS

ホスト OS

仮想化


データベース

RDBMS

OS

仮想化

ホスト OS

容易な管理

Azure Database
















Write

Write Ack

Ack

Read

value write

Ack

特徴











Queries

SQL Database

Deeper Insight

Query

Store





領域改善後









Flexible


ow

er

BI A

pp

Serv

ices

Data

Fact

ory

A

naly

tics

ML

Co

gn

itiv

e B

othellip


Azure Compute

SQL Data

Warehouse

Azure Storage

SQL Database




Azure Database

Services Platform


SQLDB








SQLDB

SQLDB

SQLDB

SQLDB




D51

D52

D53

D60

hellip

D41

D42

D43

D50

hellip


D31

D32

D33

D40hellip

D21

D22

D23

D30

hellip

D11

D12

D13

D20

hellip

D1

D2

D3

D10

hellip

SQL DB





D51

D52

D53

D60


D41

D42

D43

D50

hellip


D31

D32

D33

D40hellip


D21

D22

D23

D30

hellip


D11

D12

D13

D20

hellip


D1

D2

D3

D10

hellip

SQL DB

SQL DB

SQL DB

SQL DB

SQL DB

SQL DB


bull

bull

bull

bull

bull




PB 級のデータ


GB 級のデータ

Cosmos DB

9 PM PST





11 PM PST11 PM PST

Read lt 2 ms

Writes lt 6 ms

Read lt 10 ms

Writes lt 15 ms

9950

キーバリュー

ドキュメント

グラフ

SQL (DocumentDB)

MongoDB

Azure Table Storage

Gremlin グラフ


MySQL




SQL



Azure Storage

Standard Table API

01

Strong

Bounded

Staleness

Session

Consistent

Prefix

Eventual

アプリ





45















LOB

Applications

Schema-on-Write




ソースからの

データ収集

Schema-on-Read






機械学習




デバイス


49












Tier 1

MachineLearning










分析処理





SQL Data Warehouse


Data LakeAnalytics



Tier 2


Tier 3

Analytics

Storage




Azure Data Lake

LOB Applications

SocialDevices

Clickstream

Sensors

Video

Web

Relational

HDInsight

Data Lake Analytics

Machine Learning

Spark

R



Azure Storage Blob


HDInsight








(New)

(New)



INSERT

OUTPUT

OUTPUT

SELECThellip

FROMhellip

WHEREhellip

+

PythonR

EXTRACT

EXTRACT

SELECT

SELECT

Azure

Data

Lake

Azure

Data

Lake

Azure

SQL

DB

Azure

Storage

Blobs

Azure

Storage

Blobs

RowSet RowSet

Cognitive Services


rs =

EXTRACT

UserID string

Start DateTime

End DateTime

Region string

SitesVisited string

PagesVisited string













C の関数


に近い）



58

進捗

統計情報


Business apps

Custom apps

Sensors and devices

ADL StorePeople


Azure SQL DW

Azure AD

Power BI

ADF

ADL Analytics


Business apps

Custom apps

Sensors and devices

HDInsight


データの連携

Power BI

File System

Database

Database

Hadoop

DWH

Data Mart










support


















service







computeHadoop





usserviceshdinsight


Store







Store




SQL Server Stretch



Insights

Azure SQL Database

SQL Server 2016


Azure SQL DW

ADLS amp ADLA

DocumentDB

HDInsight

Hadoop



Key


On

-pre

mis

es

Clo

ud






Azure

Data Lake Store

Azure



streaming







just supported

Generally Available





Want Hadoop






premises


Azure HDInsight






(StormSpark)





StudioU-SQL (C+SQL)






If NET or VS shop




Audit

ADLS Apache

Ranger

ADLS AAD Security

Groups (data)

Encryption TD

Audit


Scala Java Python

R

U-SQL T-SQL


External File

Types

ORC TXT

Parquet RCFile

ORC CSV Parquet

+ others

Parquet JSON

Hive + others


RCFile



Schema

Definition

Schema on Write

Polybase


Polybase


= 1PB)






learning-dt


Mediaamp CDN


Data

Analytics amp IoT

MY APP


デバイス

MachineLearning

StreamAnalytics

SQL Database

Azure Storage

HDInsight(Hadoop)

Event Hubs


機器

制御装置


Data LakeStore

SQL

Data

Warehouse

業務システム

Machine Learning

API

IoT Hub

DocumentDB

Data Lake

Analytics service

Revolution R

Enterprise

Recommendations

customer churn

forecasting etc


Cognitive Services

デバイス

MachineLearning

StreamAnalytics

SQL Database

Azure Storage

HDInsight(Hadoop)

Event Hubs


機器

制御装置


Data LakeStore

SQL

Data

Warehouse

業務システム

Machine Learning

API

IoT Hub

DocumentDB

Data Lake

Analytics service

Revolution R

Enterprise

Recommendations

customer churn

forecasting etc


Cognitive Services






⑥機械学習

⑦Hadoop解析

















IoT Hub

Event Hubs

Data Warehouse

Data Factory


Monitor

CDN

Search Power BI


Open amp

Interoperable

Manageable amp

Cost Efficient

Scalable amp

Performant

Secure amp

Compliant

Durable amp

Available

Machine Learning

Stream Analytics

Batch

Functions

Data Lake Analytics

Azure HDInsight

Big Data Use Cases



scale for cloud

services





データベース

RDBMS

OS

Azure が管理

利用者が管理

VM上の RDBMS

ホスト OS

仮想化


データベース

RDBMS

OS

仮想化

ホスト OS

容易な管理

Azure Database
















Write

Write Ack

Ack

Read

value write

Ack

特徴











Queries

SQL Database

Deeper Insight

Query

Store





領域改善後









Flexible


ow

er

BI A

pp

Serv

ices

Data

Fact

ory

A

naly

tics

ML

Co

gn

itiv

e B

othellip


Azure Compute

SQL Data

Warehouse

Azure Storage

SQL Database




Azure Database

Services Platform


SQLDB








SQLDB

SQLDB

SQLDB

SQLDB




D51

D52

D53

D60

hellip

D41

D42

D43

D50

hellip


D31

D32

D33

D40hellip

D21

D22

D23

D30

hellip

D11

D12

D13

D20

hellip

D1

D2

D3

D10

hellip

SQL DB





D51

D52

D53

D60


D41

D42

D43

D50

hellip


D31

D32

D33

D40hellip


D21

D22

D23

D30

hellip


D11

D12

D13

D20

hellip


D1

D2

D3

D10

hellip

SQL DB

SQL DB

SQL DB

SQL DB

SQL DB

SQL DB


bull

bull

bull

bull

bull




PB 級のデータ


GB 級のデータ

Cosmos DB

9 PM PST





11 PM PST11 PM PST

Read lt 2 ms

Writes lt 6 ms

Read lt 10 ms

Writes lt 15 ms

9950

キーバリュー

ドキュメント

グラフ

SQL (DocumentDB)

MongoDB

Azure Table Storage

Gremlin グラフ


MySQL




SQL



Azure Storage

Standard Table API

01

Strong

Bounded

Staleness

Session

Consistent

Prefix

Eventual

アプリ





45















LOB

Applications

Schema-on-Write




ソースからの

データ収集

Schema-on-Read






機械学習




デバイス


49












Tier 1

MachineLearning










分析処理





SQL Data Warehouse


Data LakeAnalytics



Tier 2


Tier 3

Analytics

Storage




Azure Data Lake

LOB Applications

SocialDevices

Clickstream

Sensors

Video

Web

Relational

HDInsight

Data Lake Analytics

Machine Learning

Spark

R



Azure Storage Blob


HDInsight








(New)

(New)



INSERT

OUTPUT

OUTPUT

SELECThellip

FROMhellip

WHEREhellip

+

PythonR

EXTRACT

EXTRACT

SELECT

SELECT

Azure

Data

Lake

Azure

Data

Lake

Azure

SQL

DB

Azure

Storage

Blobs

Azure

Storage

Blobs

RowSet RowSet

Cognitive Services


rs =

EXTRACT

UserID string

Start DateTime

End DateTime

Region string

SitesVisited string

PagesVisited string













C の関数


に近い）



58

進捗

統計情報


Business apps

Custom apps

Sensors and devices

ADL StorePeople


Azure SQL DW

Azure AD

Power BI

ADF

ADL Analytics


Business apps

Custom apps

Sensors and devices

HDInsight


データの連携

Power BI

File System

Database

Database

Hadoop

DWH

Data Mart










support


















service







computeHadoop





usserviceshdinsight


Store







Store




SQL Server Stretch



Insights

Azure SQL Database

SQL Server 2016


Azure SQL DW

ADLS amp ADLA

DocumentDB

HDInsight

Hadoop



Key


On

-pre

mis

es

Clo

ud






Azure

Data Lake Store

Azure



streaming







just supported

Generally Available





Want Hadoop






premises


Azure HDInsight






(StormSpark)





StudioU-SQL (C+SQL)






If NET or VS shop




Audit

ADLS Apache

Ranger

ADLS AAD Security

Groups (data)

Encryption TD

Audit


Scala Java Python

R

U-SQL T-SQL


External File

Types

ORC TXT

Parquet RCFile

ORC CSV Parquet

+ others

Parquet JSON

Hive + others


RCFile



Schema

Definition

Schema on Write

Polybase


Polybase


= 1PB)






learning-dt

デバイス

MachineLearning

StreamAnalytics

SQL Database

Azure Storage

HDInsight(Hadoop)

Event Hubs


機器

制御装置


Data LakeStore

SQL

Data

Warehouse

業務システム

Machine Learning

API

IoT Hub

DocumentDB

Data Lake

Analytics service

Revolution R

Enterprise

Recommendations

customer churn

forecasting etc


Cognitive Services

デバイス

MachineLearning

StreamAnalytics

SQL Database

Azure Storage

HDInsight(Hadoop)

Event Hubs


機器

制御装置


Data LakeStore

SQL

Data

Warehouse

業務システム

Machine Learning

API

IoT Hub

DocumentDB

Data Lake

Analytics service

Revolution R

Enterprise

Recommendations

customer churn

forecasting etc


Cognitive Services






⑥機械学習

⑦Hadoop解析

















IoT Hub

Event Hubs

Data Warehouse

Data Factory


Monitor

CDN

Search Power BI


Open amp

Interoperable

Manageable amp

Cost Efficient

Scalable amp

Performant

Secure amp

Compliant

Durable amp

Available

Machine Learning

Stream Analytics

Batch

Functions

Data Lake Analytics

Azure HDInsight

Big Data Use Cases



scale for cloud

services





データベース

RDBMS

OS

Azure が管理

利用者が管理

VM上の RDBMS

ホスト OS

仮想化


データベース

RDBMS

OS

仮想化

ホスト OS

容易な管理

Azure Database
















Write

Write Ack

Ack

Read

value write

Ack

特徴











Queries

SQL Database

Deeper Insight

Query

Store





領域改善後









Flexible


ow

er

BI A

pp

Serv

ices

Data

Fact

ory

A

naly

tics

ML

Co

gn

itiv

e B

othellip


Azure Compute

SQL Data

Warehouse

Azure Storage

SQL Database




Azure Database

Services Platform


SQLDB








SQLDB

SQLDB

SQLDB

SQLDB




D51

D52

D53

D60

hellip

D41

D42

D43

D50

hellip


D31

D32

D33

D40hellip

D21

D22

D23

D30

hellip

D11

D12

D13

D20

hellip

D1

D2

D3

D10

hellip

SQL DB





D51

D52

D53

D60


D41

D42

D43

D50

hellip


D31

D32

D33

D40hellip


D21

D22

D23

D30

hellip


D11

D12

D13

D20

hellip


D1

D2

D3

D10

hellip

SQL DB

SQL DB

SQL DB

SQL DB

SQL DB

SQL DB


bull

bull

bull

bull

bull




PB 級のデータ


GB 級のデータ

Cosmos DB

9 PM PST





11 PM PST11 PM PST

Read lt 2 ms

Writes lt 6 ms

Read lt 10 ms

Writes lt 15 ms

9950

キーバリュー

ドキュメント

グラフ

SQL (DocumentDB)

MongoDB

Azure Table Storage

Gremlin グラフ


MySQL




SQL



Azure Storage

Standard Table API

01

Strong

Bounded

Staleness

Session

Consistent

Prefix

Eventual

アプリ





45















LOB

Applications

Schema-on-Write




ソースからの

データ収集

Schema-on-Read






機械学習




デバイス


49












Tier 1

MachineLearning










分析処理





SQL Data Warehouse


Data LakeAnalytics



Tier 2


Tier 3

Analytics

Storage




Azure Data Lake

LOB Applications

SocialDevices

Clickstream

Sensors

Video

Web

Relational

HDInsight

Data Lake Analytics

Machine Learning

Spark

R



Azure Storage Blob


HDInsight








(New)

(New)



INSERT

OUTPUT

OUTPUT

SELECThellip

FROMhellip

WHEREhellip

+

PythonR

EXTRACT

EXTRACT

SELECT

SELECT

Azure

Data

Lake

Azure

Data

Lake

Azure

SQL

DB

Azure

Storage

Blobs

Azure

Storage

Blobs

RowSet RowSet

Cognitive Services


rs =

EXTRACT

UserID string

Start DateTime

End DateTime

Region string

SitesVisited string

PagesVisited string













C の関数


に近い）



58

進捗

統計情報


Business apps

Custom apps

Sensors and devices

ADL StorePeople


Azure SQL DW

Azure AD

Power BI

ADF

ADL Analytics


Business apps

Custom apps

Sensors and devices

HDInsight


データの連携

Power BI

File System

Database

Database

Hadoop

DWH

Data Mart










support


















service







computeHadoop





usserviceshdinsight


Store







Store




SQL Server Stretch



Insights

Azure SQL Database

SQL Server 2016


Azure SQL DW

ADLS amp ADLA

DocumentDB

HDInsight

Hadoop



Key


On

-pre

mis

es

Clo

ud






Azure

Data Lake Store

Azure



streaming







just supported

Generally Available





Want Hadoop






premises


Azure HDInsight






(StormSpark)





StudioU-SQL (C+SQL)






If NET or VS shop




Audit

ADLS Apache

Ranger

ADLS AAD Security

Groups (data)

Encryption TD

Audit


Scala Java Python

R

U-SQL T-SQL


External File

Types

ORC TXT

Parquet RCFile

ORC CSV Parquet

+ others

Parquet JSON

Hive + others


RCFile



Schema

Definition

Schema on Write

Polybase


Polybase


= 1PB)






learning-dt

デバイス

MachineLearning

StreamAnalytics

SQL Database

Azure Storage

HDInsight(Hadoop)

Event Hubs


機器

制御装置


Data LakeStore

SQL

Data

Warehouse

業務システム

Machine Learning

API

IoT Hub

DocumentDB

Data Lake

Analytics service

Revolution R

Enterprise

Recommendations

customer churn

forecasting etc


Cognitive Services






⑥機械学習

⑦Hadoop解析

















IoT Hub

Event Hubs

Data Warehouse

Data Factory


Monitor

CDN

Search Power BI


Open amp

Interoperable

Manageable amp

Cost Efficient

Scalable amp

Performant

Secure amp

Compliant

Durable amp

Available

Machine Learning

Stream Analytics

Batch

Functions

Data Lake Analytics

Azure HDInsight

Big Data Use Cases



scale for cloud

services





データベース

RDBMS

OS

Azure が管理

利用者が管理

VM上の RDBMS

ホスト OS

仮想化


データベース

RDBMS

OS

仮想化

ホスト OS

容易な管理

Azure Database
















Write

Write Ack

Ack

Read

value write

Ack

特徴











Queries

SQL Database

Deeper Insight

Query

Store





領域改善後









Flexible


ow

er

BI A

pp

Serv

ices

Data

Fact

ory

A

naly

tics

ML

Co

gn

itiv

e B

othellip


Azure Compute

SQL Data

Warehouse

Azure Storage

SQL Database




Azure Database

Services Platform


SQLDB








SQLDB

SQLDB

SQLDB

SQLDB




D51

D52

D53

D60

hellip

D41

D42

D43

D50

hellip


D31

D32

D33

D40hellip

D21

D22

D23

D30

hellip

D11

D12

D13

D20

hellip

D1

D2

D3

D10

hellip

SQL DB





D51

D52

D53

D60


D41

D42

D43

D50

hellip


D31

D32

D33

D40hellip


D21

D22

D23

D30

hellip


D11

D12

D13

D20

hellip


D1

D2

D3

D10

hellip

SQL DB

SQL DB

SQL DB

SQL DB

SQL DB

SQL DB


bull

bull

bull

bull

bull




PB 級のデータ


GB 級のデータ

Cosmos DB

9 PM PST





11 PM PST11 PM PST

Read lt 2 ms

Writes lt 6 ms

Read lt 10 ms

Writes lt 15 ms

9950

キーバリュー

ドキュメント

グラフ

SQL (DocumentDB)

MongoDB

Azure Table Storage

Gremlin グラフ


MySQL




SQL



Azure Storage

Standard Table API

01

Strong

Bounded

Staleness

Session

Consistent

Prefix

Eventual

アプリ





45















LOB

Applications

Schema-on-Write




ソースからの

データ収集

Schema-on-Read






機械学習




デバイス


49












Tier 1

MachineLearning










分析処理





SQL Data Warehouse


Data LakeAnalytics



Tier 2


Tier 3

Analytics

Storage




Azure Data Lake

LOB Applications

SocialDevices

Clickstream

Sensors

Video

Web

Relational

HDInsight

Data Lake Analytics

Machine Learning

Spark

R



Azure Storage Blob


HDInsight








(New)

(New)



INSERT

OUTPUT

OUTPUT

SELECThellip

FROMhellip

WHEREhellip

+

PythonR

EXTRACT

EXTRACT

SELECT

SELECT

Azure

Data

Lake

Azure

Data

Lake

Azure

SQL

DB

Azure

Storage

Blobs

Azure

Storage

Blobs

RowSet RowSet

Cognitive Services


rs =

EXTRACT

UserID string

Start DateTime

End DateTime

Region string

SitesVisited string

PagesVisited string













C の関数


に近い）



58

進捗

統計情報


Business apps

Custom apps

Sensors and devices

ADL StorePeople


Azure SQL DW

Azure AD

Power BI

ADF

ADL Analytics


Business apps

Custom apps

Sensors and devices

HDInsight


データの連携

Power BI

File System

Database

Database

Hadoop

DWH

Data Mart










support


















service







computeHadoop





usserviceshdinsight


Store







Store




SQL Server Stretch



Insights

Azure SQL Database

SQL Server 2016


Azure SQL DW

ADLS amp ADLA

DocumentDB

HDInsight

Hadoop



Key


On

-pre

mis

es

Clo

ud






Azure

Data Lake Store

Azure



streaming







just supported

Generally Available





Want Hadoop






premises


Azure HDInsight






(StormSpark)





StudioU-SQL (C+SQL)






If NET or VS shop




Audit

ADLS Apache

Ranger

ADLS AAD Security

Groups (data)

Encryption TD

Audit


Scala Java Python

R

U-SQL T-SQL


External File

Types

ORC TXT

Parquet RCFile

ORC CSV Parquet

+ others

Parquet JSON

Hive + others


RCFile



Schema

Definition

Schema on Write

Polybase


Polybase


= 1PB)






learning-dt









IoT Hub

Event Hubs

Data Warehouse

Data Factory


Monitor

CDN

Search Power BI


Open amp

Interoperable

Manageable amp

Cost Efficient

Scalable amp

Performant

Secure amp

Compliant

Durable amp

Available

Machine Learning

Stream Analytics

Batch

Functions

Data Lake Analytics

Azure HDInsight

Big Data Use Cases



scale for cloud

services





データベース

RDBMS

OS

Azure が管理

利用者が管理

VM上の RDBMS

ホスト OS

仮想化


データベース

RDBMS

OS

仮想化

ホスト OS

容易な管理

Azure Database
















Write

Write Ack

Ack

Read

value write

Ack

特徴











Queries

SQL Database

Deeper Insight

Query

Store





領域改善後









Flexible


ow

er

BI A

pp

Serv

ices

Data

Fact

ory

A

naly

tics

ML

Co

gn

itiv

e B

othellip


Azure Compute

SQL Data

Warehouse

Azure Storage

SQL Database




Azure Database

Services Platform


SQLDB








SQLDB

SQLDB

SQLDB

SQLDB




D51

D52

D53

D60

hellip

D41

D42

D43

D50

hellip


D31

D32

D33

D40hellip

D21

D22

D23

D30

hellip

D11

D12

D13

D20

hellip

D1

D2

D3

D10

hellip

SQL DB





D51

D52

D53

D60


D41

D42

D43

D50

hellip


D31

D32

D33

D40hellip


D21

D22

D23

D30

hellip


D11

D12

D13

D20

hellip


D1

D2

D3

D10

hellip

SQL DB

SQL DB

SQL DB

SQL DB

SQL DB

SQL DB


bull

bull

bull

bull

bull




PB 級のデータ


GB 級のデータ

Cosmos DB

9 PM PST





11 PM PST11 PM PST

Read lt 2 ms

Writes lt 6 ms

Read lt 10 ms

Writes lt 15 ms

9950

キーバリュー

ドキュメント

グラフ

SQL (DocumentDB)

MongoDB

Azure Table Storage

Gremlin グラフ


MySQL




SQL



Azure Storage

Standard Table API

01

Strong

Bounded

Staleness

Session

Consistent

Prefix

Eventual

アプリ





45















LOB

Applications

Schema-on-Write




ソースからの

データ収集

Schema-on-Read






機械学習




デバイス


49












Tier 1

MachineLearning










分析処理





SQL Data Warehouse


Data LakeAnalytics



Tier 2


Tier 3

Analytics

Storage




Azure Data Lake

LOB Applications

SocialDevices

Clickstream

Sensors

Video

Web

Relational

HDInsight

Data Lake Analytics

Machine Learning

Spark

R



Azure Storage Blob


HDInsight








(New)

(New)



INSERT

OUTPUT

OUTPUT

SELECThellip

FROMhellip

WHEREhellip

+

PythonR

EXTRACT

EXTRACT

SELECT

SELECT

Azure

Data

Lake

Azure

Data

Lake

Azure

SQL

DB

Azure

Storage

Blobs

Azure

Storage

Blobs

RowSet RowSet

Cognitive Services


rs =

EXTRACT

UserID string

Start DateTime

End DateTime

Region string

SitesVisited string

PagesVisited string













C の関数


に近い）



58

進捗

統計情報


Business apps

Custom apps

Sensors and devices

ADL StorePeople


Azure SQL DW

Azure AD

Power BI

ADF

ADL Analytics


Business apps

Custom apps

Sensors and devices

HDInsight


データの連携

Power BI

File System

Database

Database

Hadoop

DWH

Data Mart










support


















service







computeHadoop





usserviceshdinsight


Store







Store




SQL Server Stretch



Insights

Azure SQL Database

SQL Server 2016


Azure SQL DW

ADLS amp ADLA

DocumentDB

HDInsight

Hadoop



Key


On

-pre

mis

es

Clo

ud






Azure

Data Lake Store

Azure



streaming







just supported

Generally Available





Want Hadoop






premises


Azure HDInsight






(StormSpark)





StudioU-SQL (C+SQL)






If NET or VS shop




Audit

ADLS Apache

Ranger

ADLS AAD Security

Groups (data)

Encryption TD

Audit


Scala Java Python

R

U-SQL T-SQL


External File

Types

ORC TXT

Parquet RCFile

ORC CSV Parquet

+ others

Parquet JSON

Hive + others


RCFile



Schema

Definition

Schema on Write

Polybase


Polybase


= 1PB)






learning-dt

IoT Hub

Event Hubs

Data Warehouse

Data Factory


Monitor

CDN

Search Power BI


Open amp

Interoperable

Manageable amp

Cost Efficient

Scalable amp

Performant

Secure amp

Compliant

Durable amp

Available

Machine Learning

Stream Analytics

Batch

Functions

Data Lake Analytics

Azure HDInsight

Big Data Use Cases



scale for cloud

services





データベース

RDBMS

OS

Azure が管理

利用者が管理

VM上の RDBMS

ホスト OS

仮想化


データベース

RDBMS

OS

仮想化

ホスト OS

容易な管理

Azure Database
















Write

Write Ack

Ack

Read

value write

Ack

特徴











Queries

SQL Database

Deeper Insight

Query

Store





領域改善後









Flexible


ow

er

BI A

pp

Serv

ices

Data

Fact

ory

A

naly

tics

ML

Co

gn

itiv

e B

othellip


Azure Compute

SQL Data

Warehouse

Azure Storage

SQL Database




Azure Database

Services Platform


SQLDB








SQLDB

SQLDB

SQLDB

SQLDB




D51

D52

D53

D60

hellip

D41

D42

D43

D50

hellip


D31

D32

D33

D40hellip

D21

D22

D23

D30

hellip

D11

D12

D13

D20

hellip

D1

D2

D3

D10

hellip

SQL DB





D51

D52

D53

D60


D41

D42

D43

D50

hellip


D31

D32

D33

D40hellip


D21

D22

D23

D30

hellip


D11

D12

D13

D20

hellip


D1

D2

D3

D10

hellip

SQL DB

SQL DB

SQL DB

SQL DB

SQL DB

SQL DB


bull

bull

bull

bull

bull




PB 級のデータ


GB 級のデータ

Cosmos DB

9 PM PST





11 PM PST11 PM PST

Read lt 2 ms

Writes lt 6 ms

Read lt 10 ms

Writes lt 15 ms

9950

キーバリュー

ドキュメント

グラフ

SQL (DocumentDB)

MongoDB

Azure Table Storage

Gremlin グラフ


MySQL




SQL



Azure Storage

Standard Table API

01

Strong

Bounded

Staleness

Session

Consistent

Prefix

Eventual

アプリ





45















LOB

Applications

Schema-on-Write




ソースからの

データ収集

Schema-on-Read






機械学習




デバイス


49












Tier 1

MachineLearning










分析処理





SQL Data Warehouse


Data LakeAnalytics



Tier 2


Tier 3

Analytics

Storage




Azure Data Lake

LOB Applications

SocialDevices

Clickstream

Sensors

Video

Web

Relational

HDInsight

Data Lake Analytics

Machine Learning

Spark

R



Azure Storage Blob


HDInsight








(New)

(New)



INSERT

OUTPUT

OUTPUT

SELECThellip

FROMhellip

WHEREhellip

+

PythonR

EXTRACT

EXTRACT

SELECT

SELECT

Azure

Data

Lake

Azure

Data

Lake

Azure

SQL

DB

Azure

Storage

Blobs

Azure

Storage

Blobs

RowSet RowSet

Cognitive Services


rs =

EXTRACT

UserID string

Start DateTime

End DateTime

Region string

SitesVisited string

PagesVisited string













C の関数


に近い）



58

進捗

統計情報


Business apps

Custom apps

Sensors and devices

ADL StorePeople


Azure SQL DW

Azure AD

Power BI

ADF

ADL Analytics


Business apps

Custom apps

Sensors and devices

HDInsight


データの連携

Power BI

File System

Database

Database

Hadoop

DWH

Data Mart










support


















service







computeHadoop





usserviceshdinsight


Store







Store




SQL Server Stretch



Insights

Azure SQL Database

SQL Server 2016


Azure SQL DW

ADLS amp ADLA

DocumentDB

HDInsight

Hadoop



Key


On

-pre

mis

es

Clo

ud






Azure

Data Lake Store

Azure



streaming







just supported

Generally Available





Want Hadoop






premises


Azure HDInsight






(StormSpark)





StudioU-SQL (C+SQL)






If NET or VS shop




Audit

ADLS Apache

Ranger

ADLS AAD Security

Groups (data)

Encryption TD

Audit


Scala Java Python

R

U-SQL T-SQL


External File

Types

ORC TXT

Parquet RCFile

ORC CSV Parquet

+ others

Parquet JSON

Hive + others


RCFile



Schema

Definition

Schema on Write

Polybase


Polybase


= 1PB)






learning-dt


scale for cloud

services





データベース

RDBMS

OS

Azure が管理

利用者が管理

VM上の RDBMS

ホスト OS

仮想化


データベース

RDBMS

OS

仮想化

ホスト OS

容易な管理

Azure Database
















Write

Write Ack

Ack

Read

value write

Ack

特徴











Queries

SQL Database

Deeper Insight

Query

Store





領域改善後









Flexible


ow

er

BI A

pp

Serv

ices

Data

Fact

ory

A

naly

tics

ML

Co

gn

itiv

e B

othellip


Azure Compute

SQL Data

Warehouse

Azure Storage

SQL Database




Azure Database

Services Platform


SQLDB








SQLDB

SQLDB

SQLDB

SQLDB




D51

D52

D53

D60

hellip

D41

D42

D43

D50

hellip


D31

D32

D33

D40hellip

D21

D22

D23

D30

hellip

D11

D12

D13

D20

hellip

D1

D2

D3

D10

hellip

SQL DB





D51

D52

D53

D60


D41

D42

D43

D50

hellip


D31

D32

D33

D40hellip


D21

D22

D23

D30

hellip


D11

D12

D13

D20

hellip


D1

D2

D3

D10

hellip

SQL DB

SQL DB

SQL DB

SQL DB

SQL DB

SQL DB


bull

bull

bull

bull

bull




PB 級のデータ


GB 級のデータ

Cosmos DB

9 PM PST





11 PM PST11 PM PST

Read lt 2 ms

Writes lt 6 ms

Read lt 10 ms

Writes lt 15 ms

9950

キーバリュー

ドキュメント

グラフ

SQL (DocumentDB)

MongoDB

Azure Table Storage

Gremlin グラフ


MySQL




SQL



Azure Storage

Standard Table API

01

Strong

Bounded

Staleness

Session

Consistent

Prefix

Eventual

アプリ





45















LOB

Applications

Schema-on-Write




ソースからの

データ収集

Schema-on-Read






機械学習




デバイス


49












Tier 1

MachineLearning










分析処理





SQL Data Warehouse


Data LakeAnalytics



Tier 2


Tier 3

Analytics

Storage




Azure Data Lake

LOB Applications

SocialDevices

Clickstream

Sensors

Video

Web

Relational

HDInsight

Data Lake Analytics

Machine Learning

Spark

R



Azure Storage Blob


HDInsight








(New)

(New)



INSERT

OUTPUT

OUTPUT

SELECThellip

FROMhellip

WHEREhellip

+

PythonR

EXTRACT

EXTRACT

SELECT

SELECT

Azure

Data

Lake

Azure

Data

Lake

Azure

SQL

DB

Azure

Storage

Blobs

Azure

Storage

Blobs

RowSet RowSet

Cognitive Services


rs =

EXTRACT

UserID string

Start DateTime

End DateTime

Region string

SitesVisited string

PagesVisited string













C の関数


に近い）



58

進捗

統計情報


Business apps

Custom apps

Sensors and devices

ADL StorePeople


Azure SQL DW

Azure AD

Power BI

ADF

ADL Analytics


Business apps

Custom apps

Sensors and devices

HDInsight


データの連携

Power BI

File System

Database

Database

Hadoop

DWH

Data Mart










support


















service







computeHadoop





usserviceshdinsight


Store







Store




SQL Server Stretch



Insights

Azure SQL Database

SQL Server 2016


Azure SQL DW

ADLS amp ADLA

DocumentDB

HDInsight

Hadoop



Key


On

-pre

mis

es

Clo

ud






Azure

Data Lake Store

Azure



streaming







just supported

Generally Available





Want Hadoop






premises


Azure HDInsight






(StormSpark)





StudioU-SQL (C+SQL)






If NET or VS shop




Audit

ADLS Apache

Ranger

ADLS AAD Security

Groups (data)

Encryption TD

Audit


Scala Java Python

R

U-SQL T-SQL


External File

Types

ORC TXT

Parquet RCFile

ORC CSV Parquet

+ others

Parquet JSON

Hive + others


RCFile



Schema

Definition

Schema on Write

Polybase


Polybase


= 1PB)






learning-dt




データベース

RDBMS

OS

Azure が管理

利用者が管理

VM上の RDBMS

ホスト OS

仮想化


データベース

RDBMS

OS

仮想化

ホスト OS

容易な管理

Azure Database
















Write

Write Ack

Ack

Read

value write

Ack

特徴











Queries

SQL Database

Deeper Insight

Query

Store





領域改善後









Flexible


ow

er

BI A

pp

Serv

ices

Data

Fact

ory

A

naly

tics

ML

Co

gn

itiv

e B

othellip


Azure Compute

SQL Data

Warehouse

Azure Storage

SQL Database




Azure Database

Services Platform


SQLDB








SQLDB

SQLDB

SQLDB

SQLDB




D51

D52

D53

D60

hellip

D41

D42

D43

D50

hellip


D31

D32

D33

D40hellip

D21

D22

D23

D30

hellip

D11

D12

D13

D20

hellip

D1

D2

D3

D10

hellip

SQL DB





D51

D52

D53

D60


D41

D42

D43

D50

hellip


D31

D32

D33

D40hellip


D21

D22

D23

D30

hellip


D11

D12

D13

D20

hellip


D1

D2

D3

D10

hellip

SQL DB

SQL DB

SQL DB

SQL DB

SQL DB

SQL DB


bull

bull

bull

bull

bull




PB 級のデータ


GB 級のデータ

Cosmos DB

9 PM PST





11 PM PST11 PM PST

Read lt 2 ms

Writes lt 6 ms

Read lt 10 ms

Writes lt 15 ms

9950

キーバリュー

ドキュメント

グラフ

SQL (DocumentDB)

MongoDB

Azure Table Storage

Gremlin グラフ


MySQL




SQL



Azure Storage

Standard Table API

01

Strong

Bounded

Staleness

Session

Consistent

Prefix

Eventual

アプリ





45















LOB

Applications

Schema-on-Write




ソースからの

データ収集

Schema-on-Read






機械学習




デバイス


49












Tier 1

MachineLearning










分析処理





SQL Data Warehouse


Data LakeAnalytics



Tier 2


Tier 3

Analytics

Storage




Azure Data Lake

LOB Applications

SocialDevices

Clickstream

Sensors

Video

Web

Relational

HDInsight

Data Lake Analytics

Machine Learning

Spark

R



Azure Storage Blob


HDInsight








(New)

(New)



INSERT

OUTPUT

OUTPUT

SELECThellip

FROMhellip

WHEREhellip

+

PythonR

EXTRACT

EXTRACT

SELECT

SELECT

Azure

Data

Lake

Azure

Data

Lake

Azure

SQL

DB

Azure

Storage

Blobs

Azure

Storage

Blobs

RowSet RowSet

Cognitive Services


rs =

EXTRACT

UserID string

Start DateTime

End DateTime

Region string

SitesVisited string

PagesVisited string













C の関数


に近い）



58

進捗

統計情報


Business apps

Custom apps

Sensors and devices

ADL StorePeople


Azure SQL DW

Azure AD

Power BI

ADF

ADL Analytics


Business apps

Custom apps

Sensors and devices

HDInsight


データの連携

Power BI

File System

Database

Database

Hadoop

DWH

Data Mart










support


















service







computeHadoop





usserviceshdinsight


Store







Store




SQL Server Stretch



Insights

Azure SQL Database

SQL Server 2016


Azure SQL DW

ADLS amp ADLA

DocumentDB

HDInsight

Hadoop



Key


On

-pre

mis

es

Clo

ud






Azure

Data Lake Store

Azure



streaming







just supported

Generally Available





Want Hadoop






premises


Azure HDInsight






(StormSpark)





StudioU-SQL (C+SQL)






If NET or VS shop




Audit

ADLS Apache

Ranger

ADLS AAD Security

Groups (data)

Encryption TD

Audit


Scala Java Python

R

U-SQL T-SQL


External File

Types

ORC TXT

Parquet RCFile

ORC CSV Parquet

+ others

Parquet JSON

Hive + others


RCFile



Schema

Definition

Schema on Write

Polybase


Polybase


= 1PB)






learning-dt



データベース

RDBMS

OS

Azure が管理

利用者が管理

VM上の RDBMS

ホスト OS

仮想化


データベース

RDBMS

OS

仮想化

ホスト OS

容易な管理

Azure Database
















Write

Write Ack

Ack

Read

value write

Ack

特徴











Queries

SQL Database

Deeper Insight

Query

Store





領域改善後









Flexible


ow

er

BI A

pp

Serv

ices

Data

Fact

ory

A

naly

tics

ML

Co

gn

itiv

e B

othellip


Azure Compute

SQL Data

Warehouse

Azure Storage

SQL Database




Azure Database

Services Platform


SQLDB








SQLDB

SQLDB

SQLDB

SQLDB




D51

D52

D53

D60

hellip

D41

D42

D43

D50

hellip


D31

D32

D33

D40hellip

D21

D22

D23

D30

hellip

D11

D12

D13

D20

hellip

D1

D2

D3

D10

hellip

SQL DB





D51

D52

D53

D60


D41

D42

D43

D50

hellip


D31

D32

D33

D40hellip


D21

D22

D23

D30

hellip


D11

D12

D13

D20

hellip


D1

D2

D3

D10

hellip

SQL DB

SQL DB

SQL DB

SQL DB

SQL DB

SQL DB


bull

bull

bull

bull

bull




PB 級のデータ


GB 級のデータ

Cosmos DB

9 PM PST





11 PM PST11 PM PST

Read lt 2 ms

Writes lt 6 ms

Read lt 10 ms

Writes lt 15 ms

9950

キーバリュー

ドキュメント

グラフ

SQL (DocumentDB)

MongoDB

Azure Table Storage

Gremlin グラフ


MySQL




SQL



Azure Storage

Standard Table API

01

Strong

Bounded

Staleness

Session

Consistent

Prefix

Eventual

アプリ





45















LOB

Applications

Schema-on-Write




ソースからの

データ収集

Schema-on-Read






機械学習




デバイス


49












Tier 1

MachineLearning










分析処理





SQL Data Warehouse


Data LakeAnalytics



Tier 2


Tier 3

Analytics

Storage




Azure Data Lake

LOB Applications

SocialDevices

Clickstream

Sensors

Video

Web

Relational

HDInsight

Data Lake Analytics

Machine Learning

Spark

R



Azure Storage Blob


HDInsight








(New)

(New)



INSERT

OUTPUT

OUTPUT

SELECThellip

FROMhellip

WHEREhellip

+

PythonR

EXTRACT

EXTRACT

SELECT

SELECT

Azure

Data

Lake

Azure

Data

Lake

Azure

SQL

DB

Azure

Storage

Blobs

Azure

Storage

Blobs

RowSet RowSet

Cognitive Services


rs =

EXTRACT

UserID string

Start DateTime

End DateTime

Region string

SitesVisited string

PagesVisited string













C の関数


に近い）



58

進捗

統計情報


Business apps

Custom apps

Sensors and devices

ADL StorePeople


Azure SQL DW

Azure AD

Power BI

ADF

ADL Analytics


Business apps

Custom apps

Sensors and devices

HDInsight


データの連携

Power BI

File System

Database

Database

Hadoop

DWH

Data Mart










support


















service







computeHadoop





usserviceshdinsight


Store







Store




SQL Server Stretch



Insights

Azure SQL Database

SQL Server 2016


Azure SQL DW

ADLS amp ADLA

DocumentDB

HDInsight

Hadoop



Key


On

-pre

mis

es

Clo

ud






Azure

Data Lake Store

Azure



streaming







just supported

Generally Available





Want Hadoop






premises


Azure HDInsight






(StormSpark)





StudioU-SQL (C+SQL)






If NET or VS shop




Audit

ADLS Apache

Ranger

ADLS AAD Security

Groups (data)

Encryption TD

Audit


Scala Java Python

R

U-SQL T-SQL


External File

Types

ORC TXT

Parquet RCFile

ORC CSV Parquet

+ others

Parquet JSON

Hive + others


RCFile



Schema

Definition

Schema on Write

Polybase


Polybase


= 1PB)






learning-dt


データベース

RDBMS

OS

Azure が管理

利用者が管理

VM上の RDBMS

ホスト OS

仮想化


データベース

RDBMS

OS

仮想化

ホスト OS

容易な管理

Azure Database
















Write

Write Ack

Ack

Read

value write

Ack

特徴











Queries

SQL Database

Deeper Insight

Query

Store





領域改善後









Flexible


ow

er

BI A

pp

Serv

ices

Data

Fact

ory

A

naly

tics

ML

Co

gn

itiv

e B

othellip


Azure Compute

SQL Data

Warehouse

Azure Storage

SQL Database




Azure Database

Services Platform


SQLDB








SQLDB

SQLDB

SQLDB

SQLDB




D51

D52

D53

D60

hellip

D41

D42

D43

D50

hellip


D31

D32

D33

D40hellip

D21

D22

D23

D30

hellip

D11

D12

D13

D20

hellip

D1

D2

D3

D10

hellip

SQL DB





D51

D52

D53

D60


D41

D42

D43

D50

hellip


D31

D32

D33

D40hellip


D21

D22

D23

D30

hellip


D11

D12

D13

D20

hellip


D1

D2

D3

D10

hellip

SQL DB

SQL DB

SQL DB

SQL DB

SQL DB

SQL DB


bull

bull

bull

bull

bull




PB 級のデータ


GB 級のデータ

Cosmos DB

9 PM PST





11 PM PST11 PM PST

Read lt 2 ms

Writes lt 6 ms

Read lt 10 ms

Writes lt 15 ms

9950

キーバリュー

ドキュメント

グラフ

SQL (DocumentDB)

MongoDB

Azure Table Storage

Gremlin グラフ


MySQL




SQL



Azure Storage

Standard Table API

01

Strong

Bounded

Staleness

Session

Consistent

Prefix

Eventual

アプリ





45















LOB

Applications

Schema-on-Write




ソースからの

データ収集

Schema-on-Read






機械学習




デバイス


49












Tier 1

MachineLearning










分析処理





SQL Data Warehouse


Data LakeAnalytics



Tier 2


Tier 3

Analytics

Storage




Azure Data Lake

LOB Applications

SocialDevices

Clickstream

Sensors

Video

Web

Relational

HDInsight

Data Lake Analytics

Machine Learning

Spark

R



Azure Storage Blob


HDInsight








(New)

(New)



INSERT

OUTPUT

OUTPUT

SELECThellip

FROMhellip

WHEREhellip

+

PythonR

EXTRACT

EXTRACT

SELECT

SELECT

Azure

Data

Lake

Azure

Data

Lake

Azure

SQL

DB

Azure

Storage

Blobs

Azure

Storage

Blobs

RowSet RowSet

Cognitive Services


rs =

EXTRACT

UserID string

Start DateTime

End DateTime

Region string

SitesVisited string

PagesVisited string













C の関数


に近い）



58

進捗

統計情報


Business apps

Custom apps

Sensors and devices

ADL StorePeople


Azure SQL DW

Azure AD

Power BI

ADF

ADL Analytics


Business apps

Custom apps

Sensors and devices

HDInsight


データの連携

Power BI

File System

Database

Database

Hadoop

DWH

Data Mart










support


















service







computeHadoop





usserviceshdinsight


Store







Store




SQL Server Stretch



Insights

Azure SQL Database

SQL Server 2016


Azure SQL DW

ADLS amp ADLA

DocumentDB

HDInsight

Hadoop



Key


On

-pre

mis

es

Clo

ud






Azure

Data Lake Store

Azure



streaming







just supported

Generally Available





Want Hadoop






premises


Azure HDInsight






(StormSpark)





StudioU-SQL (C+SQL)






If NET or VS shop




Audit

ADLS Apache

Ranger

ADLS AAD Security

Groups (data)

Encryption TD

Audit


Scala Java Python

R

U-SQL T-SQL


External File

Types

ORC TXT

Parquet RCFile

ORC CSV Parquet

+ others

Parquet JSON

Hive + others


RCFile



Schema

Definition

Schema on Write

Polybase


Polybase


= 1PB)






learning-dt
















Write

Write Ack

Ack

Read

value write

Ack

特徴











Queries

SQL Database

Deeper Insight

Query

Store





領域改善後









Flexible


ow

er

BI A

pp

Serv

ices

Data

Fact

ory

A

naly

tics

ML

Co

gn

itiv

e B

othellip


Azure Compute

SQL Data

Warehouse

Azure Storage

SQL Database




Azure Database

Services Platform


SQLDB








SQLDB

SQLDB

SQLDB

SQLDB




D51

D52

D53

D60

hellip

D41

D42

D43

D50

hellip


D31

D32

D33

D40hellip

D21

D22

D23

D30

hellip

D11

D12

D13

D20

hellip

D1

D2

D3

D10

hellip

SQL DB





D51

D52

D53

D60


D41

D42

D43

D50

hellip


D31

D32

D33

D40hellip


D21

D22

D23

D30

hellip


D11

D12

D13

D20

hellip


D1

D2

D3

D10

hellip

SQL DB

SQL DB

SQL DB

SQL DB

SQL DB

SQL DB


bull

bull

bull

bull

bull




PB 級のデータ


GB 級のデータ

Cosmos DB

9 PM PST





11 PM PST11 PM PST

Read lt 2 ms

Writes lt 6 ms

Read lt 10 ms

Writes lt 15 ms

9950

キーバリュー

ドキュメント

グラフ

SQL (DocumentDB)

MongoDB

Azure Table Storage

Gremlin グラフ


MySQL




SQL



Azure Storage

Standard Table API

01

Strong

Bounded

Staleness

Session

Consistent

Prefix

Eventual

アプリ





45















LOB

Applications

Schema-on-Write




ソースからの

データ収集

Schema-on-Read






機械学習




デバイス


49












Tier 1

MachineLearning










分析処理





SQL Data Warehouse


Data LakeAnalytics



Tier 2


Tier 3

Analytics

Storage




Azure Data Lake

LOB Applications

SocialDevices

Clickstream

Sensors

Video

Web

Relational

HDInsight

Data Lake Analytics

Machine Learning

Spark

R



Azure Storage Blob


HDInsight








(New)

(New)



INSERT

OUTPUT

OUTPUT

SELECThellip

FROMhellip

WHEREhellip

+

PythonR

EXTRACT

EXTRACT

SELECT

SELECT

Azure

Data

Lake

Azure

Data

Lake

Azure

SQL

DB

Azure

Storage

Blobs

Azure

Storage

Blobs

RowSet RowSet

Cognitive Services


rs =

EXTRACT

UserID string

Start DateTime

End DateTime

Region string

SitesVisited string

PagesVisited string













C の関数


に近い）



58

進捗

統計情報


Business apps

Custom apps

Sensors and devices

ADL StorePeople


Azure SQL DW

Azure AD

Power BI

ADF

ADL Analytics


Business apps

Custom apps

Sensors and devices

HDInsight


データの連携

Power BI

File System

Database

Database

Hadoop

DWH

Data Mart










support


















service







computeHadoop





usserviceshdinsight


Store







Store




SQL Server Stretch



Insights

Azure SQL Database

SQL Server 2016


Azure SQL DW

ADLS amp ADLA

DocumentDB

HDInsight

Hadoop



Key


On

-pre

mis

es

Clo

ud






Azure

Data Lake Store

Azure



streaming







just supported

Generally Available





Want Hadoop






premises


Azure HDInsight






(StormSpark)





StudioU-SQL (C+SQL)






If NET or VS shop




Audit

ADLS Apache

Ranger

ADLS AAD Security

Groups (data)

Encryption TD

Audit


Scala Java Python

R

U-SQL T-SQL


External File

Types

ORC TXT

Parquet RCFile

ORC CSV Parquet

+ others

Parquet JSON

Hive + others


RCFile



Schema

Definition

Schema on Write

Polybase


Polybase


= 1PB)






learning-dt













Write

Write Ack

Ack

Read

value write

Ack

特徴











Queries

SQL Database

Deeper Insight

Query

Store





領域改善後









Flexible


ow

er

BI A

pp

Serv

ices

Data

Fact

ory

A

naly

tics

ML

Co

gn

itiv

e B

othellip


Azure Compute

SQL Data

Warehouse

Azure Storage

SQL Database




Azure Database

Services Platform


SQLDB








SQLDB

SQLDB

SQLDB

SQLDB




D51

D52

D53

D60

hellip

D41

D42

D43

D50

hellip


D31

D32

D33

D40hellip

D21

D22

D23

D30

hellip

D11

D12

D13

D20

hellip

D1

D2

D3

D10

hellip

SQL DB





D51

D52

D53

D60


D41

D42

D43

D50

hellip


D31

D32

D33

D40hellip


D21

D22

D23

D30

hellip


D11

D12

D13

D20

hellip


D1

D2

D3

D10

hellip

SQL DB

SQL DB

SQL DB

SQL DB

SQL DB

SQL DB


bull

bull

bull

bull

bull




PB 級のデータ


GB 級のデータ

Cosmos DB

9 PM PST





11 PM PST11 PM PST

Read lt 2 ms

Writes lt 6 ms

Read lt 10 ms

Writes lt 15 ms

9950

キーバリュー

ドキュメント

グラフ

SQL (DocumentDB)

MongoDB

Azure Table Storage

Gremlin グラフ


MySQL




SQL



Azure Storage

Standard Table API

01

Strong

Bounded

Staleness

Session

Consistent

Prefix

Eventual

アプリ





45















LOB

Applications

Schema-on-Write




ソースからの

データ収集

Schema-on-Read






機械学習




デバイス


49












Tier 1

MachineLearning










分析処理





SQL Data Warehouse


Data LakeAnalytics



Tier 2


Tier 3

Analytics

Storage




Azure Data Lake

LOB Applications

SocialDevices

Clickstream

Sensors

Video

Web

Relational

HDInsight

Data Lake Analytics

Machine Learning

Spark

R



Azure Storage Blob


HDInsight








(New)

(New)



INSERT

OUTPUT

OUTPUT

SELECThellip

FROMhellip

WHEREhellip

+

PythonR

EXTRACT

EXTRACT

SELECT

SELECT

Azure

Data

Lake

Azure

Data

Lake

Azure

SQL

DB

Azure

Storage

Blobs

Azure

Storage

Blobs

RowSet RowSet

Cognitive Services


rs =

EXTRACT

UserID string

Start DateTime

End DateTime

Region string

SitesVisited string

PagesVisited string













C の関数


に近い）



58

進捗

統計情報


Business apps

Custom apps

Sensors and devices

ADL StorePeople


Azure SQL DW

Azure AD

Power BI

ADF

ADL Analytics


Business apps

Custom apps

Sensors and devices

HDInsight


データの連携

Power BI

File System

Database

Database

Hadoop

DWH

Data Mart










support


















service







computeHadoop





usserviceshdinsight


Store







Store




SQL Server Stretch



Insights

Azure SQL Database

SQL Server 2016


Azure SQL DW

ADLS amp ADLA

DocumentDB

HDInsight

Hadoop



Key


On

-pre

mis

es

Clo

ud






Azure

Data Lake Store

Azure



streaming







just supported

Generally Available





Want Hadoop






premises


Azure HDInsight






(StormSpark)





StudioU-SQL (C+SQL)






If NET or VS shop




Audit

ADLS Apache

Ranger

ADLS AAD Security

Groups (data)

Encryption TD

Audit


Scala Java Python

R

U-SQL T-SQL


External File

Types

ORC TXT

Parquet RCFile

ORC CSV Parquet

+ others

Parquet JSON

Hive + others


RCFile



Schema

Definition

Schema on Write

Polybase


Polybase


= 1PB)






learning-dt










Write

Write Ack

Ack

Read

value write

Ack

特徴











Queries

SQL Database

Deeper Insight

Query

Store





領域改善後









Flexible


ow

er

BI A

pp

Serv

ices

Data

Fact

ory

A

naly

tics

ML

Co

gn

itiv

e B

othellip


Azure Compute

SQL Data

Warehouse

Azure Storage

SQL Database




Azure Database

Services Platform


SQLDB








SQLDB

SQLDB

SQLDB

SQLDB




D51

D52

D53

D60

hellip

D41

D42

D43

D50

hellip


D31

D32

D33

D40hellip

D21

D22

D23

D30

hellip

D11

D12

D13

D20

hellip

D1

D2

D3

D10

hellip

SQL DB





D51

D52

D53

D60


D41

D42

D43

D50

hellip


D31

D32

D33

D40hellip


D21

D22

D23

D30

hellip


D11

D12

D13

D20

hellip


D1

D2

D3

D10

hellip

SQL DB

SQL DB

SQL DB

SQL DB

SQL DB

SQL DB


bull

bull

bull

bull

bull




PB 級のデータ


GB 級のデータ

Cosmos DB

9 PM PST





11 PM PST11 PM PST

Read lt 2 ms

Writes lt 6 ms

Read lt 10 ms

Writes lt 15 ms

9950

キーバリュー

ドキュメント

グラフ

SQL (DocumentDB)

MongoDB

Azure Table Storage

Gremlin グラフ


MySQL




SQL



Azure Storage

Standard Table API

01

Strong

Bounded

Staleness

Session

Consistent

Prefix

Eventual

アプリ





45















LOB

Applications

Schema-on-Write




ソースからの

データ収集

Schema-on-Read






機械学習




デバイス


49












Tier 1

MachineLearning










分析処理





SQL Data Warehouse


Data LakeAnalytics



Tier 2


Tier 3

Analytics

Storage




Azure Data Lake

LOB Applications

SocialDevices

Clickstream

Sensors

Video

Web

Relational

HDInsight

Data Lake Analytics

Machine Learning

Spark

R



Azure Storage Blob


HDInsight








(New)

(New)



INSERT

OUTPUT

OUTPUT

SELECThellip

FROMhellip

WHEREhellip

+

PythonR

EXTRACT

EXTRACT

SELECT

SELECT

Azure

Data

Lake

Azure

Data

Lake

Azure

SQL

DB

Azure

Storage

Blobs

Azure

Storage

Blobs

RowSet RowSet

Cognitive Services


rs =

EXTRACT

UserID string

Start DateTime

End DateTime

Region string

SitesVisited string

PagesVisited string













C の関数


に近い）



58

進捗

統計情報


Business apps

Custom apps

Sensors and devices

ADL StorePeople


Azure SQL DW

Azure AD

Power BI

ADF

ADL Analytics


Business apps

Custom apps

Sensors and devices

HDInsight


データの連携

Power BI

File System

Database

Database

Hadoop

DWH

Data Mart










support


















service







computeHadoop





usserviceshdinsight


Store







Store




SQL Server Stretch



Insights

Azure SQL Database

SQL Server 2016


Azure SQL DW

ADLS amp ADLA

DocumentDB

HDInsight

Hadoop



Key


On

-pre

mis

es

Clo

ud






Azure

Data Lake Store

Azure



streaming







just supported

Generally Available





Want Hadoop






premises


Azure HDInsight






(StormSpark)





StudioU-SQL (C+SQL)






If NET or VS shop




Audit

ADLS Apache

Ranger

ADLS AAD Security

Groups (data)

Encryption TD

Audit


Scala Java Python

R

U-SQL T-SQL


External File

Types

ORC TXT

Parquet RCFile

ORC CSV Parquet

+ others

Parquet JSON

Hive + others


RCFile



Schema

Definition

Schema on Write

Polybase


Polybase


= 1PB)






learning-dt


Write

Write Ack

Ack

Read

value write

Ack

特徴











Queries

SQL Database

Deeper Insight

Query

Store





領域改善後









Flexible


ow

er

BI A

pp

Serv

ices

Data

Fact

ory

A

naly

tics

ML

Co

gn

itiv

e B

othellip


Azure Compute

SQL Data

Warehouse

Azure Storage

SQL Database




Azure Database

Services Platform


SQLDB








SQLDB

SQLDB

SQLDB

SQLDB




D51

D52

D53

D60

hellip

D41

D42

D43

D50

hellip


D31

D32

D33

D40hellip

D21

D22

D23

D30

hellip

D11

D12

D13

D20

hellip

D1

D2

D3

D10

hellip

SQL DB





D51

D52

D53

D60


D41

D42

D43

D50

hellip


D31

D32

D33

D40hellip


D21

D22

D23

D30

hellip


D11

D12

D13

D20

hellip


D1

D2

D3

D10

hellip

SQL DB

SQL DB

SQL DB

SQL DB

SQL DB

SQL DB


bull

bull

bull

bull

bull




PB 級のデータ


GB 級のデータ

Cosmos DB

9 PM PST





11 PM PST11 PM PST

Read lt 2 ms

Writes lt 6 ms

Read lt 10 ms

Writes lt 15 ms

9950

キーバリュー

ドキュメント

グラフ

SQL (DocumentDB)

MongoDB

Azure Table Storage

Gremlin グラフ


MySQL




SQL



Azure Storage

Standard Table API

01

Strong

Bounded

Staleness

Session

Consistent

Prefix

Eventual

アプリ





45















LOB

Applications

Schema-on-Write




ソースからの

データ収集

Schema-on-Read






機械学習




デバイス


49












Tier 1

MachineLearning










分析処理





SQL Data Warehouse


Data LakeAnalytics



Tier 2


Tier 3

Analytics

Storage




Azure Data Lake

LOB Applications

SocialDevices

Clickstream

Sensors

Video

Web

Relational

HDInsight

Data Lake Analytics

Machine Learning

Spark

R



Azure Storage Blob


HDInsight








(New)

(New)



INSERT

OUTPUT

OUTPUT

SELECThellip

FROMhellip

WHEREhellip

+

PythonR

EXTRACT

EXTRACT

SELECT

SELECT

Azure

Data

Lake

Azure

Data

Lake

Azure

SQL

DB

Azure

Storage

Blobs

Azure

Storage

Blobs

RowSet RowSet

Cognitive Services


rs =

EXTRACT

UserID string

Start DateTime

End DateTime

Region string

SitesVisited string

PagesVisited string













C の関数


に近い）



58

進捗

統計情報


Business apps

Custom apps

Sensors and devices

ADL StorePeople


Azure SQL DW

Azure AD

Power BI

ADF

ADL Analytics


Business apps

Custom apps

Sensors and devices

HDInsight


データの連携

Power BI

File System

Database

Database

Hadoop

DWH

Data Mart










support


















service







computeHadoop





usserviceshdinsight


Store







Store




SQL Server Stretch



Insights

Azure SQL Database

SQL Server 2016


Azure SQL DW

ADLS amp ADLA

DocumentDB

HDInsight

Hadoop



Key


On

-pre

mis

es

Clo

ud






Azure

Data Lake Store

Azure



streaming







just supported

Generally Available





Want Hadoop






premises


Azure HDInsight






(StormSpark)





StudioU-SQL (C+SQL)






If NET or VS shop




Audit

ADLS Apache

Ranger

ADLS AAD Security

Groups (data)

Encryption TD

Audit


Scala Java Python

R

U-SQL T-SQL


External File

Types

ORC TXT

Parquet RCFile

ORC CSV Parquet

+ others

Parquet JSON

Hive + others


RCFile



Schema

Definition

Schema on Write

Polybase


Polybase


= 1PB)






learning-dt







Queries

SQL Database

Deeper Insight

Query

Store





領域改善後









Flexible


ow

er

BI A

pp

Serv

ices

Data

Fact

ory

A

naly

tics

ML

Co

gn

itiv

e B

othellip


Azure Compute

SQL Data

Warehouse

Azure Storage

SQL Database




Azure Database

Services Platform


SQLDB








SQLDB

SQLDB

SQLDB

SQLDB




D51

D52

D53

D60

hellip

D41

D42

D43

D50

hellip


D31

D32

D33

D40hellip

D21

D22

D23

D30

hellip

D11

D12

D13

D20

hellip

D1

D2

D3

D10

hellip

SQL DB





D51

D52

D53

D60


D41

D42

D43

D50

hellip


D31

D32

D33

D40hellip


D21

D22

D23

D30

hellip


D11

D12

D13

D20

hellip


D1

D2

D3

D10

hellip

SQL DB

SQL DB

SQL DB

SQL DB

SQL DB

SQL DB


bull

bull

bull

bull

bull




PB 級のデータ


GB 級のデータ

Cosmos DB

9 PM PST





11 PM PST11 PM PST

Read lt 2 ms

Writes lt 6 ms

Read lt 10 ms

Writes lt 15 ms

9950

キーバリュー

ドキュメント

グラフ

SQL (DocumentDB)

MongoDB

Azure Table Storage

Gremlin グラフ


MySQL




SQL



Azure Storage

Standard Table API

01

Strong

Bounded

Staleness

Session

Consistent

Prefix

Eventual

アプリ





45















LOB

Applications

Schema-on-Write




ソースからの

データ収集

Schema-on-Read






機械学習




デバイス


49












Tier 1

MachineLearning










分析処理





SQL Data Warehouse


Data LakeAnalytics



Tier 2


Tier 3

Analytics

Storage




Azure Data Lake

LOB Applications

SocialDevices

Clickstream

Sensors

Video

Web

Relational

HDInsight

Data Lake Analytics

Machine Learning

Spark

R



Azure Storage Blob


HDInsight








(New)

(New)



INSERT

OUTPUT

OUTPUT

SELECThellip

FROMhellip

WHEREhellip

+

PythonR

EXTRACT

EXTRACT

SELECT

SELECT

Azure

Data

Lake

Azure

Data

Lake

Azure

SQL

DB

Azure

Storage

Blobs

Azure

Storage

Blobs

RowSet RowSet

Cognitive Services


rs =

EXTRACT

UserID string

Start DateTime

End DateTime

Region string

SitesVisited string

PagesVisited string













C の関数


に近い）



58

進捗

統計情報


Business apps

Custom apps

Sensors and devices

ADL StorePeople


Azure SQL DW

Azure AD

Power BI

ADF

ADL Analytics


Business apps

Custom apps

Sensors and devices

HDInsight


データの連携

Power BI

File System

Database

Database

Hadoop

DWH

Data Mart










support


















service







computeHadoop





usserviceshdinsight


Store







Store




SQL Server Stretch



Insights

Azure SQL Database

SQL Server 2016


Azure SQL DW

ADLS amp ADLA

DocumentDB

HDInsight

Hadoop



Key


On

-pre

mis

es

Clo

ud






Azure

Data Lake Store

Azure



streaming







just supported

Generally Available





Want Hadoop






premises


Azure HDInsight






(StormSpark)





StudioU-SQL (C+SQL)






If NET or VS shop




Audit

ADLS Apache

Ranger

ADLS AAD Security

Groups (data)

Encryption TD

Audit


Scala Java Python

R

U-SQL T-SQL


External File

Types

ORC TXT

Parquet RCFile

ORC CSV Parquet

+ others

Parquet JSON

Hive + others


RCFile



Schema

Definition

Schema on Write

Polybase


Polybase


= 1PB)






learning-dt





Queries

SQL Database

Deeper Insight

Query

Store





領域改善後









Flexible


ow

er

BI A

pp

Serv

ices

Data

Fact

ory

A

naly

tics

ML

Co

gn

itiv

e B

othellip


Azure Compute

SQL Data

Warehouse

Azure Storage

SQL Database




Azure Database

Services Platform


SQLDB








SQLDB

SQLDB

SQLDB

SQLDB




D51

D52

D53

D60

hellip

D41

D42

D43

D50

hellip


D31

D32

D33

D40hellip

D21

D22

D23

D30

hellip

D11

D12

D13

D20

hellip

D1

D2

D3

D10

hellip

SQL DB





D51

D52

D53

D60


D41

D42

D43

D50

hellip


D31

D32

D33

D40hellip


D21

D22

D23

D30

hellip


D11

D12

D13

D20

hellip


D1

D2

D3

D10

hellip

SQL DB

SQL DB

SQL DB

SQL DB

SQL DB

SQL DB


bull

bull

bull

bull

bull




PB 級のデータ


GB 級のデータ

Cosmos DB

9 PM PST





11 PM PST11 PM PST

Read lt 2 ms

Writes lt 6 ms

Read lt 10 ms

Writes lt 15 ms

9950

キーバリュー

ドキュメント

グラフ

SQL (DocumentDB)

MongoDB

Azure Table Storage

Gremlin グラフ


MySQL




SQL



Azure Storage

Standard Table API

01

Strong

Bounded

Staleness

Session

Consistent

Prefix

Eventual

アプリ





45















LOB

Applications

Schema-on-Write




ソースからの

データ収集

Schema-on-Read






機械学習




デバイス


49












Tier 1

MachineLearning










分析処理





SQL Data Warehouse


Data LakeAnalytics



Tier 2


Tier 3

Analytics

Storage




Azure Data Lake

LOB Applications

SocialDevices

Clickstream

Sensors

Video

Web

Relational

HDInsight

Data Lake Analytics

Machine Learning

Spark

R



Azure Storage Blob


HDInsight








(New)

(New)



INSERT

OUTPUT

OUTPUT

SELECThellip

FROMhellip

WHEREhellip

+

PythonR

EXTRACT

EXTRACT

SELECT

SELECT

Azure

Data

Lake

Azure

Data

Lake

Azure

SQL

DB

Azure

Storage

Blobs

Azure

Storage

Blobs

RowSet RowSet

Cognitive Services


rs =

EXTRACT

UserID string

Start DateTime

End DateTime

Region string

SitesVisited string

PagesVisited string













C の関数


に近い）



58

進捗

統計情報


Business apps

Custom apps

Sensors and devices

ADL StorePeople


Azure SQL DW

Azure AD

Power BI

ADF

ADL Analytics


Business apps

Custom apps

Sensors and devices

HDInsight


データの連携

Power BI

File System

Database

Database

Hadoop

DWH

Data Mart










support


















service







computeHadoop





usserviceshdinsight


Store







Store




SQL Server Stretch



Insights

Azure SQL Database

SQL Server 2016


Azure SQL DW

ADLS amp ADLA

DocumentDB

HDInsight

Hadoop



Key


On

-pre

mis

es

Clo

ud






Azure

Data Lake Store

Azure



streaming







just supported

Generally Available





Want Hadoop






premises


Azure HDInsight






(StormSpark)





StudioU-SQL (C+SQL)






If NET or VS shop




Audit

ADLS Apache

Ranger

ADLS AAD Security

Groups (data)

Encryption TD

Audit


Scala Java Python

R

U-SQL T-SQL


External File

Types

ORC TXT

Parquet RCFile

ORC CSV Parquet

+ others

Parquet JSON

Hive + others


RCFile



Schema

Definition

Schema on Write

Polybase


Polybase


= 1PB)






learning-dt

Queries

SQL Database

Deeper Insight

Query

Store





領域改善後









Flexible


ow

er

BI A

pp

Serv

ices

Data

Fact

ory

A

naly

tics

ML

Co

gn

itiv

e B

othellip


Azure Compute

SQL Data

Warehouse

Azure Storage

SQL Database




Azure Database

Services Platform


SQLDB








SQLDB

SQLDB

SQLDB

SQLDB




D51

D52

D53

D60

hellip

D41

D42

D43

D50

hellip


D31

D32

D33

D40hellip

D21

D22

D23

D30

hellip

D11

D12

D13

D20

hellip

D1

D2

D3

D10

hellip

SQL DB





D51

D52

D53

D60


D41

D42

D43

D50

hellip


D31

D32

D33

D40hellip


D21

D22

D23

D30

hellip


D11

D12

D13

D20

hellip


D1

D2

D3

D10

hellip

SQL DB

SQL DB

SQL DB

SQL DB

SQL DB

SQL DB


bull

bull

bull

bull

bull




PB 級のデータ


GB 級のデータ

Cosmos DB

9 PM PST





11 PM PST11 PM PST

Read lt 2 ms

Writes lt 6 ms

Read lt 10 ms

Writes lt 15 ms

9950

キーバリュー

ドキュメント

グラフ

SQL (DocumentDB)

MongoDB

Azure Table Storage

Gremlin グラフ


MySQL




SQL



Azure Storage

Standard Table API

01

Strong

Bounded

Staleness

Session

Consistent

Prefix

Eventual

アプリ





45















LOB

Applications

Schema-on-Write




ソースからの

データ収集

Schema-on-Read






機械学習




デバイス


49












Tier 1

MachineLearning










分析処理





SQL Data Warehouse


Data LakeAnalytics



Tier 2


Tier 3

Analytics

Storage




Azure Data Lake

LOB Applications

SocialDevices

Clickstream

Sensors

Video

Web

Relational

HDInsight

Data Lake Analytics

Machine Learning

Spark

R



Azure Storage Blob


HDInsight








(New)

(New)



INSERT

OUTPUT

OUTPUT

SELECThellip

FROMhellip

WHEREhellip

+

PythonR

EXTRACT

EXTRACT

SELECT

SELECT

Azure

Data

Lake

Azure

Data

Lake

Azure

SQL

DB

Azure

Storage

Blobs

Azure

Storage

Blobs

RowSet RowSet

Cognitive Services


rs =

EXTRACT

UserID string

Start DateTime

End DateTime

Region string

SitesVisited string

PagesVisited string













C の関数


に近い）



58

進捗

統計情報


Business apps

Custom apps

Sensors and devices

ADL StorePeople


Azure SQL DW

Azure AD

Power BI

ADF

ADL Analytics


Business apps

Custom apps

Sensors and devices

HDInsight


データの連携

Power BI

File System

Database

Database

Hadoop

DWH

Data Mart










support


















service







computeHadoop





usserviceshdinsight


Store







Store




SQL Server Stretch



Insights

Azure SQL Database

SQL Server 2016


Azure SQL DW

ADLS amp ADLA

DocumentDB

HDInsight

Hadoop



Key


On

-pre

mis

es

Clo

ud






Azure

Data Lake Store

Azure



streaming







just supported

Generally Available





Want Hadoop






premises


Azure HDInsight






(StormSpark)





StudioU-SQL (C+SQL)






If NET or VS shop




Audit

ADLS Apache

Ranger

ADLS AAD Security

Groups (data)

Encryption TD

Audit


Scala Java Python

R

U-SQL T-SQL


External File

Types

ORC TXT

Parquet RCFile

ORC CSV Parquet

+ others

Parquet JSON

Hive + others


RCFile



Schema

Definition

Schema on Write

Polybase


Polybase


= 1PB)






learning-dt





領域改善後









Flexible


ow

er

BI A

pp

Serv

ices

Data

Fact

ory

A

naly

tics

ML

Co

gn

itiv

e B

othellip


Azure Compute

SQL Data

Warehouse

Azure Storage

SQL Database




Azure Database

Services Platform


SQLDB








SQLDB

SQLDB

SQLDB

SQLDB




D51

D52

D53

D60

hellip

D41

D42

D43

D50

hellip


D31

D32

D33

D40hellip

D21

D22

D23

D30

hellip

D11

D12

D13

D20

hellip

D1

D2

D3

D10

hellip

SQL DB





D51

D52

D53

D60


D41

D42

D43

D50

hellip


D31

D32

D33

D40hellip


D21

D22

D23

D30

hellip


D11

D12

D13

D20

hellip


D1

D2

D3

D10

hellip

SQL DB

SQL DB

SQL DB

SQL DB

SQL DB

SQL DB


bull

bull

bull

bull

bull




PB 級のデータ


GB 級のデータ

Cosmos DB

9 PM PST





11 PM PST11 PM PST

Read lt 2 ms

Writes lt 6 ms

Read lt 10 ms

Writes lt 15 ms

9950

キーバリュー

ドキュメント

グラフ

SQL (DocumentDB)

MongoDB

Azure Table Storage

Gremlin グラフ


MySQL




SQL



Azure Storage

Standard Table API

01

Strong

Bounded

Staleness

Session

Consistent

Prefix

Eventual

アプリ





45















LOB

Applications

Schema-on-Write




ソースからの

データ収集

Schema-on-Read






機械学習




デバイス


49












Tier 1

MachineLearning










分析処理





SQL Data Warehouse


Data LakeAnalytics



Tier 2


Tier 3

Analytics

Storage




Azure Data Lake

LOB Applications

SocialDevices

Clickstream

Sensors

Video

Web

Relational

HDInsight

Data Lake Analytics

Machine Learning

Spark

R



Azure Storage Blob


HDInsight








(New)

(New)



INSERT

OUTPUT

OUTPUT

SELECThellip

FROMhellip

WHEREhellip

+

PythonR

EXTRACT

EXTRACT

SELECT

SELECT

Azure

Data

Lake

Azure

Data

Lake

Azure

SQL

DB

Azure

Storage

Blobs

Azure

Storage

Blobs

RowSet RowSet

Cognitive Services


rs =

EXTRACT

UserID string

Start DateTime

End DateTime

Region string

SitesVisited string

PagesVisited string













C の関数


に近い）



58

進捗

統計情報


Business apps

Custom apps

Sensors and devices

ADL StorePeople


Azure SQL DW

Azure AD

Power BI

ADF

ADL Analytics


Business apps

Custom apps

Sensors and devices

HDInsight


データの連携

Power BI

File System

Database

Database

Hadoop

DWH

Data Mart










support


















service







computeHadoop





usserviceshdinsight


Store







Store




SQL Server Stretch



Insights

Azure SQL Database

SQL Server 2016


Azure SQL DW

ADLS amp ADLA

DocumentDB

HDInsight

Hadoop



Key


On

-pre

mis

es

Clo

ud






Azure

Data Lake Store

Azure



streaming







just supported

Generally Available





Want Hadoop






premises


Azure HDInsight






(StormSpark)





StudioU-SQL (C+SQL)






If NET or VS shop




Audit

ADLS Apache

Ranger

ADLS AAD Security

Groups (data)

Encryption TD

Audit


Scala Java Python

R

U-SQL T-SQL


External File

Types

ORC TXT

Parquet RCFile

ORC CSV Parquet

+ others

Parquet JSON

Hive + others


RCFile



Schema

Definition

Schema on Write

Polybase


Polybase


= 1PB)






learning-dt

領域改善後









Flexible


ow

er

BI A

pp

Serv

ices

Data

Fact

ory

A

naly

tics

ML

Co

gn

itiv

e B

othellip


Azure Compute

SQL Data

Warehouse

Azure Storage

SQL Database




Azure Database

Services Platform


SQLDB








SQLDB

SQLDB

SQLDB

SQLDB




D51

D52

D53

D60

hellip

D41

D42

D43

D50

hellip


D31

D32

D33

D40hellip

D21

D22

D23

D30

hellip

D11

D12

D13

D20

hellip

D1

D2

D3

D10

hellip

SQL DB





D51

D52

D53

D60


D41

D42

D43

D50

hellip


D31

D32

D33

D40hellip


D21

D22

D23

D30

hellip


D11

D12

D13

D20

hellip


D1

D2

D3

D10

hellip

SQL DB

SQL DB

SQL DB

SQL DB

SQL DB

SQL DB


bull

bull

bull

bull

bull




PB 級のデータ


GB 級のデータ

Cosmos DB

9 PM PST





11 PM PST11 PM PST

Read lt 2 ms

Writes lt 6 ms

Read lt 10 ms

Writes lt 15 ms

9950

キーバリュー

ドキュメント

グラフ

SQL (DocumentDB)

MongoDB

Azure Table Storage

Gremlin グラフ


MySQL




SQL



Azure Storage

Standard Table API

01

Strong

Bounded

Staleness

Session

Consistent

Prefix

Eventual

アプリ





45















LOB

Applications

Schema-on-Write




ソースからの

データ収集

Schema-on-Read






機械学習




デバイス


49












Tier 1

MachineLearning










分析処理





SQL Data Warehouse


Data LakeAnalytics



Tier 2


Tier 3

Analytics

Storage




Azure Data Lake

LOB Applications

SocialDevices

Clickstream

Sensors

Video

Web

Relational

HDInsight

Data Lake Analytics

Machine Learning

Spark

R



Azure Storage Blob


HDInsight








(New)

(New)



INSERT

OUTPUT

OUTPUT

SELECThellip

FROMhellip

WHEREhellip

+

PythonR

EXTRACT

EXTRACT

SELECT

SELECT

Azure

Data

Lake

Azure

Data

Lake

Azure

SQL

DB

Azure

Storage

Blobs

Azure

Storage

Blobs

RowSet RowSet

Cognitive Services


rs =

EXTRACT

UserID string

Start DateTime

End DateTime

Region string

SitesVisited string

PagesVisited string













C の関数


に近い）



58

進捗

統計情報


Business apps

Custom apps

Sensors and devices

ADL StorePeople


Azure SQL DW

Azure AD

Power BI

ADF

ADL Analytics


Business apps

Custom apps

Sensors and devices

HDInsight


データの連携

Power BI

File System

Database

Database

Hadoop

DWH

Data Mart










support


















service







computeHadoop





usserviceshdinsight


Store







Store




SQL Server Stretch



Insights

Azure SQL Database

SQL Server 2016


Azure SQL DW

ADLS amp ADLA

DocumentDB

HDInsight

Hadoop



Key


On

-pre

mis

es

Clo

ud






Azure

Data Lake Store

Azure



streaming







just supported

Generally Available





Want Hadoop






premises


Azure HDInsight






(StormSpark)





StudioU-SQL (C+SQL)






If NET or VS shop




Audit

ADLS Apache

Ranger

ADLS AAD Security

Groups (data)

Encryption TD

Audit


Scala Java Python

R

U-SQL T-SQL


External File

Types

ORC TXT

Parquet RCFile

ORC CSV Parquet

+ others

Parquet JSON

Hive + others


RCFile



Schema

Definition

Schema on Write

Polybase


Polybase


= 1PB)






learning-dt



Flexible


ow

er

BI A

pp

Serv

ices

Data

Fact

ory

A

naly

tics

ML

Co

gn

itiv

e B

othellip


Azure Compute

SQL Data

Warehouse

Azure Storage

SQL Database




Azure Database

Services Platform


SQLDB








SQLDB

SQLDB

SQLDB

SQLDB




D51

D52

D53

D60

hellip

D41

D42

D43

D50

hellip


D31

D32

D33

D40hellip

D21

D22

D23

D30

hellip

D11

D12

D13

D20

hellip

D1

D2

D3

D10

hellip

SQL DB





D51

D52

D53

D60


D41

D42

D43

D50

hellip


D31

D32

D33

D40hellip


D21

D22

D23

D30

hellip


D11

D12

D13

D20

hellip


D1

D2

D3

D10

hellip

SQL DB

SQL DB

SQL DB

SQL DB

SQL DB

SQL DB


bull

bull

bull

bull

bull




PB 級のデータ


GB 級のデータ

Cosmos DB

9 PM PST





11 PM PST11 PM PST

Read lt 2 ms

Writes lt 6 ms

Read lt 10 ms

Writes lt 15 ms

9950

キーバリュー

ドキュメント

グラフ

SQL (DocumentDB)

MongoDB

Azure Table Storage

Gremlin グラフ


MySQL




SQL



Azure Storage

Standard Table API

01

Strong

Bounded

Staleness

Session

Consistent

Prefix

Eventual

アプリ





45















LOB

Applications

Schema-on-Write




ソースからの

データ収集

Schema-on-Read






機械学習




デバイス


49












Tier 1

MachineLearning










分析処理





SQL Data Warehouse


Data LakeAnalytics



Tier 2


Tier 3

Analytics

Storage




Azure Data Lake

LOB Applications

SocialDevices

Clickstream

Sensors

Video

Web

Relational

HDInsight

Data Lake Analytics

Machine Learning

Spark

R



Azure Storage Blob


HDInsight








(New)

(New)



INSERT

OUTPUT

OUTPUT

SELECThellip

FROMhellip

WHEREhellip

+

PythonR

EXTRACT

EXTRACT

SELECT

SELECT

Azure

Data

Lake

Azure

Data

Lake

Azure

SQL

DB

Azure

Storage

Blobs

Azure

Storage

Blobs

RowSet RowSet

Cognitive Services


rs =

EXTRACT

UserID string

Start DateTime

End DateTime

Region string

SitesVisited string

PagesVisited string













C の関数


に近い）



58

進捗

統計情報


Business apps

Custom apps

Sensors and devices

ADL StorePeople


Azure SQL DW

Azure AD

Power BI

ADF

ADL Analytics


Business apps

Custom apps

Sensors and devices

HDInsight


データの連携

Power BI

File System

Database

Database

Hadoop

DWH

Data Mart










support


















service







computeHadoop





usserviceshdinsight


Store







Store




SQL Server Stretch



Insights

Azure SQL Database

SQL Server 2016


Azure SQL DW

ADLS amp ADLA

DocumentDB

HDInsight

Hadoop



Key


On

-pre

mis

es

Clo

ud






Azure

Data Lake Store

Azure



streaming







just supported

Generally Available





Want Hadoop






premises


Azure HDInsight






(StormSpark)





StudioU-SQL (C+SQL)






If NET or VS shop




Audit

ADLS Apache

Ranger

ADLS AAD Security

Groups (data)

Encryption TD

Audit


Scala Java Python

R

U-SQL T-SQL


External File

Types

ORC TXT

Parquet RCFile

ORC CSV Parquet

+ others

Parquet JSON

Hive + others


RCFile



Schema

Definition

Schema on Write

Polybase


Polybase


= 1PB)






learning-dt

Flexible


ow

er

BI A

pp

Serv

ices

Data

Fact

ory

A

naly

tics

ML

Co

gn

itiv

e B

othellip


Azure Compute

SQL Data

Warehouse

Azure Storage

SQL Database




Azure Database

Services Platform


SQLDB








SQLDB

SQLDB

SQLDB

SQLDB




D51

D52

D53

D60

hellip

D41

D42

D43

D50

hellip


D31

D32

D33

D40hellip

D21

D22

D23

D30

hellip

D11

D12

D13

D20

hellip

D1

D2

D3

D10

hellip

SQL DB





D51

D52

D53

D60


D41

D42

D43

D50

hellip


D31

D32

D33

D40hellip


D21

D22

D23

D30

hellip


D11

D12

D13

D20

hellip


D1

D2

D3

D10

hellip

SQL DB

SQL DB

SQL DB

SQL DB

SQL DB

SQL DB


bull

bull

bull

bull

bull




PB 級のデータ


GB 級のデータ

Cosmos DB

9 PM PST





11 PM PST11 PM PST

Read lt 2 ms

Writes lt 6 ms

Read lt 10 ms

Writes lt 15 ms

9950

キーバリュー

ドキュメント

グラフ

SQL (DocumentDB)

MongoDB

Azure Table Storage

Gremlin グラフ


MySQL




SQL



Azure Storage

Standard Table API

01

Strong

Bounded

Staleness

Session

Consistent

Prefix

Eventual

アプリ





45















LOB

Applications

Schema-on-Write




ソースからの

データ収集

Schema-on-Read






機械学習




デバイス


49












Tier 1

MachineLearning










分析処理





SQL Data Warehouse


Data LakeAnalytics



Tier 2


Tier 3

Analytics

Storage




Azure Data Lake

LOB Applications

SocialDevices

Clickstream

Sensors

Video

Web

Relational

HDInsight

Data Lake Analytics

Machine Learning

Spark

R



Azure Storage Blob


HDInsight








(New)

(New)



INSERT

OUTPUT

OUTPUT

SELECThellip

FROMhellip

WHEREhellip

+

PythonR

EXTRACT

EXTRACT

SELECT

SELECT

Azure

Data

Lake

Azure

Data

Lake

Azure

SQL

DB

Azure

Storage

Blobs

Azure

Storage

Blobs

RowSet RowSet

Cognitive Services


rs =

EXTRACT

UserID string

Start DateTime

End DateTime

Region string

SitesVisited string

PagesVisited string













C の関数


に近い）



58

進捗

統計情報


Business apps

Custom apps

Sensors and devices

ADL StorePeople


Azure SQL DW

Azure AD

Power BI

ADF

ADL Analytics


Business apps

Custom apps

Sensors and devices

HDInsight


データの連携

Power BI

File System

Database

Database

Hadoop

DWH

Data Mart










support


















service







computeHadoop





usserviceshdinsight


Store







Store




SQL Server Stretch



Insights

Azure SQL Database

SQL Server 2016


Azure SQL DW

ADLS amp ADLA

DocumentDB

HDInsight

Hadoop



Key


On

-pre

mis

es

Clo

ud






Azure

Data Lake Store

Azure



streaming







just supported

Generally Available





Want Hadoop






premises


Azure HDInsight






(StormSpark)





StudioU-SQL (C+SQL)






If NET or VS shop




Audit

ADLS Apache

Ranger

ADLS AAD Security

Groups (data)

Encryption TD

Audit


Scala Java Python

R

U-SQL T-SQL


External File

Types

ORC TXT

Parquet RCFile

ORC CSV Parquet

+ others

Parquet JSON

Hive + others


RCFile



Schema

Definition

Schema on Write

Polybase


Polybase


= 1PB)






learning-dt


SQLDB








SQLDB

SQLDB

SQLDB

SQLDB




D51

D52

D53

D60

hellip

D41

D42

D43

D50

hellip


D31

D32

D33

D40hellip

D21

D22

D23

D30

hellip

D11

D12

D13

D20

hellip

D1

D2

D3

D10

hellip

SQL DB





D51

D52

D53

D60


D41

D42

D43

D50

hellip


D31

D32

D33

D40hellip


D21

D22

D23

D30

hellip


D11

D12

D13

D20

hellip


D1

D2

D3

D10

hellip

SQL DB

SQL DB

SQL DB

SQL DB

SQL DB

SQL DB


bull

bull

bull

bull

bull




PB 級のデータ


GB 級のデータ

Cosmos DB

9 PM PST





11 PM PST11 PM PST

Read lt 2 ms

Writes lt 6 ms

Read lt 10 ms

Writes lt 15 ms

9950

キーバリュー

ドキュメント

グラフ

SQL (DocumentDB)

MongoDB

Azure Table Storage

Gremlin グラフ


MySQL




SQL



Azure Storage

Standard Table API

01

Strong

Bounded

Staleness

Session

Consistent

Prefix

Eventual

アプリ





45















LOB

Applications

Schema-on-Write




ソースからの

データ収集

Schema-on-Read






機械学習




デバイス


49












Tier 1

MachineLearning










分析処理





SQL Data Warehouse


Data LakeAnalytics



Tier 2


Tier 3

Analytics

Storage




Azure Data Lake

LOB Applications

SocialDevices

Clickstream

Sensors

Video

Web

Relational

HDInsight

Data Lake Analytics

Machine Learning

Spark

R



Azure Storage Blob


HDInsight








(New)

(New)



INSERT

OUTPUT

OUTPUT

SELECThellip

FROMhellip

WHEREhellip

+

PythonR

EXTRACT

EXTRACT

SELECT

SELECT

Azure

Data

Lake

Azure

Data

Lake

Azure

SQL

DB

Azure

Storage

Blobs

Azure

Storage

Blobs

RowSet RowSet

Cognitive Services


rs =

EXTRACT

UserID string

Start DateTime

End DateTime

Region string

SitesVisited string

PagesVisited string













C の関数


に近い）



58

進捗

統計情報


Business apps

Custom apps

Sensors and devices

ADL StorePeople


Azure SQL DW

Azure AD

Power BI

ADF

ADL Analytics


Business apps

Custom apps

Sensors and devices

HDInsight


データの連携

Power BI

File System

Database

Database

Hadoop

DWH

Data Mart










support


















service







computeHadoop





usserviceshdinsight


Store







Store




SQL Server Stretch



Insights

Azure SQL Database

SQL Server 2016


Azure SQL DW

ADLS amp ADLA

DocumentDB

HDInsight

Hadoop



Key


On

-pre

mis

es

Clo

ud






Azure

Data Lake Store

Azure



streaming







just supported

Generally Available





Want Hadoop






premises


Azure HDInsight






(StormSpark)





StudioU-SQL (C+SQL)






If NET or VS shop




Audit

ADLS Apache

Ranger

ADLS AAD Security

Groups (data)

Encryption TD

Audit


Scala Java Python

R

U-SQL T-SQL


External File

Types

ORC TXT

Parquet RCFile

ORC CSV Parquet

+ others

Parquet JSON

Hive + others


RCFile



Schema

Definition

Schema on Write

Polybase


Polybase


= 1PB)






learning-dt




D51

D52

D53

D60

hellip

D41

D42

D43

D50

hellip


D31

D32

D33

D40hellip

D21

D22

D23

D30

hellip

D11

D12

D13

D20

hellip

D1

D2

D3

D10

hellip

SQL DB





D51

D52

D53

D60


D41

D42

D43

D50

hellip


D31

D32

D33

D40hellip


D21

D22

D23

D30

hellip


D11

D12

D13

D20

hellip


D1

D2

D3

D10

hellip

SQL DB

SQL DB

SQL DB

SQL DB

SQL DB

SQL DB


bull

bull

bull

bull

bull




PB 級のデータ


GB 級のデータ

Cosmos DB

9 PM PST





11 PM PST11 PM PST

Read lt 2 ms

Writes lt 6 ms

Read lt 10 ms

Writes lt 15 ms

9950

キーバリュー

ドキュメント

グラフ

SQL (DocumentDB)

MongoDB

Azure Table Storage

Gremlin グラフ


MySQL




SQL



Azure Storage

Standard Table API

01

Strong

Bounded

Staleness

Session

Consistent

Prefix

Eventual

アプリ





45















LOB

Applications

Schema-on-Write




ソースからの

データ収集

Schema-on-Read






機械学習




デバイス


49












Tier 1

MachineLearning










分析処理





SQL Data Warehouse


Data LakeAnalytics



Tier 2


Tier 3

Analytics

Storage




Azure Data Lake

LOB Applications

SocialDevices

Clickstream

Sensors

Video

Web

Relational

HDInsight

Data Lake Analytics

Machine Learning

Spark

R



Azure Storage Blob


HDInsight








(New)

(New)



INSERT

OUTPUT

OUTPUT

SELECThellip

FROMhellip

WHEREhellip

+

PythonR

EXTRACT

EXTRACT

SELECT

SELECT

Azure

Data

Lake

Azure

Data

Lake

Azure

SQL

DB

Azure

Storage

Blobs

Azure

Storage

Blobs

RowSet RowSet

Cognitive Services


rs =

EXTRACT

UserID string

Start DateTime

End DateTime

Region string

SitesVisited string

PagesVisited string













C の関数


に近い）



58

進捗

統計情報


Business apps

Custom apps

Sensors and devices

ADL StorePeople


Azure SQL DW

Azure AD

Power BI

ADF

ADL Analytics


Business apps

Custom apps

Sensors and devices

HDInsight


データの連携

Power BI

File System

Database

Database

Hadoop

DWH

Data Mart










support


















service







computeHadoop





usserviceshdinsight


Store







Store




SQL Server Stretch



Insights

Azure SQL Database

SQL Server 2016


Azure SQL DW

ADLS amp ADLA

DocumentDB

HDInsight

Hadoop



Key


On

-pre

mis

es

Clo

ud






Azure

Data Lake Store

Azure



streaming







just supported

Generally Available





Want Hadoop






premises


Azure HDInsight






(StormSpark)





StudioU-SQL (C+SQL)






If NET or VS shop




Audit

ADLS Apache

Ranger

ADLS AAD Security

Groups (data)

Encryption TD

Audit


Scala Java Python

R

U-SQL T-SQL


External File

Types

ORC TXT

Parquet RCFile

ORC CSV Parquet

+ others

Parquet JSON

Hive + others


RCFile



Schema

Definition

Schema on Write

Polybase


Polybase


= 1PB)






learning-dt





D51

D52

D53

D60


D41

D42

D43

D50

hellip


D31

D32

D33

D40hellip


D21

D22

D23

D30

hellip


D11

D12

D13

D20

hellip


D1

D2

D3

D10

hellip

SQL DB

SQL DB

SQL DB

SQL DB

SQL DB

SQL DB


bull

bull

bull

bull

bull




PB 級のデータ


GB 級のデータ

Cosmos DB

9 PM PST





11 PM PST11 PM PST

Read lt 2 ms

Writes lt 6 ms

Read lt 10 ms

Writes lt 15 ms

9950

キーバリュー

ドキュメント

グラフ

SQL (DocumentDB)

MongoDB

Azure Table Storage

Gremlin グラフ


MySQL




SQL



Azure Storage

Standard Table API

01

Strong

Bounded

Staleness

Session

Consistent

Prefix

Eventual

アプリ





45















LOB

Applications

Schema-on-Write




ソースからの

データ収集

Schema-on-Read






機械学習




デバイス


49












Tier 1

MachineLearning










分析処理





SQL Data Warehouse


Data LakeAnalytics



Tier 2


Tier 3

Analytics

Storage




Azure Data Lake

LOB Applications

SocialDevices

Clickstream

Sensors

Video

Web

Relational

HDInsight

Data Lake Analytics

Machine Learning

Spark

R



Azure Storage Blob


HDInsight








(New)

(New)



INSERT

OUTPUT

OUTPUT

SELECThellip

FROMhellip

WHEREhellip

+

PythonR

EXTRACT

EXTRACT

SELECT

SELECT

Azure

Data

Lake

Azure

Data

Lake

Azure

SQL

DB

Azure

Storage

Blobs

Azure

Storage

Blobs

RowSet RowSet

Cognitive Services


rs =

EXTRACT

UserID string

Start DateTime

End DateTime

Region string

SitesVisited string

PagesVisited string













C の関数


に近い）



58

進捗

統計情報


Business apps

Custom apps

Sensors and devices

ADL StorePeople


Azure SQL DW

Azure AD

Power BI

ADF

ADL Analytics


Business apps

Custom apps

Sensors and devices

HDInsight


データの連携

Power BI

File System

Database

Database

Hadoop

DWH

Data Mart










support


















service







computeHadoop





usserviceshdinsight


Store







Store




SQL Server Stretch



Insights

Azure SQL Database

SQL Server 2016


Azure SQL DW

ADLS amp ADLA

DocumentDB

HDInsight

Hadoop



Key


On

-pre

mis

es

Clo

ud






Azure

Data Lake Store

Azure



streaming







just supported

Generally Available





Want Hadoop






premises


Azure HDInsight






(StormSpark)





StudioU-SQL (C+SQL)






If NET or VS shop




Audit

ADLS Apache

Ranger

ADLS AAD Security

Groups (data)

Encryption TD

Audit


Scala Java Python

R

U-SQL T-SQL


External File

Types

ORC TXT

Parquet RCFile

ORC CSV Parquet

+ others

Parquet JSON

Hive + others


RCFile



Schema

Definition

Schema on Write

Polybase


Polybase


= 1PB)






learning-dt


bull

bull

bull

bull

bull




PB 級のデータ


GB 級のデータ

Cosmos DB

9 PM PST





11 PM PST11 PM PST

Read lt 2 ms

Writes lt 6 ms

Read lt 10 ms

Writes lt 15 ms

9950

キーバリュー

ドキュメント

グラフ

SQL (DocumentDB)

MongoDB

Azure Table Storage

Gremlin グラフ


MySQL




SQL



Azure Storage

Standard Table API

01

Strong

Bounded

Staleness

Session

Consistent

Prefix

Eventual

アプリ





45















LOB

Applications

Schema-on-Write




ソースからの

データ収集

Schema-on-Read






機械学習




デバイス


49












Tier 1

MachineLearning










分析処理





SQL Data Warehouse


Data LakeAnalytics



Tier 2


Tier 3

Analytics

Storage




Azure Data Lake

LOB Applications

SocialDevices

Clickstream

Sensors

Video

Web

Relational

HDInsight

Data Lake Analytics

Machine Learning

Spark

R



Azure Storage Blob


HDInsight








(New)

(New)



INSERT

OUTPUT

OUTPUT

SELECThellip

FROMhellip

WHEREhellip

+

PythonR

EXTRACT

EXTRACT

SELECT

SELECT

Azure

Data

Lake

Azure

Data

Lake

Azure

SQL

DB

Azure

Storage

Blobs

Azure

Storage

Blobs

RowSet RowSet

Cognitive Services


rs =

EXTRACT

UserID string

Start DateTime

End DateTime

Region string

SitesVisited string

PagesVisited string













C の関数


に近い）



58

進捗

統計情報


Business apps

Custom apps

Sensors and devices

ADL StorePeople


Azure SQL DW

Azure AD

Power BI

ADF

ADL Analytics


Business apps

Custom apps

Sensors and devices

HDInsight


データの連携

Power BI

File System

Database

Database

Hadoop

DWH

Data Mart










support


















service







computeHadoop





usserviceshdinsight


Store







Store




SQL Server Stretch



Insights

Azure SQL Database

SQL Server 2016


Azure SQL DW

ADLS amp ADLA

DocumentDB

HDInsight

Hadoop



Key


On

-pre

mis

es

Clo

ud






Azure

Data Lake Store

Azure



streaming







just supported

Generally Available





Want Hadoop






premises


Azure HDInsight






(StormSpark)





StudioU-SQL (C+SQL)






If NET or VS shop




Audit

ADLS Apache

Ranger

ADLS AAD Security

Groups (data)

Encryption TD

Audit


Scala Java Python

R

U-SQL T-SQL


External File

Types

ORC TXT

Parquet RCFile

ORC CSV Parquet

+ others

Parquet JSON

Hive + others


RCFile



Schema

Definition

Schema on Write

Polybase


Polybase


= 1PB)






learning-dt

bull

bull

bull

bull

bull




PB 級のデータ


GB 級のデータ

Cosmos DB

9 PM PST





11 PM PST11 PM PST

Read lt 2 ms

Writes lt 6 ms

Read lt 10 ms

Writes lt 15 ms

9950

キーバリュー

ドキュメント

グラフ

SQL (DocumentDB)

MongoDB

Azure Table Storage

Gremlin グラフ


MySQL




SQL



Azure Storage

Standard Table API

01

Strong

Bounded

Staleness

Session

Consistent

Prefix

Eventual

アプリ





45















LOB

Applications

Schema-on-Write




ソースからの

データ収集

Schema-on-Read






機械学習




デバイス


49












Tier 1

MachineLearning










分析処理





SQL Data Warehouse


Data LakeAnalytics



Tier 2


Tier 3

Analytics

Storage




Azure Data Lake

LOB Applications

SocialDevices

Clickstream

Sensors

Video

Web

Relational

HDInsight

Data Lake Analytics

Machine Learning

Spark

R



Azure Storage Blob


HDInsight








(New)

(New)



INSERT

OUTPUT

OUTPUT

SELECThellip

FROMhellip

WHEREhellip

+

PythonR

EXTRACT

EXTRACT

SELECT

SELECT

Azure

Data

Lake

Azure

Data

Lake

Azure

SQL

DB

Azure

Storage

Blobs

Azure

Storage

Blobs

RowSet RowSet

Cognitive Services


rs =

EXTRACT

UserID string

Start DateTime

End DateTime

Region string

SitesVisited string

PagesVisited string













C の関数


に近い）



58

進捗

統計情報


Business apps

Custom apps

Sensors and devices

ADL StorePeople


Azure SQL DW

Azure AD

Power BI

ADF

ADL Analytics


Business apps

Custom apps

Sensors and devices

HDInsight


データの連携

Power BI

File System

Database

Database

Hadoop

DWH

Data Mart










support


















service







computeHadoop





usserviceshdinsight


Store







Store




SQL Server Stretch



Insights

Azure SQL Database

SQL Server 2016


Azure SQL DW

ADLS amp ADLA

DocumentDB

HDInsight

Hadoop



Key


On

-pre

mis

es

Clo

ud






Azure

Data Lake Store

Azure



streaming







just supported

Generally Available





Want Hadoop






premises


Azure HDInsight






(StormSpark)





StudioU-SQL (C+SQL)






If NET or VS shop




Audit

ADLS Apache

Ranger

ADLS AAD Security

Groups (data)

Encryption TD

Audit


Scala Java Python

R

U-SQL T-SQL


External File

Types

ORC TXT

Parquet RCFile

ORC CSV Parquet

+ others

Parquet JSON

Hive + others


RCFile



Schema

Definition

Schema on Write

Polybase


Polybase


= 1PB)






learning-dt



PB 級のデータ


GB 級のデータ

Cosmos DB

9 PM PST





11 PM PST11 PM PST

Read lt 2 ms

Writes lt 6 ms

Read lt 10 ms

Writes lt 15 ms

9950

キーバリュー

ドキュメント

グラフ

SQL (DocumentDB)

MongoDB

Azure Table Storage

Gremlin グラフ


MySQL




SQL



Azure Storage

Standard Table API

01

Strong

Bounded

Staleness

Session

Consistent

Prefix

Eventual

アプリ





45















LOB

Applications

Schema-on-Write




ソースからの

データ収集

Schema-on-Read






機械学習




デバイス


49












Tier 1

MachineLearning










分析処理





SQL Data Warehouse


Data LakeAnalytics



Tier 2


Tier 3

Analytics

Storage




Azure Data Lake

LOB Applications

SocialDevices

Clickstream

Sensors

Video

Web

Relational

HDInsight

Data Lake Analytics

Machine Learning

Spark

R



Azure Storage Blob


HDInsight








(New)

(New)



INSERT

OUTPUT

OUTPUT

SELECThellip

FROMhellip

WHEREhellip

+

PythonR

EXTRACT

EXTRACT

SELECT

SELECT

Azure

Data

Lake

Azure

Data

Lake

Azure

SQL

DB

Azure

Storage

Blobs

Azure

Storage

Blobs

RowSet RowSet

Cognitive Services


rs =

EXTRACT

UserID string

Start DateTime

End DateTime

Region string

SitesVisited string

PagesVisited string













C の関数


に近い）



58

進捗

統計情報


Business apps

Custom apps

Sensors and devices

ADL StorePeople


Azure SQL DW

Azure AD

Power BI

ADF

ADL Analytics


Business apps

Custom apps

Sensors and devices

HDInsight


データの連携

Power BI

File System

Database

Database

Hadoop

DWH

Data Mart










support


















service







computeHadoop





usserviceshdinsight


Store







Store




SQL Server Stretch



Insights

Azure SQL Database

SQL Server 2016


Azure SQL DW

ADLS amp ADLA

DocumentDB

HDInsight

Hadoop



Key


On

-pre

mis

es

Clo

ud






Azure

Data Lake Store

Azure



streaming







just supported

Generally Available





Want Hadoop






premises


Azure HDInsight






(StormSpark)





StudioU-SQL (C+SQL)






If NET or VS shop




Audit

ADLS Apache

Ranger

ADLS AAD Security

Groups (data)

Encryption TD

Audit


Scala Java Python

R

U-SQL T-SQL


External File

Types

ORC TXT

Parquet RCFile

ORC CSV Parquet

+ others

Parquet JSON

Hive + others


RCFile



Schema

Definition

Schema on Write

Polybase


Polybase


= 1PB)






learning-dt

9 PM PST





11 PM PST11 PM PST

Read lt 2 ms

Writes lt 6 ms

Read lt 10 ms

Writes lt 15 ms

9950

キーバリュー

ドキュメント

グラフ

SQL (DocumentDB)

MongoDB

Azure Table Storage

Gremlin グラフ


MySQL




SQL



Azure Storage

Standard Table API

01

Strong

Bounded

Staleness

Session

Consistent

Prefix

Eventual

アプリ





45















LOB

Applications

Schema-on-Write




ソースからの

データ収集

Schema-on-Read






機械学習




デバイス


49












Tier 1

MachineLearning










分析処理





SQL Data Warehouse


Data LakeAnalytics



Tier 2


Tier 3

Analytics

Storage




Azure Data Lake

LOB Applications

SocialDevices

Clickstream

Sensors

Video

Web

Relational

HDInsight

Data Lake Analytics

Machine Learning

Spark

R



Azure Storage Blob


HDInsight








(New)

(New)



INSERT

OUTPUT

OUTPUT

SELECThellip

FROMhellip

WHEREhellip

+

PythonR

EXTRACT

EXTRACT

SELECT

SELECT

Azure

Data

Lake

Azure

Data

Lake

Azure

SQL

DB

Azure

Storage

Blobs

Azure

Storage

Blobs

RowSet RowSet

Cognitive Services


rs =

EXTRACT

UserID string

Start DateTime

End DateTime

Region string

SitesVisited string

PagesVisited string













C の関数


に近い）



58

進捗

統計情報


Business apps

Custom apps

Sensors and devices

ADL StorePeople


Azure SQL DW

Azure AD

Power BI

ADF

ADL Analytics


Business apps

Custom apps

Sensors and devices

HDInsight


データの連携

Power BI

File System

Database

Database

Hadoop

DWH

Data Mart










support


















service







computeHadoop





usserviceshdinsight


Store







Store




SQL Server Stretch



Insights

Azure SQL Database

SQL Server 2016


Azure SQL DW

ADLS amp ADLA

DocumentDB

HDInsight

Hadoop



Key


On

-pre

mis

es

Clo

ud






Azure

Data Lake Store

Azure



streaming







just supported

Generally Available





Want Hadoop






premises


Azure HDInsight






(StormSpark)





StudioU-SQL (C+SQL)






If NET or VS shop




Audit

ADLS Apache

Ranger

ADLS AAD Security

Groups (data)

Encryption TD

Audit


Scala Java Python

R

U-SQL T-SQL


External File

Types

ORC TXT

Parquet RCFile

ORC CSV Parquet

+ others

Parquet JSON

Hive + others


RCFile



Schema

Definition

Schema on Write

Polybase


Polybase


= 1PB)






learning-dt

Read lt 2 ms

Writes lt 6 ms

Read lt 10 ms

Writes lt 15 ms

9950

キーバリュー

ドキュメント

グラフ

SQL (DocumentDB)

MongoDB

Azure Table Storage

Gremlin グラフ


MySQL




SQL



Azure Storage

Standard Table API

01

Strong

Bounded

Staleness

Session

Consistent

Prefix

Eventual

アプリ





45















LOB

Applications

Schema-on-Write




ソースからの

データ収集

Schema-on-Read






機械学習




デバイス


49












Tier 1

MachineLearning










分析処理





SQL Data Warehouse


Data LakeAnalytics



Tier 2


Tier 3

Analytics

Storage




Azure Data Lake

LOB Applications

SocialDevices

Clickstream

Sensors

Video

Web

Relational

HDInsight

Data Lake Analytics

Machine Learning

Spark

R



Azure Storage Blob


HDInsight








(New)

(New)



INSERT

OUTPUT

OUTPUT

SELECThellip

FROMhellip

WHEREhellip

+

PythonR

EXTRACT

EXTRACT

SELECT

SELECT

Azure

Data

Lake

Azure

Data

Lake

Azure

SQL

DB

Azure

Storage

Blobs

Azure

Storage

Blobs

RowSet RowSet

Cognitive Services


rs =

EXTRACT

UserID string

Start DateTime

End DateTime

Region string

SitesVisited string

PagesVisited string













C の関数


に近い）



58

進捗

統計情報


Business apps

Custom apps

Sensors and devices

ADL StorePeople


Azure SQL DW

Azure AD

Power BI

ADF

ADL Analytics


Business apps

Custom apps

Sensors and devices

HDInsight


データの連携

Power BI

File System

Database

Database

Hadoop

DWH

Data Mart










support


















service







computeHadoop





usserviceshdinsight


Store







Store




SQL Server Stretch



Insights

Azure SQL Database

SQL Server 2016


Azure SQL DW

ADLS amp ADLA

DocumentDB

HDInsight

Hadoop



Key


On

-pre

mis

es

Clo

ud






Azure

Data Lake Store

Azure



streaming







just supported

Generally Available





Want Hadoop






premises


Azure HDInsight






(StormSpark)





StudioU-SQL (C+SQL)






If NET or VS shop




Audit

ADLS Apache

Ranger

ADLS AAD Security

Groups (data)

Encryption TD

Audit


Scala Java Python

R

U-SQL T-SQL


External File

Types

ORC TXT

Parquet RCFile

ORC CSV Parquet

+ others

Parquet JSON

Hive + others


RCFile



Schema

Definition

Schema on Write

Polybase


Polybase


= 1PB)






learning-dt

キーバリュー

ドキュメント

グラフ

SQL (DocumentDB)

MongoDB

Azure Table Storage

Gremlin グラフ


MySQL




SQL



Azure Storage

Standard Table API

01

Strong

Bounded

Staleness

Session

Consistent

Prefix

Eventual

アプリ





45















LOB

Applications

Schema-on-Write




ソースからの

データ収集

Schema-on-Read






機械学習




デバイス


49












Tier 1

MachineLearning










分析処理





SQL Data Warehouse


Data LakeAnalytics



Tier 2


Tier 3

Analytics

Storage




Azure Data Lake

LOB Applications

SocialDevices

Clickstream

Sensors

Video

Web

Relational

HDInsight

Data Lake Analytics

Machine Learning

Spark

R



Azure Storage Blob


HDInsight








(New)

(New)



INSERT

OUTPUT

OUTPUT

SELECThellip

FROMhellip

WHEREhellip

+

PythonR

EXTRACT

EXTRACT

SELECT

SELECT

Azure

Data

Lake

Azure

Data

Lake

Azure

SQL

DB

Azure

Storage

Blobs

Azure

Storage

Blobs

RowSet RowSet

Cognitive Services


rs =

EXTRACT

UserID string

Start DateTime

End DateTime

Region string

SitesVisited string

PagesVisited string













C の関数


に近い）



58

進捗

統計情報


Business apps

Custom apps

Sensors and devices

ADL StorePeople


Azure SQL DW

Azure AD

Power BI

ADF

ADL Analytics


Business apps

Custom apps

Sensors and devices

HDInsight


データの連携

Power BI

File System

Database

Database

Hadoop

DWH

Data Mart










support


















service







computeHadoop





usserviceshdinsight


Store







Store




SQL Server Stretch



Insights

Azure SQL Database

SQL Server 2016


Azure SQL DW

ADLS amp ADLA

DocumentDB

HDInsight

Hadoop



Key


On

-pre

mis

es

Clo

ud






Azure

Data Lake Store

Azure



streaming







just supported

Generally Available





Want Hadoop






premises


Azure HDInsight






(StormSpark)





StudioU-SQL (C+SQL)






If NET or VS shop




Audit

ADLS Apache

Ranger

ADLS AAD Security

Groups (data)

Encryption TD

Audit


Scala Java Python

R

U-SQL T-SQL


External File

Types

ORC TXT

Parquet RCFile

ORC CSV Parquet

+ others

Parquet JSON

Hive + others


RCFile



Schema

Definition

Schema on Write

Polybase


Polybase


= 1PB)






learning-dt


MySQL




SQL



Azure Storage

Standard Table API

01

Strong

Bounded

Staleness

Session

Consistent

Prefix

Eventual

アプリ





45















LOB

Applications

Schema-on-Write




ソースからの

データ収集

Schema-on-Read






機械学習




デバイス


49












Tier 1

MachineLearning










分析処理





SQL Data Warehouse


Data LakeAnalytics



Tier 2


Tier 3

Analytics

Storage




Azure Data Lake

LOB Applications

SocialDevices

Clickstream

Sensors

Video

Web

Relational

HDInsight

Data Lake Analytics

Machine Learning

Spark

R



Azure Storage Blob


HDInsight








(New)

(New)



INSERT

OUTPUT

OUTPUT

SELECThellip

FROMhellip

WHEREhellip

+

PythonR

EXTRACT

EXTRACT

SELECT

SELECT

Azure

Data

Lake

Azure

Data

Lake

Azure

SQL

DB

Azure

Storage

Blobs

Azure

Storage

Blobs

RowSet RowSet

Cognitive Services


rs =

EXTRACT

UserID string

Start DateTime

End DateTime

Region string

SitesVisited string

PagesVisited string













C の関数


に近い）



58

進捗

統計情報


Business apps

Custom apps

Sensors and devices

ADL StorePeople


Azure SQL DW

Azure AD

Power BI

ADF

ADL Analytics


Business apps

Custom apps

Sensors and devices

HDInsight


データの連携

Power BI

File System

Database

Database

Hadoop

DWH

Data Mart










support


















service







computeHadoop





usserviceshdinsight


Store







Store




SQL Server Stretch



Insights

Azure SQL Database

SQL Server 2016


Azure SQL DW

ADLS amp ADLA

DocumentDB

HDInsight

Hadoop



Key


On

-pre

mis

es

Clo

ud






Azure

Data Lake Store

Azure



streaming







just supported

Generally Available





Want Hadoop






premises


Azure HDInsight






(StormSpark)





StudioU-SQL (C+SQL)






If NET or VS shop




Audit

ADLS Apache

Ranger

ADLS AAD Security

Groups (data)

Encryption TD

Audit


Scala Java Python

R

U-SQL T-SQL


External File

Types

ORC TXT

Parquet RCFile

ORC CSV Parquet

+ others

Parquet JSON

Hive + others


RCFile



Schema

Definition

Schema on Write

Polybase


Polybase


= 1PB)






learning-dt

01

Strong

Bounded

Staleness

Session

Consistent

Prefix

Eventual

アプリ





45















LOB

Applications

Schema-on-Write




ソースからの

データ収集

Schema-on-Read






機械学習




デバイス


49












Tier 1

MachineLearning










分析処理





SQL Data Warehouse


Data LakeAnalytics



Tier 2


Tier 3

Analytics

Storage




Azure Data Lake

LOB Applications

SocialDevices

Clickstream

Sensors

Video

Web

Relational

HDInsight

Data Lake Analytics

Machine Learning

Spark

R



Azure Storage Blob


HDInsight








(New)

(New)



INSERT

OUTPUT

OUTPUT

SELECThellip

FROMhellip

WHEREhellip

+

PythonR

EXTRACT

EXTRACT

SELECT

SELECT

Azure

Data

Lake

Azure

Data

Lake

Azure

SQL

DB

Azure

Storage

Blobs

Azure

Storage

Blobs

RowSet RowSet

Cognitive Services


rs =

EXTRACT

UserID string

Start DateTime

End DateTime

Region string

SitesVisited string

PagesVisited string













C の関数


に近い）



58

進捗

統計情報


Business apps

Custom apps

Sensors and devices

ADL StorePeople


Azure SQL DW

Azure AD

Power BI

ADF

ADL Analytics


Business apps

Custom apps

Sensors and devices

HDInsight


データの連携

Power BI

File System

Database

Database

Hadoop

DWH

Data Mart










support


















service







computeHadoop





usserviceshdinsight


Store







Store




SQL Server Stretch



Insights

Azure SQL Database

SQL Server 2016


Azure SQL DW

ADLS amp ADLA

DocumentDB

HDInsight

Hadoop



Key


On

-pre

mis

es

Clo

ud






Azure

Data Lake Store

Azure



streaming







just supported

Generally Available





Want Hadoop






premises


Azure HDInsight






(StormSpark)





StudioU-SQL (C+SQL)






If NET or VS shop




Audit

ADLS Apache

Ranger

ADLS AAD Security

Groups (data)

Encryption TD

Audit


Scala Java Python

R

U-SQL T-SQL


External File

Types

ORC TXT

Parquet RCFile

ORC CSV Parquet

+ others

Parquet JSON

Hive + others


RCFile



Schema

Definition

Schema on Write

Polybase


Polybase


= 1PB)






learning-dt




45















LOB

Applications

Schema-on-Write




ソースからの

データ収集

Schema-on-Read






機械学習




デバイス


49












Tier 1

MachineLearning










分析処理





SQL Data Warehouse


Data LakeAnalytics



Tier 2


Tier 3

Analytics

Storage




Azure Data Lake

LOB Applications

SocialDevices

Clickstream

Sensors

Video

Web

Relational

HDInsight

Data Lake Analytics

Machine Learning

Spark

R



Azure Storage Blob


HDInsight








(New)

(New)



INSERT

OUTPUT

OUTPUT

SELECThellip

FROMhellip

WHEREhellip

+

PythonR

EXTRACT

EXTRACT

SELECT

SELECT

Azure

Data

Lake

Azure

Data

Lake

Azure

SQL

DB

Azure

Storage

Blobs

Azure

Storage

Blobs

RowSet RowSet

Cognitive Services


rs =

EXTRACT

UserID string

Start DateTime

End DateTime

Region string

SitesVisited string

PagesVisited string













C の関数


に近い）



58

進捗

統計情報


Business apps

Custom apps

Sensors and devices

ADL StorePeople


Azure SQL DW

Azure AD

Power BI

ADF

ADL Analytics


Business apps

Custom apps

Sensors and devices

HDInsight


データの連携

Power BI

File System

Database

Database

Hadoop

DWH

Data Mart










support


















service







computeHadoop





usserviceshdinsight


Store







Store




SQL Server Stretch



Insights

Azure SQL Database

SQL Server 2016


Azure SQL DW

ADLS amp ADLA

DocumentDB

HDInsight

Hadoop



Key


On

-pre

mis

es

Clo

ud






Azure

Data Lake Store

Azure



streaming







just supported

Generally Available





Want Hadoop






premises


Azure HDInsight






(StormSpark)





StudioU-SQL (C+SQL)






If NET or VS shop




Audit

ADLS Apache

Ranger

ADLS AAD Security

Groups (data)

Encryption TD

Audit


Scala Java Python

R

U-SQL T-SQL


External File

Types

ORC TXT

Parquet RCFile

ORC CSV Parquet

+ others

Parquet JSON

Hive + others


RCFile



Schema

Definition

Schema on Write

Polybase


Polybase


= 1PB)






learning-dt


45















LOB

Applications

Schema-on-Write




ソースからの

データ収集

Schema-on-Read






機械学習




デバイス


49












Tier 1

MachineLearning










分析処理





SQL Data Warehouse


Data LakeAnalytics



Tier 2


Tier 3

Analytics

Storage




Azure Data Lake

LOB Applications

SocialDevices

Clickstream

Sensors

Video

Web

Relational

HDInsight

Data Lake Analytics

Machine Learning

Spark

R



Azure Storage Blob


HDInsight








(New)

(New)



INSERT

OUTPUT

OUTPUT

SELECThellip

FROMhellip

WHEREhellip

+

PythonR

EXTRACT

EXTRACT

SELECT

SELECT

Azure

Data

Lake

Azure

Data

Lake

Azure

SQL

DB

Azure

Storage

Blobs

Azure

Storage

Blobs

RowSet RowSet

Cognitive Services


rs =

EXTRACT

UserID string

Start DateTime

End DateTime

Region string

SitesVisited string

PagesVisited string













C の関数


に近い）



58

進捗

統計情報


Business apps

Custom apps

Sensors and devices

ADL StorePeople


Azure SQL DW

Azure AD

Power BI

ADF

ADL Analytics


Business apps

Custom apps

Sensors and devices

HDInsight


データの連携

Power BI

File System

Database

Database

Hadoop

DWH

Data Mart










support


















service







computeHadoop





usserviceshdinsight


Store







Store




SQL Server Stretch



Insights

Azure SQL Database

SQL Server 2016


Azure SQL DW

ADLS amp ADLA

DocumentDB

HDInsight

Hadoop



Key


On

-pre

mis

es

Clo

ud






Azure

Data Lake Store

Azure



streaming







just supported

Generally Available





Want Hadoop






premises


Azure HDInsight






(StormSpark)





StudioU-SQL (C+SQL)






If NET or VS shop




Audit

ADLS Apache

Ranger

ADLS AAD Security

Groups (data)

Encryption TD

Audit


Scala Java Python

R

U-SQL T-SQL


External File

Types

ORC TXT

Parquet RCFile

ORC CSV Parquet

+ others

Parquet JSON

Hive + others


RCFile



Schema

Definition

Schema on Write

Polybase


Polybase


= 1PB)






learning-dt

45















LOB

Applications

Schema-on-Write




ソースからの

データ収集

Schema-on-Read






機械学習




デバイス


49












Tier 1

MachineLearning










分析処理





SQL Data Warehouse


Data LakeAnalytics



Tier 2


Tier 3

Analytics

Storage




Azure Data Lake

LOB Applications

SocialDevices

Clickstream

Sensors

Video

Web

Relational

HDInsight

Data Lake Analytics

Machine Learning

Spark

R



Azure Storage Blob


HDInsight








(New)

(New)



INSERT

OUTPUT

OUTPUT

SELECThellip

FROMhellip

WHEREhellip

+

PythonR

EXTRACT

EXTRACT

SELECT

SELECT

Azure

Data

Lake

Azure

Data

Lake

Azure

SQL

DB

Azure

Storage

Blobs

Azure

Storage

Blobs

RowSet RowSet

Cognitive Services


rs =

EXTRACT

UserID string

Start DateTime

End DateTime

Region string

SitesVisited string

PagesVisited string













C の関数


に近い）



58

進捗

統計情報


Business apps

Custom apps

Sensors and devices

ADL StorePeople


Azure SQL DW

Azure AD

Power BI

ADF

ADL Analytics


Business apps

Custom apps

Sensors and devices

HDInsight


データの連携

Power BI

File System

Database

Database

Hadoop

DWH

Data Mart










support


















service







computeHadoop





usserviceshdinsight


Store







Store




SQL Server Stretch



Insights

Azure SQL Database

SQL Server 2016


Azure SQL DW

ADLS amp ADLA

DocumentDB

HDInsight

Hadoop



Key


On

-pre

mis

es

Clo

ud






Azure

Data Lake Store

Azure



streaming







just supported

Generally Available





Want Hadoop






premises


Azure HDInsight






(StormSpark)





StudioU-SQL (C+SQL)






If NET or VS shop




Audit

ADLS Apache

Ranger

ADLS AAD Security

Groups (data)

Encryption TD

Audit


Scala Java Python

R

U-SQL T-SQL


External File

Types

ORC TXT

Parquet RCFile

ORC CSV Parquet

+ others

Parquet JSON

Hive + others


RCFile



Schema

Definition

Schema on Write

Polybase


Polybase


= 1PB)






learning-dt



LOB

Applications

Schema-on-Write




ソースからの

データ収集

Schema-on-Read






機械学習




デバイス


49












Tier 1

MachineLearning










分析処理





SQL Data Warehouse


Data LakeAnalytics



Tier 2


Tier 3

Analytics

Storage




Azure Data Lake

LOB Applications

SocialDevices

Clickstream

Sensors

Video

Web

Relational

HDInsight

Data Lake Analytics

Machine Learning

Spark

R



Azure Storage Blob


HDInsight








(New)

(New)



INSERT

OUTPUT

OUTPUT

SELECThellip

FROMhellip

WHEREhellip

+

PythonR

EXTRACT

EXTRACT

SELECT

SELECT

Azure

Data

Lake

Azure

Data

Lake

Azure

SQL

DB

Azure

Storage

Blobs

Azure

Storage

Blobs

RowSet RowSet

Cognitive Services


rs =

EXTRACT

UserID string

Start DateTime

End DateTime

Region string

SitesVisited string

PagesVisited string













C の関数


に近い）



58

進捗

統計情報


Business apps

Custom apps

Sensors and devices

ADL StorePeople


Azure SQL DW

Azure AD

Power BI

ADF

ADL Analytics


Business apps

Custom apps

Sensors and devices

HDInsight


データの連携

Power BI

File System

Database

Database

Hadoop

DWH

Data Mart










support


















service







computeHadoop





usserviceshdinsight


Store







Store




SQL Server Stretch



Insights

Azure SQL Database

SQL Server 2016


Azure SQL DW

ADLS amp ADLA

DocumentDB

HDInsight

Hadoop



Key


On

-pre

mis

es

Clo

ud






Azure

Data Lake Store

Azure



streaming







just supported

Generally Available





Want Hadoop






premises


Azure HDInsight






(StormSpark)





StudioU-SQL (C+SQL)






If NET or VS shop




Audit

ADLS Apache

Ranger

ADLS AAD Security

Groups (data)

Encryption TD

Audit


Scala Java Python

R

U-SQL T-SQL


External File

Types

ORC TXT

Parquet RCFile

ORC CSV Parquet

+ others

Parquet JSON

Hive + others


RCFile



Schema

Definition

Schema on Write

Polybase


Polybase


= 1PB)






learning-dt



ソースからの

データ収集

Schema-on-Read






機械学習




デバイス


49












Tier 1

MachineLearning










分析処理





SQL Data Warehouse


Data LakeAnalytics



Tier 2


Tier 3

Analytics

Storage




Azure Data Lake

LOB Applications

SocialDevices

Clickstream

Sensors

Video

Web

Relational

HDInsight

Data Lake Analytics

Machine Learning

Spark

R



Azure Storage Blob


HDInsight








(New)

(New)



INSERT

OUTPUT

OUTPUT

SELECThellip

FROMhellip

WHEREhellip

+

PythonR

EXTRACT

EXTRACT

SELECT

SELECT

Azure

Data

Lake

Azure

Data

Lake

Azure

SQL

DB

Azure

Storage

Blobs

Azure

Storage

Blobs

RowSet RowSet

Cognitive Services


rs =

EXTRACT

UserID string

Start DateTime

End DateTime

Region string

SitesVisited string

PagesVisited string













C の関数


に近い）



58

進捗

統計情報


Business apps

Custom apps

Sensors and devices

ADL StorePeople


Azure SQL DW

Azure AD

Power BI

ADF

ADL Analytics


Business apps

Custom apps

Sensors and devices

HDInsight


データの連携

Power BI

File System

Database

Database

Hadoop

DWH

Data Mart










support


















service







computeHadoop





usserviceshdinsight


Store







Store




SQL Server Stretch



Insights

Azure SQL Database

SQL Server 2016


Azure SQL DW

ADLS amp ADLA

DocumentDB

HDInsight

Hadoop



Key


On

-pre

mis

es

Clo

ud






Azure

Data Lake Store

Azure



streaming







just supported

Generally Available





Want Hadoop






premises


Azure HDInsight






(StormSpark)





StudioU-SQL (C+SQL)






If NET or VS shop




Audit

ADLS Apache

Ranger

ADLS AAD Security

Groups (data)

Encryption TD

Audit


Scala Java Python

R

U-SQL T-SQL


External File

Types

ORC TXT

Parquet RCFile

ORC CSV Parquet

+ others

Parquet JSON

Hive + others


RCFile



Schema

Definition

Schema on Write

Polybase


Polybase


= 1PB)






learning-dt






機械学習




デバイス


49












Tier 1

MachineLearning










分析処理





SQL Data Warehouse


Data LakeAnalytics



Tier 2


Tier 3

Analytics

Storage




Azure Data Lake

LOB Applications

SocialDevices

Clickstream

Sensors

Video

Web

Relational

HDInsight

Data Lake Analytics

Machine Learning

Spark

R



Azure Storage Blob


HDInsight








(New)

(New)



INSERT

OUTPUT

OUTPUT

SELECThellip

FROMhellip

WHEREhellip

+

PythonR

EXTRACT

EXTRACT

SELECT

SELECT

Azure

Data

Lake

Azure

Data

Lake

Azure

SQL

DB

Azure

Storage

Blobs

Azure

Storage

Blobs

RowSet RowSet

Cognitive Services


rs =

EXTRACT

UserID string

Start DateTime

End DateTime

Region string

SitesVisited string

PagesVisited string













C の関数


に近い）



58

進捗

統計情報


Business apps

Custom apps

Sensors and devices

ADL StorePeople


Azure SQL DW

Azure AD

Power BI

ADF

ADL Analytics


Business apps

Custom apps

Sensors and devices

HDInsight


データの連携

Power BI

File System

Database

Database

Hadoop

DWH

Data Mart










support


















service







computeHadoop





usserviceshdinsight


Store







Store




SQL Server Stretch



Insights

Azure SQL Database

SQL Server 2016


Azure SQL DW

ADLS amp ADLA

DocumentDB

HDInsight

Hadoop



Key


On

-pre

mis

es

Clo

ud






Azure

Data Lake Store

Azure



streaming







just supported

Generally Available





Want Hadoop






premises


Azure HDInsight






(StormSpark)





StudioU-SQL (C+SQL)






If NET or VS shop




Audit

ADLS Apache

Ranger

ADLS AAD Security

Groups (data)

Encryption TD

Audit


Scala Java Python

R

U-SQL T-SQL


External File

Types

ORC TXT

Parquet RCFile

ORC CSV Parquet

+ others

Parquet JSON

Hive + others


RCFile



Schema

Definition

Schema on Write

Polybase


Polybase


= 1PB)






learning-dt

49












Tier 1

MachineLearning










分析処理





SQL Data Warehouse


Data LakeAnalytics



Tier 2


Tier 3

Analytics

Storage




Azure Data Lake

LOB Applications

SocialDevices

Clickstream

Sensors

Video

Web

Relational

HDInsight

Data Lake Analytics

Machine Learning

Spark

R



Azure Storage Blob


HDInsight








(New)

(New)



INSERT

OUTPUT

OUTPUT

SELECThellip

FROMhellip

WHEREhellip

+

PythonR

EXTRACT

EXTRACT

SELECT

SELECT

Azure

Data

Lake

Azure

Data

Lake

Azure

SQL

DB

Azure

Storage

Blobs

Azure

Storage

Blobs

RowSet RowSet

Cognitive Services


rs =

EXTRACT

UserID string

Start DateTime

End DateTime

Region string

SitesVisited string

PagesVisited string













C の関数


に近い）



58

進捗

統計情報


Business apps

Custom apps

Sensors and devices

ADL StorePeople


Azure SQL DW

Azure AD

Power BI

ADF

ADL Analytics


Business apps

Custom apps

Sensors and devices

HDInsight


データの連携

Power BI

File System

Database

Database

Hadoop

DWH

Data Mart










support


















service







computeHadoop





usserviceshdinsight


Store







Store




SQL Server Stretch



Insights

Azure SQL Database

SQL Server 2016


Azure SQL DW

ADLS amp ADLA

DocumentDB

HDInsight

Hadoop



Key


On

-pre

mis

es

Clo

ud






Azure

Data Lake Store

Azure



streaming







just supported

Generally Available





Want Hadoop






premises


Azure HDInsight






(StormSpark)





StudioU-SQL (C+SQL)






If NET or VS shop




Audit

ADLS Apache

Ranger

ADLS AAD Security

Groups (data)

Encryption TD

Audit


Scala Java Python

R

U-SQL T-SQL


External File

Types

ORC TXT

Parquet RCFile

ORC CSV Parquet

+ others

Parquet JSON

Hive + others


RCFile



Schema

Definition

Schema on Write

Polybase


Polybase


= 1PB)






learning-dt



Tier 1

MachineLearning










分析処理





SQL Data Warehouse


Data LakeAnalytics



Tier 2


Tier 3

Analytics

Storage




Azure Data Lake

LOB Applications

SocialDevices

Clickstream

Sensors

Video

Web

Relational

HDInsight

Data Lake Analytics

Machine Learning

Spark

R



Azure Storage Blob


HDInsight








(New)

(New)



INSERT

OUTPUT

OUTPUT

SELECThellip

FROMhellip

WHEREhellip

+

PythonR

EXTRACT

EXTRACT

SELECT

SELECT

Azure

Data

Lake

Azure

Data

Lake

Azure

SQL

DB

Azure

Storage

Blobs

Azure

Storage

Blobs

RowSet RowSet

Cognitive Services


rs =

EXTRACT

UserID string

Start DateTime

End DateTime

Region string

SitesVisited string

PagesVisited string













C の関数


に近い）



58

進捗

統計情報


Business apps

Custom apps

Sensors and devices

ADL StorePeople


Azure SQL DW

Azure AD

Power BI

ADF

ADL Analytics


Business apps

Custom apps

Sensors and devices

HDInsight


データの連携

Power BI

File System

Database

Database

Hadoop

DWH

Data Mart










support


















service







computeHadoop





usserviceshdinsight


Store







Store




SQL Server Stretch



Insights

Azure SQL Database

SQL Server 2016


Azure SQL DW

ADLS amp ADLA

DocumentDB

HDInsight

Hadoop



Key


On

-pre

mis

es

Clo

ud






Azure

Data Lake Store

Azure



streaming







just supported

Generally Available





Want Hadoop






premises


Azure HDInsight






(StormSpark)





StudioU-SQL (C+SQL)






If NET or VS shop




Audit

ADLS Apache

Ranger

ADLS AAD Security

Groups (data)

Encryption TD

Audit


Scala Java Python

R

U-SQL T-SQL


External File

Types

ORC TXT

Parquet RCFile

ORC CSV Parquet

+ others

Parquet JSON

Hive + others


RCFile



Schema

Definition

Schema on Write

Polybase


Polybase


= 1PB)






learning-dt

Analytics

Storage




Azure Data Lake

LOB Applications

SocialDevices

Clickstream

Sensors

Video

Web

Relational

HDInsight

Data Lake Analytics

Machine Learning

Spark

R



Azure Storage Blob


HDInsight








(New)

(New)



INSERT

OUTPUT

OUTPUT

SELECThellip

FROMhellip

WHEREhellip

+

PythonR

EXTRACT

EXTRACT

SELECT

SELECT

Azure

Data

Lake

Azure

Data

Lake

Azure

SQL

DB

Azure

Storage

Blobs

Azure

Storage

Blobs

RowSet RowSet

Cognitive Services


rs =

EXTRACT

UserID string

Start DateTime

End DateTime

Region string

SitesVisited string

PagesVisited string













C の関数


に近い）



58

進捗

統計情報


Business apps

Custom apps

Sensors and devices

ADL StorePeople


Azure SQL DW

Azure AD

Power BI

ADF

ADL Analytics


Business apps

Custom apps

Sensors and devices

HDInsight


データの連携

Power BI

File System

Database

Database

Hadoop

DWH

Data Mart










support


















service







computeHadoop





usserviceshdinsight


Store







Store




SQL Server Stretch



Insights

Azure SQL Database

SQL Server 2016


Azure SQL DW

ADLS amp ADLA

DocumentDB

HDInsight

Hadoop



Key


On

-pre

mis

es

Clo

ud






Azure

Data Lake Store

Azure



streaming







just supported

Generally Available





Want Hadoop






premises


Azure HDInsight






(StormSpark)





StudioU-SQL (C+SQL)






If NET or VS shop




Audit

ADLS Apache

Ranger

ADLS AAD Security

Groups (data)

Encryption TD

Audit


Scala Java Python

R

U-SQL T-SQL


External File

Types

ORC TXT

Parquet RCFile

ORC CSV Parquet

+ others

Parquet JSON

Hive + others


RCFile



Schema

Definition

Schema on Write

Polybase


Polybase


= 1PB)






learning-dt

LOB Applications

SocialDevices

Clickstream

Sensors

Video

Web

Relational

HDInsight

Data Lake Analytics

Machine Learning

Spark

R



Azure Storage Blob


HDInsight








(New)

(New)



INSERT

OUTPUT

OUTPUT

SELECThellip

FROMhellip

WHEREhellip

+

PythonR

EXTRACT

EXTRACT

SELECT

SELECT

Azure

Data

Lake

Azure

Data

Lake

Azure

SQL

DB

Azure

Storage

Blobs

Azure

Storage

Blobs

RowSet RowSet

Cognitive Services


rs =

EXTRACT

UserID string

Start DateTime

End DateTime

Region string

SitesVisited string

PagesVisited string













C の関数


に近い）



58

進捗

統計情報


Business apps

Custom apps

Sensors and devices

ADL StorePeople


Azure SQL DW

Azure AD

Power BI

ADF

ADL Analytics


Business apps

Custom apps

Sensors and devices

HDInsight


データの連携

Power BI

File System

Database

Database

Hadoop

DWH

Data Mart










support


















service







computeHadoop





usserviceshdinsight


Store







Store




SQL Server Stretch



Insights

Azure SQL Database

SQL Server 2016


Azure SQL DW

ADLS amp ADLA

DocumentDB

HDInsight

Hadoop



Key


On

-pre

mis

es

Clo

ud






Azure

Data Lake Store

Azure



streaming







just supported

Generally Available





Want Hadoop






premises


Azure HDInsight






(StormSpark)





StudioU-SQL (C+SQL)






If NET or VS shop




Audit

ADLS Apache

Ranger

ADLS AAD Security

Groups (data)

Encryption TD

Audit


Scala Java Python

R

U-SQL T-SQL


External File

Types

ORC TXT

Parquet RCFile

ORC CSV Parquet

+ others

Parquet JSON

Hive + others


RCFile



Schema

Definition

Schema on Write

Polybase


Polybase


= 1PB)






learning-dt


Azure Storage Blob


HDInsight








(New)

(New)



INSERT

OUTPUT

OUTPUT

SELECThellip

FROMhellip

WHEREhellip

+

PythonR

EXTRACT

EXTRACT

SELECT

SELECT

Azure

Data

Lake

Azure

Data

Lake

Azure

SQL

DB

Azure

Storage

Blobs

Azure

Storage

Blobs

RowSet RowSet

Cognitive Services


rs =

EXTRACT

UserID string

Start DateTime

End DateTime

Region string

SitesVisited string

PagesVisited string













C の関数


に近い）



58

進捗

統計情報


Business apps

Custom apps

Sensors and devices

ADL StorePeople


Azure SQL DW

Azure AD

Power BI

ADF

ADL Analytics


Business apps

Custom apps

Sensors and devices

HDInsight


データの連携

Power BI

File System

Database

Database

Hadoop

DWH

Data Mart










support


















service







computeHadoop





usserviceshdinsight


Store







Store




SQL Server Stretch



Insights

Azure SQL Database

SQL Server 2016


Azure SQL DW

ADLS amp ADLA

DocumentDB

HDInsight

Hadoop



Key


On

-pre

mis

es

Clo

ud






Azure

Data Lake Store

Azure



streaming







just supported

Generally Available





Want Hadoop






premises


Azure HDInsight






(StormSpark)





StudioU-SQL (C+SQL)






If NET or VS shop




Audit

ADLS Apache

Ranger

ADLS AAD Security

Groups (data)

Encryption TD

Audit


Scala Java Python

R

U-SQL T-SQL


External File

Types

ORC TXT

Parquet RCFile

ORC CSV Parquet

+ others

Parquet JSON

Hive + others


RCFile



Schema

Definition

Schema on Write

Polybase


Polybase


= 1PB)






learning-dt

HDInsight








(New)

(New)



INSERT

OUTPUT

OUTPUT

SELECThellip

FROMhellip

WHEREhellip

+

PythonR

EXTRACT

EXTRACT

SELECT

SELECT

Azure

Data

Lake

Azure

Data

Lake

Azure

SQL

DB

Azure

Storage

Blobs

Azure

Storage

Blobs

RowSet RowSet

Cognitive Services


rs =

EXTRACT

UserID string

Start DateTime

End DateTime

Region string

SitesVisited string

PagesVisited string













C の関数


に近い）



58

進捗

統計情報


Business apps

Custom apps

Sensors and devices

ADL StorePeople


Azure SQL DW

Azure AD

Power BI

ADF

ADL Analytics


Business apps

Custom apps

Sensors and devices

HDInsight


データの連携

Power BI

File System

Database

Database

Hadoop

DWH

Data Mart










support


















service







computeHadoop





usserviceshdinsight


Store







Store




SQL Server Stretch



Insights

Azure SQL Database

SQL Server 2016


Azure SQL DW

ADLS amp ADLA

DocumentDB

HDInsight

Hadoop



Key


On

-pre

mis

es

Clo

ud






Azure

Data Lake Store

Azure



streaming







just supported

Generally Available





Want Hadoop






premises


Azure HDInsight






(StormSpark)





StudioU-SQL (C+SQL)






If NET or VS shop




Audit

ADLS Apache

Ranger

ADLS AAD Security

Groups (data)

Encryption TD

Audit


Scala Java Python

R

U-SQL T-SQL


External File

Types

ORC TXT

Parquet RCFile

ORC CSV Parquet

+ others

Parquet JSON

Hive + others


RCFile



Schema

Definition

Schema on Write

Polybase


Polybase


= 1PB)






learning-dt

(New)

(New)



INSERT

OUTPUT

OUTPUT

SELECThellip

FROMhellip

WHEREhellip

+

PythonR

EXTRACT

EXTRACT

SELECT

SELECT

Azure

Data

Lake

Azure

Data

Lake

Azure

SQL

DB

Azure

Storage

Blobs

Azure

Storage

Blobs

RowSet RowSet

Cognitive Services


rs =

EXTRACT

UserID string

Start DateTime

End DateTime

Region string

SitesVisited string

PagesVisited string













C の関数


に近い）



58

進捗

統計情報


Business apps

Custom apps

Sensors and devices

ADL StorePeople


Azure SQL DW

Azure AD

Power BI

ADF

ADL Analytics


Business apps

Custom apps

Sensors and devices

HDInsight


データの連携

Power BI

File System

Database

Database

Hadoop

DWH

Data Mart










support


















service







computeHadoop





usserviceshdinsight


Store







Store




SQL Server Stretch



Insights

Azure SQL Database

SQL Server 2016


Azure SQL DW

ADLS amp ADLA

DocumentDB

HDInsight

Hadoop



Key


On

-pre

mis

es

Clo

ud






Azure

Data Lake Store

Azure



streaming







just supported

Generally Available





Want Hadoop






premises


Azure HDInsight






(StormSpark)





StudioU-SQL (C+SQL)






If NET or VS shop




Audit

ADLS Apache

Ranger

ADLS AAD Security

Groups (data)

Encryption TD

Audit


Scala Java Python

R

U-SQL T-SQL


External File

Types

ORC TXT

Parquet RCFile

ORC CSV Parquet

+ others

Parquet JSON

Hive + others


RCFile



Schema

Definition

Schema on Write

Polybase


Polybase


= 1PB)






learning-dt



INSERT

OUTPUT

OUTPUT

SELECThellip

FROMhellip

WHEREhellip

+

PythonR

EXTRACT

EXTRACT

SELECT

SELECT

Azure

Data

Lake

Azure

Data

Lake

Azure

SQL

DB

Azure

Storage

Blobs

Azure

Storage

Blobs

RowSet RowSet

Cognitive Services


rs =

EXTRACT

UserID string

Start DateTime

End DateTime

Region string

SitesVisited string

PagesVisited string













C の関数


に近い）



58

進捗

統計情報


Business apps

Custom apps

Sensors and devices

ADL StorePeople


Azure SQL DW

Azure AD

Power BI

ADF

ADL Analytics


Business apps

Custom apps

Sensors and devices

HDInsight


データの連携

Power BI

File System

Database

Database

Hadoop

DWH

Data Mart










support


















service







computeHadoop





usserviceshdinsight


Store







Store




SQL Server Stretch



Insights

Azure SQL Database

SQL Server 2016


Azure SQL DW

ADLS amp ADLA

DocumentDB

HDInsight

Hadoop



Key


On

-pre

mis

es

Clo

ud






Azure

Data Lake Store

Azure



streaming







just supported

Generally Available





Want Hadoop






premises


Azure HDInsight






(StormSpark)





StudioU-SQL (C+SQL)






If NET or VS shop




Audit

ADLS Apache

Ranger

ADLS AAD Security

Groups (data)

Encryption TD

Audit


Scala Java Python

R

U-SQL T-SQL


External File

Types

ORC TXT

Parquet RCFile

ORC CSV Parquet

+ others

Parquet JSON

Hive + others


RCFile



Schema

Definition

Schema on Write

Polybase


Polybase


= 1PB)






learning-dt


rs =

EXTRACT

UserID string

Start DateTime

End DateTime

Region string

SitesVisited string

PagesVisited string













C の関数


に近い）



58

進捗

統計情報


Business apps

Custom apps

Sensors and devices

ADL StorePeople


Azure SQL DW

Azure AD

Power BI

ADF

ADL Analytics


Business apps

Custom apps

Sensors and devices

HDInsight


データの連携

Power BI

File System

Database

Database

Hadoop

DWH

Data Mart










support


















service







computeHadoop





usserviceshdinsight


Store







Store




SQL Server Stretch



Insights

Azure SQL Database

SQL Server 2016


Azure SQL DW

ADLS amp ADLA

DocumentDB

HDInsight

Hadoop



Key


On

-pre

mis

es

Clo

ud






Azure

Data Lake Store

Azure



streaming







just supported

Generally Available





Want Hadoop






premises


Azure HDInsight






(StormSpark)





StudioU-SQL (C+SQL)






If NET or VS shop




Audit

ADLS Apache

Ranger

ADLS AAD Security

Groups (data)

Encryption TD

Audit


Scala Java Python

R

U-SQL T-SQL


External File

Types

ORC TXT

Parquet RCFile

ORC CSV Parquet

+ others

Parquet JSON

Hive + others


RCFile



Schema

Definition

Schema on Write

Polybase


Polybase


= 1PB)






learning-dt


58

進捗

統計情報


Business apps

Custom apps

Sensors and devices

ADL StorePeople


Azure SQL DW

Azure AD

Power BI

ADF

ADL Analytics


Business apps

Custom apps

Sensors and devices

HDInsight


データの連携

Power BI

File System

Database

Database

Hadoop

DWH

Data Mart










support


















service







computeHadoop





usserviceshdinsight


Store







Store




SQL Server Stretch



Insights

Azure SQL Database

SQL Server 2016


Azure SQL DW

ADLS amp ADLA

DocumentDB

HDInsight

Hadoop



Key


On

-pre

mis

es

Clo

ud






Azure

Data Lake Store

Azure



streaming







just supported

Generally Available





Want Hadoop






premises


Azure HDInsight






(StormSpark)





StudioU-SQL (C+SQL)






If NET or VS shop




Audit

ADLS Apache

Ranger

ADLS AAD Security

Groups (data)

Encryption TD

Audit


Scala Java Python

R

U-SQL T-SQL


External File

Types

ORC TXT

Parquet RCFile

ORC CSV Parquet

+ others

Parquet JSON

Hive + others


RCFile



Schema

Definition

Schema on Write

Polybase


Polybase


= 1PB)






learning-dt

進捗

統計情報


Business apps

Custom apps

Sensors and devices

ADL StorePeople


Azure SQL DW

Azure AD

Power BI

ADF

ADL Analytics


Business apps

Custom apps

Sensors and devices

HDInsight


データの連携

Power BI

File System

Database

Database

Hadoop

DWH

Data Mart










support


















service







computeHadoop





usserviceshdinsight


Store







Store




SQL Server Stretch



Insights

Azure SQL Database

SQL Server 2016


Azure SQL DW

ADLS amp ADLA

DocumentDB

HDInsight

Hadoop



Key


On

-pre

mis

es

Clo

ud






Azure

Data Lake Store

Azure



streaming







just supported

Generally Available





Want Hadoop






premises


Azure HDInsight






(StormSpark)





StudioU-SQL (C+SQL)






If NET or VS shop




Audit

ADLS Apache

Ranger

ADLS AAD Security

Groups (data)

Encryption TD

Audit


Scala Java Python

R

U-SQL T-SQL


External File

Types

ORC TXT

Parquet RCFile

ORC CSV Parquet

+ others

Parquet JSON

Hive + others


RCFile



Schema

Definition

Schema on Write

Polybase


Polybase


= 1PB)






learning-dt


Business apps

Custom apps

Sensors and devices

ADL StorePeople


Azure SQL DW

Azure AD

Power BI

ADF

ADL Analytics


Business apps

Custom apps

Sensors and devices

HDInsight


データの連携

Power BI

File System

Database

Database

Hadoop

DWH

Data Mart










support


















service







computeHadoop





usserviceshdinsight


Store







Store




SQL Server Stretch



Insights

Azure SQL Database

SQL Server 2016


Azure SQL DW

ADLS amp ADLA

DocumentDB

HDInsight

Hadoop



Key


On

-pre

mis

es

Clo

ud






Azure

Data Lake Store

Azure



streaming







just supported

Generally Available





Want Hadoop






premises


Azure HDInsight






(StormSpark)





StudioU-SQL (C+SQL)






If NET or VS shop




Audit

ADLS Apache

Ranger

ADLS AAD Security

Groups (data)

Encryption TD

Audit


Scala Java Python

R

U-SQL T-SQL


External File

Types

ORC TXT

Parquet RCFile

ORC CSV Parquet

+ others

Parquet JSON

Hive + others


RCFile



Schema

Definition

Schema on Write

Polybase


Polybase


= 1PB)






learning-dt










support


















service







computeHadoop





usserviceshdinsight


Store







Store




SQL Server Stretch



Insights

Azure SQL Database

SQL Server 2016


Azure SQL DW

ADLS amp ADLA

DocumentDB

HDInsight

Hadoop



Key


On

-pre

mis

es

Clo

ud






Azure

Data Lake Store

Azure



streaming







just supported

Generally Available





Want Hadoop






premises


Azure HDInsight






(StormSpark)





StudioU-SQL (C+SQL)






If NET or VS shop




Audit

ADLS Apache

Ranger

ADLS AAD Security

Groups (data)

Encryption TD

Audit


Scala Java Python

R

U-SQL T-SQL


External File

Types

ORC TXT

Parquet RCFile

ORC CSV Parquet

+ others

Parquet JSON

Hive + others


RCFile



Schema

Definition

Schema on Write

Polybase


Polybase


= 1PB)






learning-dt









support


















service







computeHadoop





usserviceshdinsight


Store







Store




SQL Server Stretch



Insights

Azure SQL Database

SQL Server 2016


Azure SQL DW

ADLS amp ADLA

DocumentDB

HDInsight

Hadoop



Key


On

-pre

mis

es

Clo

ud






Azure

Data Lake Store

Azure



streaming







just supported

Generally Available





Want Hadoop






premises


Azure HDInsight






(StormSpark)





StudioU-SQL (C+SQL)






If NET or VS shop




Audit

ADLS Apache

Ranger

ADLS AAD Security

Groups (data)

Encryption TD

Audit


Scala Java Python

R

U-SQL T-SQL


External File

Types

ORC TXT

Parquet RCFile

ORC CSV Parquet

+ others

Parquet JSON

Hive + others


RCFile



Schema

Definition

Schema on Write

Polybase


Polybase


= 1PB)






learning-dt


SQL Server Stretch



Insights

Azure SQL Database

SQL Server 2016


Azure SQL DW

ADLS amp ADLA

DocumentDB

HDInsight

Hadoop



Key


On

-pre

mis

es

Clo

ud






Azure

Data Lake Store

Azure



streaming







just supported

Generally Available





Want Hadoop






premises


Azure HDInsight






(StormSpark)





StudioU-SQL (C+SQL)






If NET or VS shop




Audit

ADLS Apache

Ranger

ADLS AAD Security

Groups (data)

Encryption TD

Audit


Scala Java Python

R

U-SQL T-SQL


External File

Types

ORC TXT

Parquet RCFile

ORC CSV Parquet

+ others

Parquet JSON

Hive + others


RCFile



Schema

Definition

Schema on Write

Polybase


Polybase


= 1PB)






learning-dt

Azure

Data Lake Store

Azure



streaming







just supported

Generally Available





Want Hadoop






premises


Azure HDInsight






(StormSpark)





StudioU-SQL (C+SQL)






If NET or VS shop




Audit

ADLS Apache

Ranger

ADLS AAD Security

Groups (data)

Encryption TD

Audit


Scala Java Python

R

U-SQL T-SQL


External File

Types

ORC TXT

Parquet RCFile

ORC CSV Parquet

+ others

Parquet JSON

Hive + others


RCFile



Schema

Definition

Schema on Write

Polybase


Polybase


= 1PB)






learning-dt

Want Hadoop






premises


Azure HDInsight






(StormSpark)





StudioU-SQL (C+SQL)






If NET or VS shop




Audit

ADLS Apache

Ranger

ADLS AAD Security

Groups (data)

Encryption TD

Audit


Scala Java Python

R

U-SQL T-SQL


External File

Types

ORC TXT

Parquet RCFile

ORC CSV Parquet

+ others

Parquet JSON

Hive + others


RCFile



Schema

Definition

Schema on Write

Polybase


Polybase


= 1PB)






learning-dt




Audit

ADLS Apache

Ranger

ADLS AAD Security

Groups (data)

Encryption TD

Audit


Scala Java Python

R

U-SQL T-SQL


External File

Types

ORC TXT

Parquet RCFile

ORC CSV Parquet

+ others

Parquet JSON

Hive + others


RCFile



Schema

Definition

Schema on Write

Polybase


Polybase


= 1PB)






learning-dt





learning-dt



learning-dt

Engineering

Azure上の データベース 機能の選び方。KVSからDWHまで

Azure上のデータベース機能の選び方。KVSからDWHまで