Upload
cloudera-japan
View
1.894
Download
1
Embed Size (px)
Citation preview
1 Cloudera, Inc. All rights reserved.
HadoopCloudera Director 2.0 Cloudera
2 Cloudera, Inc. All rights reserved.
()20114ClouderaClouderaemail: [email protected] twitter: @shiumachi
5 Cloudera, Inc. All rights reserved.
Cloudera Enterprise Hadoop Fast / Easy / Secure
Cloudera: Fast : Easy : Secure :
OPERATIONS DATA MANAGEMENT
STRUCTURED UNSTRUCTURED
PROCESS, ANALYZE, SERVE
UNIFIED SERVICES
RESOURCE MANAGEMENT SECURITY
FILESYSTEM RELATIONAL NoSQL
STORE
INTEGRATE
BATCH STREAM SQL SEARCH SDK
Public Cloud Private Cloud Hybrid Environments
Hybrid Deployment Flexibility
6 Cloudera, Inc. All rights reserved.
Hadoop
Object Store
STORE
COMPUTE
7 Cloudera, Inc. All rights reserved.
Hadoop
Hadoop
8 Cloudera, Inc. All rights reserved.
Cloudera: Hadoop
CDHHadoop
2009 2012 2013 2014 2015
AWS
Cloudera Enterprise MSP
ClouderaAzure
ClouderaGCP
Hadoop
9 Cloudera, Inc. All rights reserved.
Hadoop
11 Cloudera, Inc. All rights reserved.
ETL/
BI/
15 Cloudera, Inc. All rights reserved.
Easy:
Launch Cluster
Submit Job
Record Results
12
3Auto-Termina
te 4
16 Cloudera, Inc. All rights reserved.
CUSTOMER 360
: http://blog.godatadriven.com/schiphol-implements-datasciencesuite.html
21 Cloudera, Inc. All rights reserved.
$120M(130)
22 Cloudera, Inc. All rights reserved.
: hZp://techspec[ve.net/2015/08/03/how-gopro-is-using-amazon-bmc-and-cloudera-to-kick-everyone-elses-buZ/
26 Cloudera, Inc. All rights reserved.
FINRA monitors 50B market events per day to build a holis[c picture of US market ac[vity and make real-[me decisions, while saving $10-20M annually
27 Cloudera, Inc. All rights reserved.
Airbnb improved their overall booking rate through machine learning algorithms and beZer search to more effec[vely match customers with the right rental property
CUSTOMER 360
34 Cloudera, Inc. All rights reserved.
Cloudera Director 2.0 & C5.5 Releases
API Hive on S3 Spark on S3
Impala on S3 (beta)
BI/ HA HA/Kerberos
DB UI
ALL WORKLOADS: AWS s3a GUI :
35 Cloudera, Inc. All rights reserved.
Power BI
Microso> Azure
Marketplace
Marketplace Delivers Full cloud deployment; no hardware dependency Start work in
36 Cloudera, Inc. All rights reserved.
Get Started AWS Reference Guide GCP Reference Guide Download Cloudera Director www.cloudera.com/downloads
Try It Out Cloudera Live (includes step-by-step tutorial) AWS Quickstart Azure Marketplace
Resources API Integra[on & Scrip[ng hZps://github.com/cloudera/director-sdk
hZps://github.com/cloudera/director-scripts
Addi[onal Cloud Integra[on hZps://github.com/cloudera/director-spi hZps://github.com/cloudera/director-google-plugin
37 Cloudera, Inc. All rights reserved.
Cloudera on AWS
38 Cloudera, Inc. All rights reserved.
Cloudera on AWS HDFSHDFS
S3CPU /
39 Cloudera, Inc. All rights reserved.
S3
HDFS
Hadoop
()
Hadoop
HDFSHadoop
HDFS
EBS IO
IO EC2/EBS
OS
40 Cloudera, Inc. All rights reserved.
MapReduce YARN Spark Hive Pig Crunch
c3.8xlarge d2.2xlarge i2.2xlarge i2.4xlarge i2.8xlarge r3.8xlarge m2.4xlarge
c3.8xlarge d2.8xlarge i2.2xlarge i2.4xlarge i2.8xlarge r3.8xlarge
HBase Solr Impala
c3.8xlarge d2.2xlarge i2.4xlarge i2.8xlarge r3.8xlarge
d2.8xlarge i2.4xlarge i2.8xlarge
CDH d2.2xlarge i2.2xlarge i2.4xlarge
d2.8xlarge
41 Cloudera, Inc. All rights reserved.
VPC 11ACL
Flume
IP
GWNAT
NAT Linux EC2 DC (VPC or Direct Connect)
43 Cloudera, Inc. All rights reserved.
S3S3distcpHDFSHadoop
HDFSHadoopS3
44 Cloudera, Inc. All rights reserved.
S3distcp / ACL HDFS
HBase
2AZHiveRDBMSAmazon RDS
distcpS3HDFS
45 Cloudera, Inc. All rights reserved.
Hadoop
Impala
SparkKafka Flume
HDFS HBase
46 Cloudera, Inc. All rights reserved.
S3()
(Impala)
47 Cloudera, Inc. All rights reserved.
Impala on S3 C5.5 JOIN: HDFS, HBase, S3 JOIN(SentryACL): :
DML
INSERT / LOAD DATA / CREATE TABLE AS SELECT
HDFS
48 Cloudera, Inc. All rights reserved.
Thank you