Upload
ahmed-shouman
View
114
Download
0
Embed Size (px)
Citation preview
شكر و تقدير
TAB ONE TAB TWO TAB THREE TAB FOUR TAB FIVE
FaaS(Food as a service )
ICT (Information and Communication Technology)
Cloud computing
Project objectives
•1 -
Project objectives2 .
Project objectives
3 .
Project objectives4 .
Project Achievements
TAB ONE TAB TWO TAB THREE TAB FOUR TAB FIVE
Introduction to Cloud team
What, which, how and why?
• In the simplest terms, cloud computing means storing and accessing data and programs over the Internet instead of your computer's hard drive.
• When it's smarter to rent than to buy…
• Cloud computing is an emerging computing technology that uses the internet and central remote servers to maintain data and application.
But where does The Cloud come in??
Hardware
Operating System
App App App
Traditional StackHardware
OS
App App App
Hypervisor
OS OS
Virtualized Stack
Key Technology: Virtualization
A complex system consisting of complex subsystems
Architecture Cloud Computing
Hardware
CentOS
KVM
OpenStack
Big Data Web
Open Shift
Host OS. distro
Why centos?• Familiar of deploy and configure “recently released RDO”.
• Community support.
• Performance benchmarks “relating to updates”.
• Number of bugs.
Why GO open?• Multi-Vendor Virtual.
• Brings the efficiency, agility, and cost benefits of cloud.
• prevents one vendor from controlling your access to the greatest innovation
• Large supporting community.
Architecture Cloud Computing
Hardware
CentOS
KVM
OpenStack
Big Data Web
Open Shift
Virtualization
Architecture Cloud Computing
Hardware
CentOS
KVM
OpenStack
Big Data Web
Open Shift
OpenStack Architecture
Horizon
Nova
Neutron
SwiftCinder
Keystone
Glance
Ceilometer
Heat
OpenStack Installation• Preparation for installation:• Update OS.• Remove Network manager.• Add repositories.
Installation
Manual installation
Scripted installation
Installing services one by one “Manual installation”
• Service “keystone” 1. Installing the service: # yum install openstack-keystone python- keystoneclient
2. Configure the service:# openstack-config--set/etc/keystone/keystone.conf\ sqlconnectionmysql://keystone:KEYSTONE_DBPASS@controller/keystone
3. Start and enable Service#serviceopenstack-keystonestart #chkconfigopenstack-keystone on
• Service “glance”• Service “nova” • Enable networking• Add the dashboard “horizon”
OpenStack scripted deployment “RDO”
An automated installation with minimal user intervention. Answer file is a simple text file contains configurations of OS.Quick start:1. Getting software repositories.# yum update –y # yum install -y http://rdo.fedorapeople.org/rdo-release.rpm 2. Getting Packstack installer.# yum install -y openstack-packstack
3. Run Packstack to install openstack# packstack--allinone
OpenStack administration | cloud images
VM Image: Virtual disk that has a bootable operating system installed on it.
OpenStack supported image formats: RAW QCOW2: commonly used with the KVM hypervisor.
• The image size is smaller.• Support for snapshots.
VM Image metadata: CPU architecture. The OS installed on the image (Linux, Windows,… ). OS_Distro (Centos, Fedora, Ubuntu,…).
OpenStack Linux image requirements to be sure that the image supports the OpenStack features:
• Resize root partition on boot.• No hard-coded MAC address information.• Ensure ssh server runs.
• Access instance by using ssh public key.• Use cloud-init to fetch the public key. • Inject ssh key into the instance in the first boot.
• Process user data (host name, ssh key). Disable firewall and use the host firewall.
We built two cloud images: Centos cloud image: LAMP web server cloud Image:
• Linux operating system (Centos).• Apache HTTP Server is the most popular web server.• MySQL database.• PHP server-side scripting language designed for web.
Tools to modify images: Guestfish: modify the files inside image. Guestmount: mount the image's file system directly.
OpenStack administration | Internal Network Architecture
manage project. View and manage quotas. manage user. Manage flavors.
OpenStack administration | Admin user
OpenStack administration | End user
• Overview about menus.• Creating network.• Creating instances.• Setting IP’s.• Ping instances.
Architecture Cloud Computing
Hardware
CentOS
KVM
OpenStack
Big Data Web
Open Shift
What is Platform as a Service (PaaS)?
PaaS helps organizations, specifically developers, realize the benefits of cloud computing by providing a fast and scalable way to host applications in the cloud.
Technically, a PaaS is an Application Platform comprised of an operating system, middleware and other software that allows applications to run on the cloud with much of the management.
Platform as a Service is changing the way developers approach developing software.
What is OpenShift Origin? • OpenShift Origin is Red Hat’s open source Platform as a Service (PaaS)
offering.• OpenShift Origin is an application platform where application
developers and teams can build, test, deploy, and run their applications.
Code
Deploy
Enjoy
Flavors of OpenShift
origin
Public Cloud
Service
On-premise or Private
Cloud Software
Open Source Project
Modernizing App Development with PaaS
OpenShift Machines
Key OpenShift Terms• Broker – Management host, orchestration of Nodes• Node – Compute host containing Gears• Gear – Allocation of fixed memory, compute, and storage resources
for running applications• Cartridge – A technology/framework (PHP, Perl, Java/JEE, Ruby,
Python, MySQL, etc.) to build applications.
Simple Application Creation
Architecture Cloud Computing
Hardware
CentOS
KVM
OpenStack
Big Data Web
Open Shift
TAB ONE TAB TWO TAB THREE TAB FOUR TAB FIVE
Big Data Architecture
Cloud Architecture
HDFS / MapReduce
Hive SqoopCloudera Platform
Eco Systems
Introduction to hadoop
What Makes it Big Data?
Introduction to hadoop
Big Data
How to store? How to work?
Hadoop environment
High level hadoop architecture
Introduction to HDFS
Introduction to MapReduce
The important thing in hadoop is configuration file
Practice work at lab
Big Data Architecture
Cloud Architecture
HDFS / MapReduce
Hive Sqoop Cloudera Manager and New Solutions
Eco Systems
What’s the problem?
2 TB(compresed) raw data per day today
What’s the solution ?HIVE
Data Warehousing at Facebook Today
Web ServersScribe Servers
Filers
Hive on Hadoop Cluster
Oracle RAC Federated MySQL
HIVE Components
HIVE x BIG
Big Data Architecture
Cloud Architecture
HDFS / MapReduce
Hive Sqoop Cloudera Manager and New Solutions
Eco Systems
Sqoop
•Why Sqoop•What’s Sqoop
•How Sqoop works
Import data
1. Run Import2. Pull metadata
3. Lunch MapReduce job
4. Pull data from database
5. Writ to data sink
sqoop import --connect jdbc:mysql://vm2.faas.com/sqoop --username root --password root --table yelid --target-dir /input/
Export data
1. Run Sqoop Export
2. Launch MapReduce Job
3. Write to staging table
4. sql> INSERT INTO target (SELECT*FROM staging)
sqoop export --connect jdbc:mysql://vm2.faas.com/sqoop --username root --password root --table location --export-dir location
Big Data Architecture
Cloud Architecture
HDFS / MapReduce
Hive Sqoop Cloudera Manager and New Solutions
Eco Systems
What’s Cloudera CDH?• CDH : Is 100% Open Source Distribution including the core elements of Hadoop plus
several open source projects.
What's Inside ?• Apache Yarn : (Yet Another Resource Negotiator) Is the data operating system of Hadoop that enablesyou to process data simultaneously in multiple ways.
What's Inside?• Apache HUE : Hue is a suite of applications that provide web-based access to CDH
components and a platform for building custom applications.
What’s Cloudera Manager C.M?• Cloudera Manager : is a unified management interface that makes it easy to install, configure, and manage a CDH cluster through a web interface “Admin
Console”
Other Solutions1-Hortonworks : 100% Open Source Enterprise Apache Hadoop.
TAB ONE TAB TWO TAB THREE TAB FOUR TAB FIVE
System Overview
Web Design and
Development
Big Data Solution
Cloud Computing
Web Design and Development
TAB ONE TAB TWO TAB THREE TAB FOUR TAB FIVE
Our Next Step.!• Cloud: Integrate openShift with openStack.
• Big Data: Online data processing (real time) using HBase.
• Web: Automate matching between farmer and food processor.
Thank You