Configuration of Private Cloud Using Open Source Technology

5/14/2018 Configuration of Private Cloud Using Open Source Technology - slidepdf.com

http://slidepdf.com/reader/full/configuration-of-private-cloud-using-open-source-technology 1/11

CONFIGURATION OF PRIVATE CLOUD USING OPEN

SOURCE TECHNOLOGY

What is Private Cloud?

A System in which Infrastructure is dedicatedly allocated to an individual or an enterpriseand provides cloud based services: Dynamic Provision, Elastic Resources, Chargeback andAutomation. A Private cloud provides shared hosting environment for multiple applicationsand environments owned or managed by an Enterprise or individual. It‘s a preferred platformfor most enterprises who fear of security, compliance regulations and ownership of dataproblems with Public cloud and yet need the elastic power of computing to be available on-demand.

What is open source technology?

Open source software and standards are not just beneficial, but highly essential for aheterogeneous, shared and scalable environment such as the ―cloud”. open source software(which indicates the source code software is open to all) or free software (which means it isfree for users to run, copy, distribute, study, change and improve) which have specificdefinitions, cloud computing is a more loosely defined term that can refer either toinfrastructure, services or applications.

“Open source has built the cloud”

Sharing, the freedom to mix and match, choice, and many other characteristics of cloudcomputing demonstrate a likeness to the principles of open source software. More than justthe similarities, there is also a sheer necessity for open source software and open standards inclouds that comprise heterogeneous, and often proprietary, infrastructure. The availability of source code, the freedom to modify and redistribute, the flexibility and constant evolution,and other open philosophies greatly favour the cloud schema of things.

Why built private cloud using open source technology?

1. Open source lies at the foundation of many of the earliest cloud implementations . 2. When we think of the services we consume on the cloud, from Facebook and Google

to Amazon, none would have been affordable or scalable using a traditional licensingmodel.

3. The fact that most of the public clouds, including Amazon‘s Elastic Compute Cloud(EC2), run on Linux-based platforms is just one facet of the story. In fact, that is nowtaken so much for granted that the focus has shifted to tools and platforms that enablethe building of private or hybrid clouds, the integration of legacy infrastructure withthe cloud.

4. A rather large number of such open source tools are now available, right fromplatforms and development tools, to management dashboards and automatedmigration tools for applications. Existing open source platforms are also fast adaptingto the needs of cloud computing and include features such as intelligent workload

management and cloud-enabled scalability considerations, to help massive horizontalscalability at all the layers of the technology stack.



Softwares and tools available to implement our project..

Large choice of open source applications for the cloud includes Red Hat, TrafficServer, Puppet, Zoho, Cloudera, Enomaly and Joyent. Among these Ubuntu, Red Hat,and almost every other open source platform now has a stable cloud offering.

Eucalyptus is a modular platform that is capable of working with a variety of interfaces, including Amazon‘s EC2 and Simple Storage Service (S3) services.Eucalyptus works with various distros, including Red Hat Enterprise Linux (RHEL),CentOS, SUSE Linux Enterprise Server (SLES), OpenSUSE, Debian and Fedora. Itcan also host MS Windows images. It is capable of working with many virtualisationtechnologies such as VMware, Xen and KVM hypervisors, in order to implement theabstraction demanded by a cloud environment.

Though, Ubuntu prefers KVM for the visualization support and has added to the

default packages. But limitation of KVM to run only virtualization extensions (Intel

VT or AMD-V) processors. So XEN is most preferable for virtualization.

XEN is the most popular Open Source Virtualization software that allows multipleOS to run on the same computer hardware concurrently, thereby improving the

effective usage and efficiency of the underlying hardware. It benefits the enterprises

with the power of consolidation, increased utilization and rapid provisioning.

Setting up Eucalyptus Cloud on Xen :-

In any Eucalyptus Cloud Installation, there are 2 top-level components: Cloud

Controller (CLC) and Walrus. These 2 components manage the various clusters, where

cluster is a set of physical machines that host the Virtual Instances. In each cluster, there are 2

components that interact with the high level components: Cluster Controller (CC) and

Storage Controller (SC). CC and SC are cluster – level components. Each cluster is

composed of various Nodes, or physical machines. Each Node will run a Node Controller

(NC) that will control the hypervisor for managing the Virtual Instances. For this setup, we

have implemented a Single-Cluster Installation, where all the components except NC are co-

located on one machine. As per Eucalyptus documentation, this co-located system is called:

front-end. So in a gist, we have 1 physical machine which hosts CLC, Walrus, CC, SC, and 5

other machines that hosts NC each. The Node Controller uses Xen as a hypervisor. The NC

service runs on Domain-0 kernel in the Xen Setup.



Hardwares and softwares required..

Hardware:-1 Admin Machine with the config : Intel Core 2 Duo Processor 1.8 GHz, with 1GB RAM,160

GB HDD, and 5 Node Controllers each with the config : Intel Core 2 Duo Processor 1.8 GHz

(VT enabled) with 2 GB RAM, 160 GB HDD.

Software:-

Eucalyptus 1.6.1 running on Ubuntu 9.04 (Jaunty), Xen 3.3 hypervisor.

Eucalyptus has 3 major packages.

Cluster Controller: provides support for the virtual network overlay

Cloud Controller: includes the front-end services and the Walrus storage system.

Node Controller: that interacts with XEN to manage individual VMs.The basic architecture how client, cloud controller, cluster controller and node controllers

communicate with each other in a eucalyptus setup.

— > eucalyptus-nc

|

|

Client — > eucalyptus-cloud — > eucalyptus-cc —— > eucalyptus-nc

|

|

— > eucalyptus-nc Packages provided by Ubuntu are: eucalyptus-cloud, eucalyptus-cc and eucalyptus-nc.



Implementation method..

Front end setup:

Front end runs both eucalyptus-cloud and eucalyptus-cc. So I install both packages on this machine.

Follow the steps for installing the front end:

Step 1: Prepare a raw Ubuntu OS system.

Install Ubuntu 9.04 on a machine. Select SSH server to be installed on the machine by default, so that we

can connect to machine remotely.

Step 2: Install postfix.

Postfix is a mail server. Eucalyptus send mails to users that are added by admin on

eucalyptus web interface. So we install postfix that eucalyptus will use as mail server. Steps to

install postfix and to setup configurations and testing successful Postfix installation

command# apt-get install postfix

Set configuration

command# sudo dpkg-reconfigure postfix

Set following parameters with given values.

General type of mail configuration: Internet Site

NONE doesn‟t appear to be requested in current config

System mail name: server1.example.com

Root and postmaster mail recipient: <admin_user_name> (Optional)

Other destinations for mail: server1.example.com, example.com,

localhost.example.com, localhost

Force synchronous updates on mail queue?: No

Local networks: 127.0.0.0/8

Yes doesn‟t appear to be requested in current config

Mialbox size limit (bytes): 0 Local address extension character: +

Internet protocols to use: all

Test postfix installation

command# EHLO localhost

mail from: root@localhost

rcpt to: [email protected]

data

subject: your sub (press enter twice)

Type your message and finish with “.” then Enter

Finish with “Quit”



Check your mail (if received your mail is working)

Step 3: Install Cloud and Cluster controller

Install the packages provided by Ubuntu, all the dependencies will be installed automatically by

package manager.

command# apt-get install eucalyptus-cloud eucalyptus-cc

Step 4: Install DHCP server

Install the DHCP server which is used by eucalyptus for assigning the IPs to instances. DHCP

server should not be in running state and set the DHCP server not to run when system is booted

up again.

command# apt-get install dhcp3-servercommand# /etc/init.d/dhcdbd stop

command# update-rc.d -f dhcdbd remove

Set the DHCP server path in the eucalyptus config file „ /etc/eucalyptus/eucalyptus.conf „

VNET_DHCPDAEMON=”/usr/sbin/dhcpd3″

Step 5: Eucalyptus configuration

Set the static mode for eucalyptus so that eucalyptus uses IP‟s from the pool of IP‟s provided in

the config file and use the machine addresses provided along with IPs.

Configuration file for eucalyptus is „ /etc/eucalyptus/eucalyptus.conf „

open /etc/eucalyptus/eucalyptus.conf and set the following parameters

VNET_INTERFACE=”peth0″

VNET_BRIDGE=”eth0″

Locate „kvm„ and replace to „xen„

locateVNET_MODE=”SYSTEM” and disable it

locateVNET_MODE=”STATIC” and enable it

Add machine and IPs for instances to be used and change the netowork setting to current

network settings VNET_MODE=”STATIC”

VNET_SUBNET=”192.168.145.0″

VNET_NETMASK=”255.255.255.0″

VNET_BROADCAST=”192.168.145.255″

VNET_ROUTER=”192.168.145.1″

VNET_DNS=”192.168.150.42″

VNET_MACMAP=”AA:DD:11:CE:FF:ED=192.168.145.73

AA:DD:11:CE:FF:EE=192.168.145.75

AA:DD:11:CE:FF:EF=192.168.145.77″



Step 6: Other configurations

Edit : /usr/share/eucalyptus/gen_libvirt_xml

Locate all „sda„ to „xvda„

Edit : /etc/xen/xend-config.sxp

(xend-http-server yes)

gateway = 192.168.145.1

netmask = 255.255.255.0

broadcast = 192.168.145.255

Step 7 Reboot the system to take effect

Step 8 Cloud configuarions

Once eucalyptus front end has been installed, we can use the eucalyptus web interface and setupcluster and set other parameters.

Eucalyptus cloud web URL: https://<front end ip>:8443/

Under configuration tab add cluster name

Set host to „localhost‘

Create kernel and ramdisk images and set in eucalyptus site. (Procedure to create images will

be described later in the page)

Restart the eucalyptus daemon to take effects

Back end setup

On the systems where we want to run instances, we will install node controller and XEN

3.0. Since I used the XEN as virtualization package, I install the node controller

package eucalyptus-ncmanually. I installed required dependencies using the package

manager apt-get, but the eucalyptus-nc was installed by donwloading and manually installing

bypassing dependencies list.

Step 1: Hardware compatibility

eucalyptus-nc requires ‘ libvirt-bin‘ package. Libvirt is a C toolkit to interact with thevirtualization capabilities of recent versions of Linux (and other OSes). More info can be

found at: http://libvirt.org/

To install ‘ libvirt-bin‘ package, certain flags should be supported by CPU.

command# egrep ‘(vmx|svm)’ /proc/cpuinfo

[if no print], libvirt-bin cant be installed

Stop the apparmor for Ubuntu System

command# /etc/init.d/apparmor stop

command# update-rc.d -f apparmor remove



Step 2: Install XEN package

Use the XEN virtualization package instead of Ubuntu‘s default KVM, I install the

eucalyptus-nc dependencies manually.

command# apt-get install iproute iptables module-init-tools python2.5 python2.6

command# apt-get install xen-utils command# apt-get install ubuntu-xen-server

Step 3: Install libvirt-bin package

eucalyptus-nc requires ‘ libvirt-bin‘ package. Libvirt is a C toolkit to interact with the

virtualization capabilities of recent versions of Linux (and other OSes). More info can be

found at: http://libvirt.org/

command# apt-get install adduser bridge-utils dnsmasq-base iptables libsasl2-2 libxen3

logrotate

command# apt-get install netcat-openbsd libavahi-client3 libavahi-common3 libc6libdbus-1-3

command# apt-get install libgcrypt11 libgnutls26 libhal1 libpolkit-dbus2 libpolkit2

libreadline5 libselinux1

command# apt-get install libtasn1-3 libvirt0 libxml2 zlib1g policykit

Download the libvirt-bin package from URL: http://packages.ubuntu.com/jaunty/libvirt-

bin

I used i386 package, http://mirrors.kernel.org/ubuntu/pool/main/libv/libvirt/libvirt-

bin_0.6.1-0ubuntu5_i386.deb

Install lib-virt

command# dpkg -i libvirt-bin_0.6.1-0ubuntu5_i386.deb

command# sudo adduser $USER libvirtd

Step 4: Install eucalyptus node package

command# apt-get install apache2 eucalyptus-common eucalyptus-gl libapache2-mod-

axis2c

command# apt-get install libaxis2c0 librampart0 dhcp3-server vlan aoetools

command# apt-get install libc6 libcurl3-gnutls libvirt0 zlib1g

Download eucalyptus-nc package from Ubuntu package repository.

eucalyptus-nc package URL: http://packages.ubuntu.com/jaunty/eucalyptus-ncDownload the i386 package from URL:

http://mirrors.kernel.org/ubuntu/pool/universe/e/eucalyptus/eucalyptus-

nc_1.5~bzr266-0ubuntu2_i386.deb

command# dpkg -i eucalyptus<version>

Step 5: Grub loader for Xen Kernel

Setup grub menus and load xen images to boot into XEN kernel.

Download the modules and image from URLs:

wget http://ftp.debian.org/debian/pool/main/l/linux-2.6/linux-modules-2.6.26-2-xen-686_2.6.26-15_i386.deb

http://packages.ubuntu.com/jaunty/libvirt-bin


http://packages.ubuntu.com/jaunty/eucalyptus-nc

http://packages.ubuntu.com/jaunty/eucalyptus-nc





wget http://ftp.debian.org/debian/pool/main/l/linux-2.6/linux-image-2.6.26-2-xen-

686_2.6.26-15_i386.deb

command# sudo dpkg -i linux-modules-2.6.26-2-xen-686_2.6.26-15_i386.deb

command# sudo dpkg -i linux-image-2.6.26-2-xen-686_2.6.26-15_i386.deb

Step 6: Eucalyptus Configuerations

Stop the eucalyptus daemon and set the configuration depending upon your installation.

command# /etc/init.d/eucalytpus-nc stop

Edit : /etc/sysctl.conf

(uncomment net.ipv4.ip_forward=1)

Edit : /etc/eucalyptus/eucalyptus.conf

(set)

VNET_BRIDGE=‖eth0″

VNET_INTERFACE=‖peth0″

HYPERVISOR=‖xen‖ VNET_MODE=‖STATIC‖

Edit : /usr/share/eucalyptus/gen_libvirt_xml

locate ‗sda’ s and replace to ‗xvda‗ s

for static ips

Edit : /etc/xen-tools/xen-tools.conf

gateway 192.168.145.1

broadcast 192.168.145.255

netmask 255.255.255.0

(xend-http-server yes)(xend-unix-server yes)

command# /etc/init.d/eucalyptus-nc start

Cloud integration

Now, we have front controller and node controllers ready. We have to setup now thecommunication between cluster controller and node controllers. We will add the nodes list tothe cloud, so cloud know about the available node.

command# sudo euca_conf -addnode <node ip> <node ip> <node ip>

This will add nodes to cloud config file and replicates the permission and certificate files tothe nodes to be able to have password-less communication between nodes and cloud.To check the cloud installation, and node communication, you can run ‗euca-describe-

availability-zones verbose‗ command and check if all the nodes are listed in outp ut.



ec2 or euca2ools setup

Euca2ools setup

Install dependencies

command# apt-get install curl libopenssl-ruby ruby

Download the appropriate tarball from http://open.eucalyptus.com/downloads

Steps command# tar zxvf euca2ools-1.0-*.tar.gz command# cd euca2ools-1.0-* (Replace this with the actual directory name) command# sudo -s command# echo deb file://${PWD} ./ >> /etc/apt/sources.list command# apt-get update command# apt-get install euca2ools

WARNING: The following packages cannot be authenticated! …

Install these packages without verification [y/N]? y

ec2 tools Setup

Eucalyptus supports ec2-ami-tools-1.3-26357 and ec2-api-tools-1.3-30349 versions of

ec2 tools.Install dependencies

Install jdk1.6.0_12 java version.

StepsDownload ec2-ami-tools-1.3-26357 and ec2-api-tools-1.3-30349.unzip these archivesCopy the ‗lib‘ and ‗bin‘ folders under ~/.euca folder.

Export parameters export EC2_HOME=~/.euca

export JAVA_HOME=<directory>/jdk1.6.0_12export PATH=$PATH:<directory>/jdk1.6.0_12/binexport PATH=$PATH:~/.euca/bin

Set URL and credentials

Download ‗euca2-admin-x509.zip‗ file from cloud site at https://<front end ip>:8443/ .

Unzip this archive under ~/.euca folder and export the eucarc file.

command# source ~/.eucarc



Create guest machine

I create my image using vmbuilder.More info on vmbuilder can be found

on: https://help.ubuntu.com/8.04/serverguide/C/ubuntu-vm-builder.html

StepsSelect a directory where all files will be stored. I used ‗ /usr/akash/cloud‗ Create a ‗part‗ file under directory.The content of ‗part‗ is following:

root 400 /mnt/ephemeral 0 /dev/sda2swap 1 /dev/sda3

Create ‗firstboot‗ file, which is run first time, instance boots up. apt-get -y install openssh-server

Create the image. command# vmbuilder xen ubuntu – firstboot ./firstboot –

mirror=http://192.168.152.87:9999/ubuntu – addpkg ssh – rootsize=4096 – arch i386 –

verbose – debugIt creates the image file that i used for creating emi. You can found the image under

‗ubuntu-xen‗ folder.This image can be tested, if its compatible to xen. Use xm create command.

command# xm create /usr/akash/cloud/ubuntu-xen/xen.cfg Check the status of emi using xm list command.

command# xm list If you see your emi ID in the output, your image can be run with XEN.

Creating EMIs

Now, cloud is setup, we should create images that can be run using eucalyptus. We willcreate emis for kernel, ramdisk and instance.

kernel emi

Steps

command# mkdir kernel

command# ec2-bundle-image -i vmlinuz-2.6.28-13-server -d ./kernel – kernel true

command# ec2-upload-bundle -b eucaKernel -m ./kernel/vmlinuz-2.6.28-13-server.manifest.xml

command# EKI=èc2-register eucaKernel/vmlinuz-2.6.28-13-server.manifest.xml | awk ‗{print

$2}‘`

command# echo $EKI



ramdisk emi

Steps

command# mkdir ramdisk

command# ec2-bundle-image -i initrd.img-2.6.28-13-server -d ./ramdisk – ramdisk true command# ec2-upload-bundle -b eucaRamdisk -m ./ramdisk/initrd.img-2.6.28-13-

server.manifest.xml

command# ERI=èc2-register eucaRamdisk/initrd.img-2.6.28-13-server.manifest.xml | awk ‗{print

$2}‘`

command# echo $ERI

instance emi

Steps

command# mkdir image

command# ec2-bundle-image -i root.img -d ./image

command# ec2-upload-bundle -b plabImage -m ./image/root.img.manifest.xml

command# EMI=èc2-register plabImage/root.img.manifest.xml | awk ‗{print $2}‘`

command# echo $EMI

Run instance

Add key pair

command# ec2-add-keypair mykey > ~/.euca/mykey.priv

command# chmod 0600 ~/.euca/mykey.priv

command# ec2-run-instances emi-73FD0EAD -k mykey – kernel eki-8D701398 – ramdisk eri-

E47F14E8 – instance-type m1.xlarge

Documents

Configuration of Private Cloud Using Open Source Technology