Presented by:Manoj kumar s

Pradeep naik

Cloud computing is Internet-based computing, whereby shared resources, software, and information are provided to computers and other devices on demand, like the electricity grid. 

Cloud computing describes both a platform and a type of application. A cloud computing platform dynamically provisions, configures, reconfigures, and deprovisions servers as needed.

The main objective of the project is to improve the downloading of large amount

of data stored in Amazon S3 to multiple EC2 nodes, by presenting two most

efficient algorithms. The two algorithm will increase the through put by following

ways:

They can construct an overlay network on clouds without network topology

information.

They can optimize the total throughput between compute nodes dynamically.

They can increase the download throughput from cloud storage by letting

nodes cooperate with each other, resembling work stealing techniques.

Existing System

Proposed System

Traditionally, HPC applications have been run on distributed parallel computers

such as super computers and large cluster computers.

Especially data intensive HPC applications, such as BLAST, require very large

computational and storage resources For example, the ATLAS experiment.

While clouds are currently mainly used for personal and commercial use (e.g. for

web applications), their large number of storage and computational resources, high

accessibility, reliability and simple cost model make them very attractive for HPC

applications as well.

One or more spanning trees constructed based on the network topology and

network monitoring data in order to maximize available bandwidth and avoid

bottleneck links.

However, delivering optimal performance becomes difficult once the available

bandwidth changes dynamically.

In this paper, we focus on Amazon EC2/S3 (the most commonly used cloud

platform today) and propose two high performance multicast algorithms. These

algorithms make it possible to efficiently transfer large amounts of data stored in

Amazon S3 to multiple Amazon EC2 nodes.

Processor : Pentium-III

Speed : 1.1GHz

RAM : 512MB

Hard Disk : 40GB

General : Keyboard, Monitor , Mouse

Operating System : Windows XP

Software : jdk 1.5, Servlets

Back End : Oracle / Mysql

Server : Tomcat server / Weblogic server

Software specification :

Hardware specification :

Algorithms:

Steal algorithm

Non-Steal algorithm

Modules Used:

The project is divided into 2 major modules

Server Module

Client Module

Client Clouds Server

Register & Login

Get Access key & secret Key

Upload Files

Store user credentials and files

Download Files

Context Level DFD

Data Flow Diagram

Secret keyAccess Key

Login

Services

Create Bucket Refresh

bucket List Delete Bucket File upload

File list

Files in Bucket

Network Credentials

Service selection

SelectSelect Select

Select

Select

Show

Level One DFDLevel One DFD

Secret keyAccess Key

Login Services

Create Bucket

Stored Files

Server

Clients

Network Credentials

select

Accept Files From User

Retrieve files from Bucket

Download Upload

Level Two DFDLevel Two DFD

Secret keyAccess Key

Login

Network Credentials

Bucket

Server

File Sharing

Client1 Client2 Client3

Accept files from Client

Stored files

Retrieve files from Bucket

List

List ListDownload

Download

Download

Show Show

Non-Steal AlgorithmNon-Steal Algorithm

Secret keyAccess Key

Login

Network Credentials

Bucket

Server

File Sharing

Client1 Client2 Client3

Accept files from Client

Stored files

Retrieve files from Bucket

List

List

ListDownload

Download

Download

Show Show

Request/Response Request/Response

Request/Response

Steal AlgorithmSteal Algorithm

Register/login

Get access key and

secret key

Upload files

Download files

Client Server

Client Side

Use Diagram for Client SideUse Diagram for Client Side

Server

Server side

Store usercredentials

Generate access and secret key

Send keys to user

Accept files from user

Store filesin the bucket

Retrieve files from Bucket and send to users

Server Side use case

Server Amazon S3

Get_connection()

Connection established

Create bucket()

Bucket created

Upload files

File uploaded

Sequence diagram: Sequence diagram: Services provided by Amazon S3Services provided by Amazon S3

client server Amazon S3

get Connection

Request File ListGet file list from bucket

Send file List

Download file List

Download file from S3

File downloaded

C Client 1 Client 2 Client 3

Request to Download Files

Request to Download Files

Downloading files

Send Files

Send Files

Work Stealing mechanism between Clients using Work Stealing mechanism between Clients using Steal algorithmSteal algorithm

Home

Register/Login

Generating access key and secret key

Send keys to user

Store files into bucket

Retrieve files from bucket

Divide and send files to user

Main_console

ServerServer_socketAwt_components

Init_components()Init_iostream()Getawsconnections()Start_Server()Action_performed()

ServerServer_socketAwt_components

Init_components()Init_iostream()Start_Server()Action_performed()

Client

Start

Fetch files from the Bucket

Divide the files into n equal parts

Send filenames to nodes

Send file contents to nodes

Stop

We have presented two high performance multicast algorithms. These algorithms

make it possible to transfer large amounts of data stored in S3 to multiple EC2

nodes efficiently.

Future Enhancement :

Merge the two phases into one phase so as to overlap the S3 to EC2 and intra-

EC2 transfers to allow early dissemination of already fetched data.

Use multiple connections from each EC2 node to S3 for satisfying each node’s

total available download throughput. The resulting multicast performance should be

improved even further.

 [1] Dynamic Load-Balanced Multicast for Data-Intensive Applications on Clouds Tatsuhiro Chiba, Mathijs den Burger, Thilo Kielmann, and Satoshi Matsuoka. 2010 10th IEEE/ACM International Conference on Cluster, Cloud and Grid Computing

[2] T. Chiba, T. Endo, and S. Matsuoka, “High-performance mpi broadcast algorithm for grid environments utilizing multi-lane NICs,” in 7th IEEE International Symposium on Cluster Computing and the Grid (CCGrid), 2007, pp. 487–494.

[3] Amazon Web Services, http://aws.amazon.com/.

Bibliography