Research on cloud computing application in the peer-to-peer based video-on-demand systems

2012.04.13

Speaker : 吳靖緯 MA0G0101

2011 3rd International Workshop on Intelligent Systems and Applications (ISA),

On page(s): 1- 4, May 2011

Authors:SUN Bing-jue, WU Kai-jun

Outline

• Introduction

• Related work

• System design

• Simulation and result analysis

• Conclusion 2

Introduction

• In the peer-to-peer based video-on-demand systems, when the number of users online reaches a certain scale.

• It will greatly influence the bandwidth and the speed of response server.

• According to the usage analysis of P2P/VOD video system, user demand video via the cache server which will generate a cache file for the next user call.

3

Introduction

• This raises the question that if the original video files have 1000.

• So it is possible to generate 1,000 cache files which remain in the cache server within the server's disk.

• When the buffer disk is full, it will clear the cache files automatically and move the new file to the disk, but in the process of deleting and writing files, the client appears the phenomenon of the video buffer, which is blocking in the play.

4

Introduction

• Traditional method is to use load-balancing with multiple cache servers to solve this problem.

• Because load-balancing server can’t distinguish between user needs, resulting in more than one cache servers stored the same cache files in the video.

• According to the thinking of "cloud computing", the paper designed to work a cross platform of cache server cluster based on operation stored of Hadoop synchronization which applied in P2P/VoD.

5

Related work

• Figure 1 is a flow chart of Hadoop for data processing and transmission of map, reduce data phase.

6

Related work

• Hadoop processing needs to be done by using 3 data transfer process: file input to the map, map transfer to reduce, reduce output to HDFS.

• When they run in the same time, system manage the data segmentation, job scheduling, load balancing, fault tolerance and network communications.

• MapReduce provides a simple interface that enables researchers to design parallel and distributed applications easily.

7

Related work

The data flow analysis on demand• Map read data from the server in the way to pull the data.

• The output data of Map side is first written to local disk, it notified JobTracker when the task is completed.

8

Related work

The integration of computing and storage

9

System design

• The whole system is divided into hardware and software.

• Hardware components include: Web system modules, file server system modules, DNS server, storage module and the composition of cache server cluster.

• The design of software components: the operating environment of server will be mainly based on Windows 2003 server operating system and FreeBSD operating systems.

10

System design

• Database store data by using a distributed way of SQLserver.

• Parts of the site use ASP.net compiler to achieve the management of background data and the view of front static page.

• Play system of video program uses the design of P2P technology.

11

System design

Video services implement the process in the system which was shown in the following.

• Internet users first access to the video website for inquiring the video information, then send the video request.

• Web server inquire the information of video address in the database after it obtain video request, and then send the requests of video data to the file server.

12

System design

• File server send video data for users in a variety of ways: Direct transmission, transmission through the cache server, users through other Internet video services transmitted by P2P.

• The eventual realization of the video data streams transmitted parallelly to the client, complete high-definition video playback.

13

Simulation and result analysis

• The experimental data from access logs of our school VOD system user.

• The average daily traffic of VOD system is up to 1000 requests / day, the system of the user logs records the various interactive operations in detail, a unique user ID correspond to one visit, but the user's interactive operation, one visit may contain multiple user’s requests.

14

Simulation and result analysis

• The experiment compare the traditional system of load balancing plus 3 sets of buffer server with Hadoop cluster, the results shown in Table 1.

15

Simulation and result analysis

• Figure4 shows that when the traditional manner executes, execution time almost linearly increases.

16

Simulation and result analysis

• Storage occupancy rate of Cache server is true as shown in Figure 5.

17

Simulation and result analysis

• Theoretically, for a particular cluster, the trend of execution time that is slowly grow, with the growth of input data, will not last.

• From this experiment can be seen, when the configuration of parameters accord with the scale of cluster, Hadoop framework compared to the traditional way has a huge advantage for time-consuming aspects of data processing in large.

18

Conclusion

• The simulation of P2P/VoD video systems which designed based on "Hadoop" technology theoretically realizes unlimited expansion and unlimited data computing power.

• This design does not require too much hardware investment, and transform the existing equipment of network transmission, you can achieve fast data transmission.

• And facing the possibly increases of future traffic, the implementation does not change the system structure and upgrading core equipment, just add the appropriate equipment to meet the demand. 19

Conclusion

• If such P2P/VoD system which is promoted widely in the major institutions can form a large cluster of the streaming media.

• These cluster nodes joined together to form a huge video network through the network between the cache server farms, streaming media will become the main platform for online teaching.

20