Final ProjectImage Compression Storage Policy for Openstack Swift

Group 7Ikwhan Chang / Priya Vishnu / Prashanth Iyengar

Prof. Younghee Park

12-Dec 2016

1

Team

Ikwhan Chang

Class ID: 4

Student ID: 010754103

Priya Vishnu

Class ID: 25

Student ID: 010739261

Prashanth Iyengar

Class ID: 9

Student ID: 010711233

2

From Computer Engineering..

INDEX

• Background

• Our Approach

• Project Flow

• Image Compression Technique

• Challenges

• Testing

• Demo

• Performance

• Future Scope

• Q&A

3

Background

• Why Image Compression Storage Policy?

• Recently, individuals have started using cloud storage services to back up their personal files including images.

• 64% percent of the website content are image types [Source: http://httparchive.org ]

• Cloud platform has been used as Storage as a Service to store the web page image contents

4

Why Image Optimization is needed?5

KeyCDN - 18 Tips for Website Performance OptimizationForbes - How To Optimize Your Website's Performance And Keep Customers Happy

Many expert recommend image optimization to increase web performance

Our Approach

• Analyzed the Swift Code as our first step

• Understood the swift proxy code flow

• Identified the files that we need to modify to implement an additional storage policy

• Identified the most suitable image compression technique

• Modified the Swift code to add the new image compression policy and integrated the compression procedure

• Developed a web page to demonstrate the usage of the new policy

6

_finalize_put_()

Image Object Upload Flow7

Image Object Download Flow8

Image Compression Technique

• Ref: https://pillow.readthedocs.io/en/3.4.x/handbook/overview.html

9

• We have made use of the Pillow library to achieve image compression

• It is a free Python Imaging Library (PIL) that adds image processing capabilities to the Python

• PIL supports multiple image file formats

• It is efficient and has many powerful features to support the manipulation of the image files

Challenges

• Understand the swift proxy server code flow

• Understand how the existing storage policy has been implemented

• Understand the swift code base to identify the files that needed to be modified to implement the compression policy

• Integration of the compression policy with the existing storage policy

• Developing the web page to demonstrate the usage

10

Flow Diagram Comparison11

Swift Proxy

File Upload

Zone

Ring

Storage Server #1without Image

compression policy

Container #2

Container #3

Container #1

Region

Response: image1.png500 MB

500MB

curl -v -X PUT image1.png http://10.240.0.5/v1/AUTH_test/Normal

Swift Proxy

File Upload

Zone

Ring

Storage Server #1with Image compression

policy

Container #2

Container #3

Container #1

Region

125MB

curl -v -X PUT image1.png http://10.240.0.5/v1/AUTH_test/Compressed

Request: image1.png500 MB Response:

image1_compressed.png125MB

Request: image1.png500 MB

Request: image1.png500 MB

500MB

1. Test Upload 2. Middleware3. Save image into Swift

with Compression Policy

4. Run the demo5. Check the loading speed of resources

/etc/swift/swift.conf

Swift stat

Demo15

https://www.youtube.com/watch?v=lqAnk5Hf4xM

Performance16

Reduced Size: 6.4MB -> 1011KB (-85%)Page Loading Speed: 4.85 -> 1.01 (-80%)

Original (6.4MB) Reduced (1011KB)

Future Scope

• Read the chunk file within the proxy server and compress the chunk and then send the compressed chunk to Object Server

• Deduplication can be added as an enhancement to the storage policy

• Support multiple/widely-used image format

17

References

• Swift All In One http://docs.openstack.org/developer/swift/development_saio.html

• Swift Command Line http://docs.openstack.org/cli-reference/swift.html

• Adding Storage Policies to an Existing SAIO http://docs.openstack.org/developer/swift/policies_saio.html

• OpenStack Object Storage (Swift) Essentials by Amar Kapadia; Sreedhar Varma; Kris Rajana, Published by Packt Publishing, 2015

• Uday’s swift policy code https://github.com/udaykd09/swift

18

19

Questions??• Source Code

• Swift: https://github.com/ikwhan-chang/swift

• Test Client: https://github.com/ikwhan-chang/OpenStack_Swift_Image_Compression.git

Thank You20