A Dropbox-like Personal Cloud for OpenStack Swift...A Dropbox-like Personal Cloud for OpenStack Swift Pedro García López Adrián Moreno Martínez Cristian Cotes González OpenStack

A Dropbox-like Personal Cloud for OpenStack Swift

Pedro García López Adrián Moreno Martínez Cristian Cotes González

OpenStack Summit May 2014 - Atlanta

CloudSpaces project ● Open Service Platform for the Next Generation

of Open Personal Clouds: http://cloudspaces.eu

● Objectives:

o Interoperability and privacy

● Partners:

o URV, Eurecom, EPFL, eyeOS (Web Desktop), TISSAT (cloud provider), NEC.

Personal Cloud We understand it as… ● Storage

● Synchronization

● Sharing

Our motivations

● Not a cloud open-source solution available StackSync and OpenStack

● Users lack control of their data Privacy, Client-side encryption, secure sharing…

● Vendor lock-in Interoperability

Big picture

First stage ● Metadata separated from data → StackSync Server → OpenStack Swift

Data flow

Metadata DB

Metadata flow

Desktop client

OpenStack Swift StackSync Server

StackSync Server StackSync Server

Second stage ● MOM (RabbitMQ) → push notifications

● Elastic sync protocol

Data flow

Metadata DB

Metadata flow (Push)

Desktop client

OpenStack Swift

StackSync Server

RabbitMQ

Current stage ● API as a Swift’s proxy

module ● Website ● Android app

Data flow

Metadata DB

Metadata flow (push)

Desktop client

OpenStack Swift RabbitMQ

StackSync API

Mobile client

Metadata flow

StackSync Server StackSync Server StackSync Server

Lifecycle and architecture

Lifecycle

Client 1



Client 2

Client 3

Data flow

Metadata DB


Desktop client


StackSync API

Mobile client

Metadata flow StackSync Server

Desktop client

Tasks

● File system watcher

● Chunking, compression, and encryption

Features ● File versioning ● File sharing ● OS integration

● Overlay icons ● Notifications

Data flow

Metadata DB


Desktop client


StackSync API

Mobile client


StackSync server

Operations

● getAccount

● getChanges

● commit

● …

{ "is_folder":false, "mimetype":"application/pdf", "status":"NEW", "parent_file_id":73990274, "file_id":-534824681, "path":"/documents/", "filename":"StackSync_Guide.pdf", "checksum":-2678858962222278590, "user":"John Doe", "client_modified":"2013-03-08 10:36:41.997", "size":775412, "version":1, "server_modified":"2013-03-08 10:36:41.997" }

Syncing conflicts ● Two users modify the same file at the same time ● Whichever version is processed first wins ● The loser creates a conflicted copy of the file with his

changes

StackSync Server User 1

User 2

Workspace ● Represents the relation between files and users ● Every user has its own workspace ● Every shared folder has its workspace ● A workspace maps into a Swift container

Workspace 2

Shared workspace

Workspace 1

Container 2

Shared container

Container 1 OpenStack Sw

ift

User 1

User 2

Encryption settings Plain

Server-side encryption

Client-side encryption

Swift mappings StackSync OpenStack Swift

user user

workspace

installation tenant

container

admin tenant admin

file N chunks

Sharing process 1. Send share proposal 2. Notify the addressee 3. Proposal gets accepted 4. Create container and

set up ACL X-Container-Read and X-Container-Write

User 1 User 2

StackSync Server

OpenStack Swift

Data flow

Metadata DB


Desktop client


StackSync API

Mobile client


RabbitMQ

RabbitMQ

● Communication between clients and server

● Push notifications

● Load-balancing

● Multicast messages per workspace


Data flow

Metadata DB


Desktop client


StackSync API

Mobile client


StackSync API

StackSync API ● Located in the proxy’s pipeline

● Used by website & mobile clients

● Communicates with StackSync server to get metadata and authorization

● Activates when the X-StackSync-API header is set

StackSync API Auth ● Module located before the API

● OAuth 1.0a implementation

● Sets the user information in the wsgi environment for the API

GET /api/file/4688390 Header: X-StackSync-API: v1

Performance

Traffic overhead

Push vs pull

Server elasticity

Time (minutes)

Num

ber o

f ins

tanc

es N

umber of requests

Using StackSync

Deployment scenarios

OpenStack Swift

StackSync Server

OpenStack Swift OpenStack Swift

Private Hybrid Public

On premises

On cloud provider

StackSync Server StackSync Server

Current deployments

● Universitat Rovira i Virgili

● Tissat (Data center)

● Rediris (Spanish university network)

Community

● Available for:

● Documentation available at stacksync.org

● Easy to deploy for swifters

Conclusions ● StackSync is a ready-to-use Personal Cloud

for OpenStack Swift

● What’s next ● Advanced privacy features

● Interoperability

● iOS and OS X clients

Thanks for your attention!

github.com/stacksync

stacksync.org

Documents

A Dropbox-like Personal Cloud for OpenStack Swift...A Dropbox-like Personal Cloud for OpenStack Swift Pedro García López Adrián Moreno Martínez Cristian Cotes González OpenStack