MongoDB on Windows Azure

Page 1

MongoDB on Windows Azure

WIP Report

MongoBoston September 2010

Page 2

Brief Survey

• Business Decision Makers?

• IaaS (ex. AWS or RackSpace) users

Page 3

Management of Resources: What you manage

Applications

Development &

Runtime Kernels

Databases

Security, Management,

Load Balancing &

Integration

Logical Servers,

Storage

Virtualization

OS

Server Hardware

Networking,

Utilities, Physical

PRIVATE(On-Premise)

Applications

Development &

Runtime Kernels

Databases

Security, Management,

Load Balancing &

Integration

Logical Servers,

Storage

Virtualization

OS

Server Hardware

Networking,

Utilities, Physical

Infrastructure

as a Service (IaaS)

Applications

Development &

Runtime Kernels

Databases

Security, Management,

Load Balancing &

Integration

Logical Servers,

Storage

Virtualization

OS

Server Hardware

Networking,

Utilities, Physical

Platform as a

Service (PaaS)

Yo

u M

an

ag

e Yo

u M

an

ag

e

Yo

u M

an

ag

e

Ma

na

ge

d b

y V

en

do

r

Ma

na

ge

d b

y V

en

do

r

Applications

Development &

Runtime Kernels

Databases

Security, Management,

Load Balancing &

Integration

Logical Servers,

Storage

Virtualization

OS

Server Hardware

Networking,

Utilities, Physical

Software as a

Service (SaaS)

Yo

u C

on

fig

ure

Ma

na

ge

d b

y V

en

do

r

Page 4

Windows Azure is an internet-scale cloud services platform hosted in Microsoft data centers around the

world, proving a simple, reliable and powerful platform for the creation of web applications and

services.

The Windows Azure Platform

Page 5

Defining the Windows Azure Architecture

Page 6

GOAL: Massive Scalability

Scale out – not up – by replicating

worker instances as needed.

Allow applications to scale

user and compute processing

independently.

Two role types: Web Role & Worker RoleWindows Azure applications are built with web roles, worker roles, or a combination of both deployed to a number of instances.

Each instance runs on its own VM (virtual machine), replicated as needed

Compute Services in Windows Azure

Page 7

BLOBS: Provide a simple

interface for storing

named files along with

metadata for the file.

TABLES:

Provide structured

storage. A table is

a set of entities

which contain a set

of properties.

QUEUES:

Provide reliable

storage and delivery

of messages for

an application.

Blobs, Tables, Queues and Drives

DRIVES:

A durable NTFS file

system volume,

sharable across

instances.

Page 8

Running even one Mongo instance

• Azure runs Windows 2008 server VMs– .NET 4 included. All else must be bundled.

– XCOPY = GOOD. Registry and MSI = BAD

– MongoDB = XCOPY = GOOD!!!

• Mongo needs disk storage.– Choose: non-durable or durable

– Local disk: non-durable. Blob: durable!

• Mongo needs connection port– Multiple instances can’t talk to each other

– Client can’t choose server instance

– Can’t scale – multiple instances are independent!

– Azure uses random ports – Mongo needs to use this port• Mongod won’t allow mapping of http server port!

Page 9

Single-Instance Solution

• Single worker role instance

• Local (non-durable) or blob (durable)

storage

• Single port mapped to mongod.exe

– Server only; no web server access

• Mongod instance as spawned process

– Not as service

– Must specify mapped port, data path, no http

• If server exits, recycle instance

Page 10

Here’s How It’s Done

var storage = RoleEnvironment.GetLocalResource("MongoDBStorage");

varstoragePath = storage.RootPath;

string mongoRoot = Path.Combine(Environment.GetEnvironmentVariable("RoleRoot") +

@"\", @"approot\MongoDB");

intport = RoleEnvironment.CurrentRoleInstance.InstanceEndpoints["MongoIn"].IPEndpoint.Port;

varcmdline = String.Format( "--nohttpinterface --port {0} --dbpath {1} ", port, storagePath);

MongoProcess= new Process()

{

StartInfo= new ProcessStartInfo(Path.Combine(mongoRoot, @"mongod.exe"), cmdline)

{

UseShellExecute= false,

WorkingDirectory= mongoRoot

}

};

MongoProcess.Start();

Page 11

Managing Instances

• MongoD.exe provides status

– http (28017), mongod shell

• Azure Load Balancer hides instances!

– Single IP/Port inbound from client apps

– Multiple port round-robin internally

• Azure-hosted apps can access instances!

– Must treat all instances equally

– Assuming specific instance access = bad

Page 12

Client works against single IP/Port

No way for client app to access individual server

No shared storage

Mongo

MongoD servers

Client-side app

Single IP

Multiple ports

Page 13

Multiple servers accessible via Azure server

Mainly for management purposes

Mongo

MongoD servers

End user

Single IP

Multiple ports

Page 14

Replica Sets Challenges

• No shell access

– how to configure?

• Single IP

– How to access?

– How to monitor?

• DB access

– How to access Master in set?

• What about storage?

Page 15

Replica Set Solution

• ReplicaSet worker role– Runs mongod with –replSet

• Management role (either worker or part of web)– Enumerates all ReplicaSet role instances

– Builds configuration document

– Connects to one node; uploads configwhich initializes replica set

• Client application– Connects to replica set via compatible driver

• Storage– Either local storage or blob storage.

– If blob storage, each replica set node has its own blob

Page 16

Areas to be Explored

• Replica set per deployment

• Self configuring replica sets and shards

• Configuration data held in WA Storage

– Instance and Replica Set information pushed

• Mongo HTTP port configuration

Page 17

Coming Soon to a Cloud Near

You• RDP access to instances

• Mongo Sharding will be simpler to

implement than replication due to its

cloud friendly architecture