Azure Storage ServicesJalpesh Vadgama@Jalpesh, http://www.dotnetjalps.com

About me• More then 10+ years of experience in Microsoft.NET and related technologies

• Currently working as Project Manager at TatvaSoft

• Have been Awarded as Microsoft Most Valuable Professional (Visual C#) for year 2010,2011 and 2012.

• Frequently writing blogs about Microsoft.NET and related technologies at my blog http://www.dotnetjalps.com

• Also awarded as Dzone Most Valuable Blogger.

• Active Member of Ahmedabad .NET User Group.

Windows Azure Storage• Cloud Storage, Any time any where access• Durable and Highly available• Pay for what you use• Massively Scalable – Easily build internet scale enterprise level application

- 35 trillion plus + Storage options - 3.5 million request + Request/sec on average

• Azure Storage uses an auto-partitioning system that automatically load-balances your data based on traffic• Exposed via Rest APIs or Also Client Libraries are available in languages like .NET, Java and Node.js.• Azure storage is accessible from any platform like desktop, Web or devices

Azure Storage Account

• An Azure storage account is a secure storage account that gives access to services in Azure Storage• Two types of accounts

- Standard - Premium

• Standard Storage account Include all the Azure Storage Abstractions (Blob, Table, Queue and File Storage)• Premium Storage account supports Azure Virtual Machine disks(SSDs) only right now

Agenda

Blobs Files

Queues Tables

Agenda

Blobs Files

Queues Tables

Microsoft AzureStorage Blob

Azure Blob Storage• Blob means Binary Large Objects• Blob Storage is service for storing large amount of Unstructured data such as binary text or images• You can use Blob Storage to store content like

DocumentsSocial data such as photos, videos, music and blogsImages and text for web application Configuration Data for cloudLog files

Azure Blob Storage• It's a platform-as-a-service (PaaS) feature of Azure, which means you just store and retrieve files, paying only for the actual amount of storage you use, and Azure automatically takes care of setting up and managing all of the VMs and disk drives required for the service.

• When you store a file in the Blob service, you can easily make it publicly available over the Internet.

Blob Storage Concepts

BlobContainerAccount

http://{account}.blob.core.windows.net/{container}/{blobname}

Pages/Blocks

contoso

PIC01.JPG

Block/Page

Block/Page

PIC02.JPGimages

VID1.AVIvideos

Agenda

Blobs Files

Queues Tables

Microsoft AzureStorage Table

• Remember It’s not an RDBMS table.

• Table storage is Microsoft’s NoSQL key/attribute store – it has a schema less design, making it different from traditional relational databases.

• Mental Pictures is Entity

• Each entity can have up to 255 properties up to 1 Mb per entity

• A Entity is acts as Rows in this table

• Each Entity must have three properties• Partition Key, Row Key and Time Stamp

Table Storage Details

• Tables are partitioned to support load balancing across storage nodes. A table's entities are organized by partition. A partition is a consecutive range of entities possessing the same partition key value

• The row key is a unique identifier for an entity within a given partition. Together the PartitionKey and RowKey uniquely identify every entity within a table

• A property is a name-value pair. Each entity can include up to 252 properties to store data.

• You can access table via following URL• http://<storage account>.table.core.windows.net/<table>

Table Storage Details

Table Storage Concepts

EntityTableAccount

contoso

customers

photos

Name =…Email = …

Name =…EMailAdd=

Photo ID =…Date =…

Photo ID =…Date =…

Agenda

Blobs Files

Queues Tables

Microsoft AzureStorage Queue

• Azure Queue storage is a service for storing large numbers of messages that can be accessed from anywhere in the world via authenticated calls using HTTP or HTTPS.• A single queue message can be up to 64 KB in size, and a queue can contain millions of messages, up to the total capacity limit of a storage account.• A storage account can contain up to 500 TB of blob, queue, and table data.

Queue details

• Common Usage of Queue Storage Includes • Creating a backlog of work to process asynchronously• Passing messages from an Azure Web role to an Azure Worker role

• Queues are accessible from following URL format • http://<storage account>.queue.core.windows.net/<queue>

• A Queue contains messages and a message can be maximum of 64 Kb of size

Queue details

Queue details

QueueStorage Account

Image Processing

Images to download

Images to Resize

Agenda

Blobs Files

Queues Tables

Microsoft AzureStorage Files

• Microsoft Azure virtual machines and cloud services can share file data across application components via mounted shares, and on-premises applications can access file data in a share via the File storage API.• File storage offers shared storage for applications using the standard SMB 2.1 protocol.• Applications running in Azure virtual machines or cloud services can mount a File storage share to access file data, just as a desktop application would mount a typical SMB share. Any number of Azure virtual machines or roles can mount and access the File storage share simultaneously.

Azure Files

• Since a File storage share is a standard SMB 2.1 file share, applications running in Azure can access data in the share via file I/O APIs. Developers can therefore leverage their existing code and skills to migrate existing applications. • Common uses of File storage include

- Storing shared application settings, for example in configuration files- Storing diagnostic data such as logs, metrics, and crash dumps in a

shared location- Files that is accessible via multiple locations

• Files are addressable using the following URL format- https://<storage

account>.file.core.windows.net/<share>/<directory/directories>/<file>

Azure Files

Azure Files

Agenda

Blobs Files

Queues Tables

Microsoft AzurePremium Storage

• Premium Storage delivers high-performance, low-latency disk support for I/O intensive workloads running on Azure Virtual Machines• You can attach several Premium Storage disks to a virtual machine (VM). With Premium Storage, your applications can have up to 32 TB of storage per VM and achieve 64,000 IOPS (input/output operations per second) per VM with extremely low latencies for read operations

Azure Premium Storage

Microsoft AzureStorage Blob

in Details

Two Types of Blobs Under the Hood

Block Blob

Page Blob

Two Types of Blobs Under the Hood• The maximum size of blob storage can be 200 GB.• Each blob are comprised of blocks and each block can be identified by an Id.• A blob can include maximum of 50,000 thousand blocks.• If you write a block blob storage that is less than 64mb of size then you can upload that in single operation.

Block Blob:

Two Types of Blobs Under the Hood

• With a block blob storage, You can upload multiple block parallel to reduce upload time•Each block can include MD5 hash to verify the Transfer, so you can track upload progress and resend blob storage if required

Block Blob:

Two Types of Blobs Under the Hood• Page blobs are a collection of 512-byte pages optimized for random read and write operations.• To create a page blob, you initialize the page blob and specify the maximum size the page blob will grow. To add or update the contents of a page blob, you write a page or pages by specifying an offset and a range that align to 512-byte page boundaries.

Page Blob:

Two Types of Blobs Under the Hood

• A write to a page blob can overwrite just one page, some pages, or up to 4 MB of the page blob.• Writes to page blobs happen in-place and are immediately committed to the blob. The maximum size for a page blob is 1 TB.

Page Blob:

When to use Block or Page blob• Block Blobs are to manage the upload of large files in parallel while Page Blobs are good for Random read and write operations on a Blob.• I have a huge VHD file which I want to be uploaded in parallel and once uploaded do random read and write on it as needed. So I am not really sure which one should I use in such a case. I guess I would go for the Page Blob because upload is a one time operation for VHD file and after that my hypervisor will keep modifying the file for the life of the file. So it is more important that the random read and write operations are faster then the upload speed.

Restrict Access to Containers and Blobs• Full public read access: Container and blob data can be read via anonymous request. Clients can enumerate blobs within the container via anonymous request, but cannot enumerate containers within the storage account• Public read access for blobs only: Blob data within this container can be read via anonymous request, but container data is not available. Clients cannot enumerate blobs within the container via anonymous request• No public read access: Container and blob data can be read by the account owner only.

Shared Access Signature

• A shared access signature is a URI that grants restricted access rights to containers, blobs, queues, and tables for a specific time interval.• By providing a client with a shared access signature, you can enable them to access resources in your storage account without sharing your account key with them.• The shared access signature URI query parameters incorporate all of the information necessary to grant controlled access to a storage resource.

Shared Access Signature

http://...blob.../pics/image.jpg?sr=c&st=2009-02-09T08:20Z&se=2009-02-10T08:30Z&sp=w

&sig= dD80ihBh5jfNpymO5Hg1IdiJIEvHcJpCMiCMnN%2fRnbI%3d

Demo

Some Useful Resources for Azure Blob Storage

http://azure.microsoft.com/en-in/documentation/services/storage/

http://azure.microsoft.com/en-us/documentation/articles/storage-manage-access-to-resources/

http://www.microsoftvirtualacademy.com/training-courses/windows-azure-storage-design-and-implementation-jump-start

http://www.microsoftvirtualacademy.com/training-courses/microsoft-azure-fundamentals-storage-and-data

Questions?

Thanks!My Blog : http://www.dotnetjalps.com

Twitter : https://twitter.com/jalpesh

Github: https://github.com/dotnetjalps

Facebook : https://www.facebook.com/DotNetJalps

Linked In : http://www.linkedin.com/in/jalpeshvadgama