Horizon Workspace: Data Deep Dive

Marcello Golfieri, VMware

Rasmus Jensen, VMware

EUC5238

#EUC5238

2

Agenda

Introduction to Horizon Workspace

Architecture

User Experience (Users, Clients, Sharing)

Data Deep Dive

Q&A

3

Whitepaper https://communities.vmware.com/docs/DOC-24651

4

Objectives

Understand the

architecture

Best practices

and

recommendations

Features

Scaling

Inner workings

of Data

5

Horizon Workspace A Short Introduction

6

• Data Access across devices

• Collaboration & sharing

• Policy based data controls

• 1-Click Request/Access

• Single Sign On for SaaS Apps

(SAML 2.0)

• Centralized access to apps

from Web Client and Mobile

• Single vApp with Flexible

Installation

• User & Group Entitlement

based on AD

• Manage Data, App and

Desktops from Single Portal

Horizon Workspace – Overview • Native View Client from

Horizon w/SSO

• Horizon Access from a View

Desktop w/SSO

• Access View Desktop from Web

Clients and tablet devices

7

Architecture Horizon Workspace Components

8

Horizon Workspace – vApp

Workspace vApp

Configurator

VA

OS (SLES)

tcserver

Service VA

OS (SLES)

App

API (Internal)

postgres tcserver

Data VA

OS (SLES)

App

API (Internal)

mysql LDAP Jetty

App

Connector

VA

OS (SLES)

tcserver

App

Gateway

VA

OS (SLES)

Nginx

Modules

• Central Wizard UI

• Distributes settings

across VAs

• Network, Gateway,

vCenter, SMTP attributes

• Add / remove modules

• Manage certs, security

• User authentication

• AD secure bind and synchronization

• Handle scheduling

• Sync View pools and ThinApp

• Enables single user-

facing domain (FQDN)

• Routes requests to

correct node

• Reverse proxy insulates

VAs

• Workspace Admin UI

• Application Catalog

• Manage users entitlements

and policies

• Reporting / Audit

• Stores files

• Controls file sharing policy for

internal and external users

• Manage file preview service

• Serves end user web UI

9

Horizon Workspace – Data VA

Data VA #1

OS (SLES)

App

API

mysql LDAP Jetty

Data VA #2

OS (SLES)

App

API

mysql Jetty

Data VA #3

OS (SLES)

App

API

mysql Jetty

10

User Accounts and Clients User Experience, Collaboration and Sharing

11

Horizon Workspace – User Accounts

Virtual User mail: virtual@acme.com

Regular User mail: user@company.com

• Stored in AD

• Synced via Connector

• User Attributes synced

and stored in Service

DB / OpenLDAP

• Created based on invites

from regular users

• Stored in Service DB

/OpenLDAP based on

email id

• Managed separately

from: user@company.com

to: virtual@acme.com

Invite send via email

12

Horizon Workspace – Sharing

Share with both internal and external users (Virtual Users)

Sharing capabilities

• Direct links to sharing a file (View Only)

• Sharing folders (View, Edit, Share)

• Sharing cannot be assigned to a Virtual User

Admins can control:

• With whom data is shared (eg. “deny: @gmail.com” – black/white listing)

• What file types can be stored (file extension based)

• This happens as part of the Class of Service (COS)

• It is possible to make changes on a per user basis

Admins cannot access user data

13

Horizon Workspace – Clients

Desktop Clients (Windows, Mac)

• Sync updates to/from Horizon Data

• Handles folders, files, conflicts etc.

• Always running in the background – cyclic polling

Mobile Clients (iOS, Android)

• Read access to files and folders with Preview feature

• Uploading of files

• Runs on demand when launched

• Option to make a file available offline (“Favorite”)

Synced files are encrypted on the mobile devices

14

Deep Dive Horizon Workspace Data – Components

15

Horizon Workspace – Class of Service (COS)

COS is defined and assigned by Horizon Workspace admins

Defines things like:

• Quota and warnings

• Max file size

• Allow/Deny public sharing

• User deleted files lifetime

• …

Data-VA nodes are members of a COS

A user can only be entitled to a single COS at the same time

Tiers

• “Gold, Silver, Bronze”

• “CXO, Sales, Marketing”

16

Horizon Workspace – OpenLDAP

The users are provisioned from AD

• Initially based on attribute mappings with AD

Additional user attributes are then populated in OpenLDAP:

• User accounts

• COS definitions

• Virtual Users

• Global and node specific

Resides on the original Data-VA, hence it should be:

• Under vSphere HA

• Excluded from any Class of Service

• Take extra care of this VA

17

Horizon Workspace – User Data Structure Breakdown

Every user is entirely hosted on single Data-VA node

• 1:1 between user data the Data-VA node

Each new file synced from any source produces:

• Metadata added in MySQL

• User Index being updated

• File added to the store

Store is accessed on disk only when attempting to:

• Download

• Preview

• Move/Delete/etc.

18

Horizon Workspace – Data Indexing

It's what allows extremely quick searches

Partial word matching

Based on Lucene 3.5.0

Every file added triggers an update to the user index folder

When searching accounts with shares in place, sharer account's

index is being inquired:

• Locally if on the same data-va

• Remotely if on another data-va

19

Horizon Workspace – Data MySQL

Holds every detail that has to be frequently and quickly read:

• Filenames

• Sharing info

• Folder structure

• Revision tracking

InnoDB tables for ACID compliance

Buffers as much as it can

New files added are stored on the active primary volume

• No encryption

• File revisions are full copies

• No application de-duping, delegated to the storage layer

20

Horizon Workspace – Data Store

MySQL has the info that defines the path to each file

• Full path is mainly determined by mail_item columns in MySQL:

Every file is stored without changes to the content

/opt/zimbra/store/ 0/ 1/msg/ 0/ 257- 3.msg

0 right bitshift by 12 of 1/

1 mail_item.mailbox_id/ 0 right bitshift by 12 of 257/

257 mail_item.id-

3 mail_item.mod_content

/opt/zimbra/store FS path of the zmvolume

If it's not in MySQL, it doesn't exist!

21

Horizon Workspace – Data Disk Layout

Contains SLES OS (40GB)

VMFS Datastore

Horizon Data Application root /opt/zimbra

User Files Store /opt/zimbra/store

/

/opt/zimbra/db

/opt/zimbra/index

/opt/zimbra/redolog

/opt/zimbra/log

/opt/zimbra/backup

/opt/zimbra/data

VMDK

VMDK

VMDK

VMDK

VMDK

VMDK

VMDK

VMDK

VMDK

MySQL database

Lucene indexes

Not being used

Main logs directory

Component backup files

tmp folder for processes

NFS

User Files Store

/opt/zimbra/store##

http://kb.vmware.com/kb/2053549

22

Deep Dive Gateway VA and Data VA Relationship

23

Horizon Workspace – Gateway-VA and Backend Relationship

24

Deep Dive Admin Operations

25

Why move or consolidate?

Running out

of space

Horizontal Scale

Adding/Removing

NFS/VMDK

De-commission a

Data VA

26

Deep Dive Admin Operations –

Moving Accounts

27

Moving Accounts

rsync

TCP/22

hzndataHost: source.domain.local

hzndataAccountStatus: active

source

Files

Index

Metadata

destination

Files

Index

Metadata

First initial rsync transfers the big bulk of the

account while it's live

28

Moving Accounts (Continued)

rsync

TCP/22

hzndataHost: source.domain.local

hzndataAccountStatus: active

source

Files

Index

Metadata

destination

Files

Index

Metadata

Smaller subsequent transfers.

This until the transfer lasts less than 30s.

rsync

TCP/22

rsync

TCP/22

29

Moving Accounts (Continued)

hzndataHost: source.domain.local

hzndataAccountStatus: maintenance

source

Files

Index

Metadata

destination

Files

Index

Metadata

A last rsync kicks in after the user account has been put in

maintenance status and every outstanding action has been

flushed and committed.

rsync

TCP/22

30

Moving Accounts (Continued)

hzndataHost: destination.domain.local

hzndataAccountStatus: active

source destination

Files

Index

Metadata

hzndataHost gets updated to point to the new data-va node,

memcached routes get updated.

Files

Index

Metadata

31

Moving Accounts (Continued)

hzndataHost: destination.domain.local

hzndataAccountStatus: active

source destination

Files

Index

Metadata

After validating an admin can purge old account if all is ok.

32

Deep Dive Consolidating Stores

33

Store Consolidation – VMDK to NFS

LVM - /opt/zimbra/store

Read-only

VMDK

NFS - /opt/zimbra/storeXX

Read-write (active)

LVM - /opt/zimbra/store

Read-write (active)

VMDK

Source Data-VA Destination Data-VA

VMDK

34

Store Consolidation – NFS to VMDK

LVM - /opt/zimbra/store

Read-write (active)

VMDK

VMDK

LVM - /opt/zimbra/store

Read-only

VMDK

NFS - /opt/zimbra/storeXX

Read-write (active)

Destination Data-VA Source Data-VA

35

Store Consolidation – Removing VMDKs

LVM - /opt/zimbra/store

Read-write (active)

VMDK

LVM - /opt/zimbra/store

Read-write (active)

Source Data-VA Destination Data-VA

VMDK

VMDK

36

Deep Dive Scaling Horizon Workspace

37

Horizon Workspace – Scalability

Horizon Workspace 1.000 users

Gateway VA is sized for 2.000 users in the above

vCPU RAM (GB)

Configurator VA 1 1

Gateway VA 6 32

Service VA 6 8

Connector VA 2 4

Data VA 6 32

38

Scalability – 1.000 Users with High Availability

NFS VMFS

Gateway

Connector

Data

Configurator

Service

39

Horizon Workspace – Preview Server Scalability

vCPU RAM (GB) Users

MS Preview Server 8 4 1.000

40

Deep Dive Performance Monitoring and Tuning

41

Horizon Workspace – Data: Performance Tuning

It's an HUGE topic and impossible to ratify in a few general rules

Initial assumptions:

• (v)Hardware has to be tailored to sustain the use case loads

• NFS storage properly sized and tuned

• When facing performance issues, GSS should be contacted, regardless.

NOTE:

This section and the tools herein described are neither officially

documented nor supported. This section is aimed at admins willing to:

• investigate on their own

• driven by pure curiosity

42

Performance Tuning – What’s Being Collected?

To properly investigate performance issues, zmdiaglog collects the

following information on each data-va node:

• General Data-VA environment info (zmdumpenv)

• Java heap dump

• Performance CSVs (cpu,mysql,io,soap, ...)

• 10 snapshots, each 10s apart of the following:

• Thread dumps

• top

• netstat

• procs

• ps

• Main logs

43

Performance Tuning – Main Action Items

Once collected, zmdiaglog data contains enough data to tune the

system. Main tuning points are usually:

• JVM options

• Memory allocations

• GC options

MySQL buffer sizing

Main OpenLDAP-based configuration changes

Change of storage targets (e.g. VMDK->NFS)

Horizontal reallocation of users

44

Performance Tuning – Charts

Charts are a great way to have a quick look at the load trends:

45

Summary

Understand the

architecture

Best practices and

recommendations

Features

Scaling

Inner workings

of Data

vApp

Gateway VA

FQDN

Configurator

Share/Collaborate

Admin

Preview

COS

Vertical/Horizontal

Look at performance

OpenLDAP

Indexing

User files

“If its not in

MySQL…”

NFS for production

#1 Data-VA

Horizontal Scale

46

Q&A

47

Whitepaper https://communities.vmware.com/docs/DOC-24651

48

Other VMware Activities Related to This Session

HOL:

HOL-MBL-1304

Horizon Workspace - Explore and Deploy

Group Discussions:

EUC1005-GD

Workspace with Rasmus Jensen

EUC5238

THANK YOU

Horizon Workspace: Data Deep Dive

Marcello Golfieri, VMware

Rasmus Jensen, VMware

EUC5238

#EUC5238