Download pdf - The basics of remote data replication

Presenter: John Tkaczewski

Duration: 30 minutes

February Webinar: The Basics of Remote Data

Replication

Presenter Introduction

| © Copyright FileCatalyst, 20132

John Tkaczewski• Co-Founder and President of Unlimi-Tech Software

• Oversees creative and technical vision for FileCatalyst Webmail and FileCatalyst Workflow

• Blog: http://www.filecatalyst.com/blog

• Sign up for our next user forum in London, UK next week on Monday (Feb 25th, 2013)

http://www.filecatalyst.com/blog

Agenda

• Remote data replication overview

• Remote data replication scenarios

• Data replication vs. sync

• Replication tools within FileCatalyst

• Platform specific gotchas

• Data replication across operating systems

• File system events

• HotFolder to HotFolder replication

• DEMO

• 30 day free trial and licensing options

• Questions (please type them in as we go)


Remote data replication overview

Remote data replication

• Data replication, also known as remote folder mirroring or folder synchronization, occurs when the same folder contents is duplicated across 2 or more computers, often across wide area networks (WANs) or geographically dispersed offices

• Must support large data sets: – Complex directory tree structures

– Thousands or even millions of files

– Combination of small and very large files up-to terabyte

• Wide Area Files System, ideally must work at file system level to prevent continuous scanning of folders.


Remote data replication scenarios

Remote data replication is often used when organizations require consistent data across multiple geographical locations.

• Possible use cases:– Remote Backups for disaster recovery

– Mirroring of Data across multiple offices (for faster data access)

– Moving the data to the edge (closer to the consumers)

– Global Content Distribution Network (CDN)


Data replication vs. sync

• Replication (or mirroring) involves a single master node and one or multiple slave nodes

• Sync involves multiple nodes all acting as master nodes. Every node can modify any file in the folder. – Sync requires complex rules to prevent and handle collisions

– Sync across more then 2 master nodes is even more challenging

– Sync is not recommended with FileCatalyst HotFolder


Replication tools within FileCatalyst


HotFolder

Replication tools within FileCatalyst (Cont’d)

Hotfolder features useful for data replication:

• Scheduler for tasks and bandwidth

• File System Events

• Delta (RSYNC)

• Data-set rules

• Monitoring and notifications

• ZIP chunking

• MD5

• File transfer cache


Platform specific gotchas

Common file attributes for all platforms:

File name, Date Created, Date Modified, Owner


Operating System File Attributes

MacOS Extended attributes (xattr), Resource Forks, Symlinks, Permissions (Read Write Execute), Access Control Levels (ACL’s)

Windows Permissions, Attributes: Read Only – Archive - Hidden – System, Security Attributes

Linux Permissions (group, user, other)

Data replication across operating

systems

• Extended File Attributes are LOST when moving a file between operating systems.

• But it’s OK because extended attributes may not be always needed.


File system events

The file system events provides a way for an application to ask for notification when the contents of a directory hierarchy are modified.

For example, an application can use this to quickly detect when the user modifies a file within a folder using another application.

• Avoid scanning large folders to detect changes! Use file system events

• File system events only work on local drives (no SMB or other network drive technology)

• Transfer the initial file set without file system events, THEN turn it on.


HotFolder to HotFolder replication

• 2 Hotfolders connected to the same FileCatalyst Server

• Replication can be done between both HotFolders

• All the data will go through the server, however no data will be stored on the server

• This reduces the amount of disk space required at the server


Demo: Acceleration with remote data

replication using FileCatalyst HotFolder

30 day free trial and licensing cost

To request free trial of FileCatalyst Direct visit http://www.filecatalyst.com/trial

Licensing Options:

• Perpetual license is about $10,000 USD (one-time fee)

• Leased license is about $500/month

• Hosted License on a shared server is about $200 per month


http://www.filecatalyst.com/trial

Thank you for attending our February webinar!

Questions? Please type in your questions.