20130305 lizard fs

your DATA our PASSION

Speaker:

Title:

Peter aNeutrino (aka Piotr Modrzyk)

TF-STORAGE - 05-06.03.2013

Architecture of LizardFS – – cloud enabling – distributed file system for data centers

Year 2003 Google Published document about Google File System.

3/19

Year 2005 CoreTechnology Developed MooseFS for Gemius

Year 2008 MooseFS is GPLv3but coreTechnology owns the code

Year 2013 ●MooseFS is forked to LizardFS●The developers team grows from ONE to 5+community.●Always OpenSource

LizardFS in numbers:

ONE instance max size:

16 ExaBytes

ONE file max size:

128 PetaBytes

Files number limit

536 000 000

4/19

LizardFS - ARCHITECTURE

5/19

Thanks to LizardFS you can save money on the HDDs price and use the biggest like 4TB+ to increase density and

minimize the power consumption.

6/19

ceny z serwisu CENEO

Unrecoverable Read Error

Total Disk Faliure

Annual Fail Rate

Mean Time Before Failure

RAID 5RAID 6

Repair time is too high

for 1TB+ HDD repair takesfrom 2h...up to 20h

oreven 14 days in Tier-4 ;)

7/19

Data ReplicationIs an answer for: AFR, MTBF, TDF

8/19

File

File transfer

LizardFS uses built-in mechanism of checksumingIt is an answer for a problem with URE and errors of other

mediums like network cables or cache memory

9/19

clientChunk server

Chunk + CRC

LizardFS uses build-in mechanism of checksuming

10/19

clientChunk server

both ends use checksums

Data flow

Snapshots are very fast and always possible

11/19

FILE 1

FILE 1

Meta data

DATA

FILE 1

12/19


FILE 1

FILE 1

Meta data

DATAFILE 1

FILE 2

FILE 1

Meta data copy

FILE 1

FILE 2

FILE 1

13/19


FILE 1

FILE 1

Meta data

DATAFILE 1

FILE 2

FILE 1FILE 1

FILE 2

Changed DATA

LizardFS has transparent trash-bin

14/19

LizardFS web Monitoring

15/19

Online DEMO....

LizardFS web Monitoring

16/19

Online DEMO....OR not

LizardFS — GPLv3

17/19

LizardFS

18/19

72 000 lines of code

Customer Driven Development

19/19

your DATA our PASSION

info @ .pl