9
Chromatophore - A Na.ve CephFS Backup and Archive Tool Brady Deetz Laureate Ins.tute for Brain Research

Chromatophore - A Nave CephFS Backup and …...Research At LIBR • Neuroscience-based, clinical and developmental research: 1. To develop neuroscience-based individually predic.ve

  • Upload
    others

  • View
    1

  • Download
    0

Embed Size (px)

Citation preview

Page 1: Chromatophore - A Nave CephFS Backup and …...Research At LIBR • Neuroscience-based, clinical and developmental research: 1. To develop neuroscience-based individually predic.ve

Chromatophore-ANa.veCephFSBackupandArchiveTool

BradyDeetzLaureateIns.tuteforBrainResearch

Page 2: Chromatophore - A Nave CephFS Backup and …...Research At LIBR • Neuroscience-based, clinical and developmental research: 1. To develop neuroscience-based individually predic.ve

ResearchAtLIBR

•  Neuroscience-based,clinicalanddevelopmentalresearch:1.  Todevelopneuroscience-basedindividuallypredic.veassessments.2.  Todevelopnovelbrain-bodybasedinterven.ons•  Focus:mood,anxiety,addic.on,orea.ngdisorders.

3.  Touseexperimentalsystemstoquicklytestassessmentsandinterven.ons.

Page 3: Chromatophore - A Nave CephFS Backup and …...Research At LIBR • Neuroscience-based, clinical and developmental research: 1. To develop neuroscience-based individually predic.ve

MyUnfortunateInheritanceNovember2015

•  1PB-SpectraLogicT950e•  LocalDisks•  SpectraLogicBlackPearl•  400TB-OracleZFS•  100TB-EMCIsilon

Page 4: Chromatophore - A Nave CephFS Backup and …...Research At LIBR • Neuroscience-based, clinical and developmental research: 1. To develop neuroscience-based individually predic.ve

CephDesign

•  8xOSDnodes–  256GBRAM–  2xIntelS3610forOS–  24x6TBEnterpriseSATA(24slotchassis)–  2xIntelP3700–  2xXeon2660V4

•  28x2.0GHz– MellanoxConnectX-4(40Gbps)->BrocadeVDX6940

Page 5: Chromatophore - A Nave CephFS Backup and …...Research At LIBR • Neuroscience-based, clinical and developmental research: 1. To develop neuroscience-based individually predic.ve

CephDesign(Cont…)

•  2xMDS(ac.ve/standby)+3xMONnode– 128GBRAM– 2xIntelS3610forOS– 2xXeon2643V4•  12x3.4GHz

– MellanoxConnectX-4(40Gbps)->BrocadeVDX6940

Page 6: Chromatophore - A Nave CephFS Backup and …...Research At LIBR • Neuroscience-based, clinical and developmental research: 1. To develop neuroscience-based individually predic.ve

Chromatophore?

•  UsesCephFS’sna.velibraries•  Parallel(MPI+OpenMP)•  Backupstodifferentkindsoftargets– Ceph– SpectraLogicS3– AmazonS3– Glacier– Tar– POSIXfilesystems

Page 7: Chromatophore - A Nave CephFS Backup and …...Research At LIBR • Neuroscience-based, clinical and developmental research: 1. To develop neuroscience-based individually predic.ve

WorkerThread

WorkerThread

WorkerThread

WorkerThread

WorkerThread

WorkerThread

WorkerThread

WorkerThread

WorkerThread

WorkerThread

WorkerThread

WorkerThread

WorkerThread

WorkerThread

WorkerThread

WorkerThread

ConfigFile(JSON) BackupHistory(HDF5)

Page 8: Chromatophore - A Nave CephFS Backup and …...Research At LIBR • Neuroscience-based, clinical and developmental research: 1. To develop neuroscience-based individually predic.ve

Wheretogetit?

•  WillbereleasedbytheendofthisDecember– Excuses…•  S.llwri.ngunittests•  Valida.ngDS3codewithSpectraLogic•  RecentlyportedfromPythontoJava

•  hkps://github.com/bdeetz/chromatophore

Page 9: Chromatophore - A Nave CephFS Backup and …...Research At LIBR • Neuroscience-based, clinical and developmental research: 1. To develop neuroscience-based individually predic.ve

Q/A

BradyDeetzLaureateIns.tuteforBrainResearch(918)[email protected]@gmail.com