41
Introduction to Synnefo open source cloud stack Thomas Schend, Systems Architect, CompuGroup Medical AG Open Source Data Center Conference 2014 08. – 10. April, Berlin 06/07/2022 OSDC 2014 1

OSDC 2014: Thomas Schend - Introduction to Synnefo open source cloud stack

  • Upload
    netways

  • View
    218

  • Download
    8

Embed Size (px)

DESCRIPTION

This talk wants to introduce you to Synnefo, an open source, scalable and production ready cloud stack. It consists of a vm management layer which is google ganeti and is essentially a cluster manager and delivers an integrated management of compute, network and storage. It also runs on of the shelf hardware and delivers live migration without or with shared storage. Synnefo is the orchestration and presentation layer which talks to ganeti via an API. To the users it presents a simple web UI and also exposes an openstack compatible API for automation called cyclades. Also it is possible to do flexible, L2 isolated networking. Also it provides a “dropbox” like storage services called pithos. It features a sync client for different platforms and a web UI. Synnefo offers all the benefits of an amazon like cloud but is geared towards persistent virtual machines. Also it is perfect as a replacement of the traditional virtualization stack and is easy to setup and use.

Citation preview

Page 1: OSDC 2014: Thomas Schend -  Introduction to Synnefo open source cloud stack

04/10/2023 OSDC 20141

Introduction to Synnefoopen source cloud stackThomas Schend, Systems Architect, CompuGroup Medical AG

Open Source Data Center Conference 2014

08. – 10. April, Berlin

Page 2: OSDC 2014: Thomas Schend -  Introduction to Synnefo open source cloud stack

04/10/2023 OSDC 20142

Agenda

IntroductionWhat do we need from IaaS?Introducing to SynnefoComponentsGanetiDemoQ & A

Page 3: OSDC 2014: Thomas Schend -  Introduction to Synnefo open source cloud stack

OSDC 2014

successfully worldwide in 35 countries for approx. 385,000 customers

10.04.20233

= 19 countries with own offices = additional 16 countries with customers

2012 Guidance 2013Customers 385.000

Group sales € 451 Mio. € 458 – 463 Mio.

EBITDA € 105 Mio. € 97 – 100 Mio.

employees 3.700

Indicators

Best provider of IT solutions for higher quality and efficiency in health care

Worldwide leader for MIT (Medical Information Technology)

High profitability and an average of 30% growth p.a.

Objectives

Page 4: OSDC 2014: Thomas Schend -  Introduction to Synnefo open source cloud stack

04/10/2023 OSDC 20144

Bio

Systems Architect, responsible for Infrastructure DesignOSS EnthusiastLPIC-2,MCITP,VCP,CCA,DCIE,…

Contact: [email protected]

Page 5: OSDC 2014: Thomas Schend -  Introduction to Synnefo open source cloud stack

04/10/2023 OSDC 20145

Agenda

IntroductionWhat do we need from IaaS?Introducing to SynnefoComponentsGanetiDemoQ & A

Page 6: OSDC 2014: Thomas Schend -  Introduction to Synnefo open source cloud stack

04/10/2023 OSDC 20146

What do we need from IaaS?

ReliableScalable (Backend and VMs)Easy to manageLong lived workloads, not all VMs are EC2 instances ;-)

Page 7: OSDC 2014: Thomas Schend -  Introduction to Synnefo open source cloud stack

04/10/2023 OSDC 20147

“VMs are not cattle, they are pets”

Vangelis Koukis, Technical Lead, Synnefo

Quote

Page 8: OSDC 2014: Thomas Schend -  Introduction to Synnefo open source cloud stack

04/10/2023 OSDC 20148

Agenda

IntroductionWhat do we need from IaaS?Introducing to SynnefoComponentsGanetiDemoQ & A

Page 9: OSDC 2014: Thomas Schend -  Introduction to Synnefo open source cloud stack

04/10/2023 OSDC 20149

Synnefo

Open SourceProduction-quality IaaSmade for persistent VMscommodity hardware – No SANLarge-scale Installationssimple UI for the end-useruses Ganeti as a Cluster Manager

Page 10: OSDC 2014: Thomas Schend -  Introduction to Synnefo open source cloud stack

04/10/2023 OSDC 201410

Synnefo History

Design started late 2010Production since July 2011~okeanos* public cloud running currently ~8700 VMs

*http://okeanos.grnet.gr

Page 11: OSDC 2014: Thomas Schend -  Introduction to Synnefo open source cloud stack

04/10/2023 OSDC 201411

Synnefo Layers

Image provided by GRNET

Page 12: OSDC 2014: Thomas Schend -  Introduction to Synnefo open source cloud stack

04/10/2023 OSDC 201412

Synnefo birdseye view

Image provided by GRNET

Page 13: OSDC 2014: Thomas Schend -  Introduction to Synnefo open source cloud stack

04/10/2023 OSDC 201413

Synnefo

Build with Ganeti as a backendAstakos: Identity/Account servicesCyclades: Compute/Network/Image/Volume servicesPithos: File/Object Storage servicesnf-manage commandline toolBasic helpdesk toolsnf-burnin / snf-burnin-runkamaki commandline clientCode on github

Page 14: OSDC 2014: Thomas Schend -  Introduction to Synnefo open source cloud stack

04/10/2023 OSDC 201414

Agenda

IntroductionWhat do we need from IaaS?Introducing to SynnefoComponentsGanetiDemoQ & A

Page 15: OSDC 2014: Thomas Schend -  Introduction to Synnefo open source cloud stack

04/10/2023 OSDC 201415

Synnefo Architecture I

Image provided by GRNET

Page 16: OSDC 2014: Thomas Schend -  Introduction to Synnefo open source cloud stack

04/10/2023 OSDC 201416

Synnefo features - Astakos

Identity serviceProvides DashboardTrack Resource usageProject ManagementMultiple Auth methods

Twitter / Google / LinkedIn / LDAP / Active-directory

Page 17: OSDC 2014: Thomas Schend -  Introduction to Synnefo open source cloud stack

04/10/2023 OSDC 201417

Synnefo features - Cyclades

Network managementL3 Routed/L2 VLANS/L2 isolation with mac filternfdhcpd

Archipelago storage abstractionceph/NFS/Gluster backendvery fast, thin cloningSnapshots

Page 18: OSDC 2014: Thomas Schend -  Introduction to Synnefo open source cloud stack

04/10/2023 OSDC 201418

Synnefo features - Pithos

Object storage serviceEfficient syncingHashing before uploadDeduplication in backendWeb UI written in GWTClients for Windows, Macos and IOSSharing public and private

Page 19: OSDC 2014: Thomas Schend -  Introduction to Synnefo open source cloud stack

04/10/2023 OSDC 201419

Synnefo / Ganeti Communication

Image provided by GRNET

Page 20: OSDC 2014: Thomas Schend -  Introduction to Synnefo open source cloud stack

04/10/2023 OSDC 201420

written in python / djangoapache/gunicornpostgres databasesnf-gnt-eventd / snf-progress-monrabbitmqsnf-dispatchersnf-vncauthproxybuild for wheezy, can run fully virtualizedspeaks openstack API but does not share any code

Synnefo Architecture II

Page 21: OSDC 2014: Thomas Schend -  Introduction to Synnefo open source cloud stack

04/10/2023 OSDC 201422

Secure image deployment of untrusted imagesIsolated from the ganeti nodeSupports Linux,Windows and BSD

customization tasksAssign hostnameDelete SSH keysChange passwordResize FilesystemRun unattended install (windows)Inject files

Synnefo - snf-image

Page 22: OSDC 2014: Thomas Schend -  Introduction to Synnefo open source cloud stack

04/10/2023 Central IT Services23

Synnefo snf-image deployment process

gnt-node evacuate -I hail node3

Imag

e pr

ovid

ed b

y G

RNET

Page 23: OSDC 2014: Thomas Schend -  Introduction to Synnefo open source cloud stack

04/10/2023 OSDC 201424

Agenda

IntroductionWhat we need from IaaS?Introducing to SynnefoComponentsGanetiDemoQ & A

Page 24: OSDC 2014: Thomas Schend -  Introduction to Synnefo open source cloud stack

04/10/2023 OSDC 201425

Ganeti History

Very mature, production readyopensourced since August 30, 2007 by googleuses Mature open source tools LVM,DRBD,python,haskellDifferent hypervisors (Xen,KVM,LXC)Low overhead for cluster management

Page 25: OSDC 2014: Thomas Schend -  Introduction to Synnefo open source cloud stack

04/10/2023 OSDC 201426

Ganeti Usage at google

Usage at googlesmall Clusters in Offices (OIB)large Clusters in Datacenters for virtual desktops

mostly using XENNot used for user facing services

Page 26: OSDC 2014: Thomas Schend -  Introduction to Synnefo open source cloud stack

04/10/2023 Central IT Services27

Ganeti features I

Scale from 1-200 nodes and beyondLive migration without shared storageNoderoles: master,master candidate,regular nodesNodegroupsOVF import toolBackup with included toolJob queueCommands start with gnt-

Page 27: OSDC 2014: Thomas Schend -  Introduction to Synnefo open source cloud stack

04/10/2023 OSDC 201428

Ganeti features II

OS Interface to provision new instancessnf-image secure image deploymentganeti-instance-debootstrap, image, ganeti-os-defsHtools to balance clusters, plan capacity and do maintenance

supported file,plain,sharedfile,ceph,NFS,Gluster Ext storage provider with plugins for Archipelago, HP EVA,,IBM Storwize

Page 28: OSDC 2014: Thomas Schend -  Introduction to Synnefo open source cloud stack

04/10/2023 OSDC 201429

Ganeti advanced features

Hooks for every action (pre/post)Tags for every objectExclusion tags to keep machines apart from each otherDeploy flash in every node for high performance (flashcache,bcache,dm-cache)OOB management interfaceGaneti watcher

Page 29: OSDC 2014: Thomas Schend -  Introduction to Synnefo open source cloud stack

04/10/2023 OSDC 201430

Ganeti advantages

No need for special hardware (SAN)But SAN/Shared Storage is supportedManage cluster as an entity not single nodes (libvirt)Few dependenciesNo databaseEasy recovery in case of hardware outageNo SPOF which can affect the reliability of VMsVery good documentation

Page 30: OSDC 2014: Thomas Schend -  Introduction to Synnefo open source cloud stack

04/10/2023 OSDC 201431

Deployment automation puppet,chef,cfengine,..Monitoring icinga/nagios/check_mk/…Openvswitch LACP + 10GBeAutorepairPackages for all major OS debian,ubuntu,rhel/centos

Ganeti best practices

Page 31: OSDC 2014: Thomas Schend -  Introduction to Synnefo open source cloud stack

04/10/2023 OSDC 201432

Ganeti management interfaces

cli with bash completionLUXIRAPI

Different web UIs using the RAPIGaneti web managerGanetimgrSynnefo

Page 32: OSDC 2014: Thomas Schend -  Introduction to Synnefo open source cloud stack

04/10/2023 OSDC 201433

Ganeti quick command reference

gnt-cluster init / info / verify / command / copyfile

gnt-node add / list / failover / evacuate

gnt-instance add / info / replace-disks

gnt-job list …

gnt-network …

gnt-backup …

Page 33: OSDC 2014: Thomas Schend -  Introduction to Synnefo open source cloud stack

04/10/2023 OSDC 201434

Ganeti htools - allocation and placement

written in haskell

hail

hbal

hspace

harep

hinfo

hroller

Page 34: OSDC 2014: Thomas Schend -  Introduction to Synnefo open source cloud stack

04/10/2023 OSDC 201435

Typical Ganeti Clsuter

Node01 Node02 Node03

gnt-cluster init [-s ip] --enabled-hypervisors=kvm clustergnt-node add [-s ip] node2gnt-node add [-s ip] node3gnt-cluster verifygnt-instance add -t drbd -I hail -o debootstrap+default vm

Veea

m v

irtua

lizati

on st

enci

ls

Page 35: OSDC 2014: Thomas Schend -  Introduction to Synnefo open source cloud stack

04/10/2023 OSDC 201436

Ganeti Failover and Repair

gnt-node modify -O yes node01gnt-node failover --ignore-consistency node01gnt-node evacuate -I hail node01

Node01 Node02 Node03

Veea

m v

irtua

lizati

on st

enci

ls

Page 36: OSDC 2014: Thomas Schend -  Introduction to Synnefo open source cloud stack

04/10/2023 OSDC 201437

“Ganeti didn't have any generally known meaning before this project. And now it means kick-ass cluster management”

Martin Beauchamp, on the Ganeti Users List

Quote

Page 37: OSDC 2014: Thomas Schend -  Introduction to Synnefo open source cloud stack

04/10/2023 OSDC 201438

Agenda

IntroductionWhat do we need from IaaS?Introducing to SynnefoComponentsGanetiDemoQ & A

Page 38: OSDC 2014: Thomas Schend -  Introduction to Synnefo open source cloud stack

04/10/2023 OSDC 201439

Login to Astakossyncing and Pithos UI and AppCreate VMs with Cyclades UISnapshot FeatureOOB ConsoleHotplug of NICs and NetworksFloating IPsClean-up

Demo

Page 39: OSDC 2014: Thomas Schend -  Introduction to Synnefo open source cloud stack

04/10/2023 OSDC 201440

Agenda

IntroductionWhat do we need from IaaS?Introducing to SynnefoComponentsGanetiDemoQ & A

Page 40: OSDC 2014: Thomas Schend -  Introduction to Synnefo open source cloud stack

04/10/2023 OSDC 201441

http://www.synnefo.org

https://github.com/grnet/synnefo

https://code.google.com/p/ganeti/

Try it!

Page 41: OSDC 2014: Thomas Schend -  Introduction to Synnefo open source cloud stack

04/10/2023 OSDC 201442

Q & A