Upload
trinhkien
View
215
Download
0
Embed Size (px)
Citation preview
HU-Stack: Eine cloud-basierte Plattform als flexible Service-Infrastruktur
Malte Dreyer
Profil / Struktur
Computer- und Medienservice (CMS)
Verwaltung (zentral)
• Ressourcen-Mgmt.• Campus-Mgmt.• Verwaltungs PCs
Hardware / Software• Endgeräte (~3.000)• SW-Lizenz-Mgmt.• Benutzerberatung
Digitale Medien• Medien-Mgmt.• Lernumgebungen• Audio-/Video-Technik
Basis-Infrastruktur• Server• Storage• Netze
„Nachwuchsgruppen“„Nachwuchsgruppen“
Profil / Struktur
Computer- und Medienservice (CMS)
Verwaltung (zentral)
• Ressourcen-Mgmt.• Campus-Mgmt.• Verwaltungs PCs
Hardware / Software• Endgeräte (~3.000)• SW-Lizenz-Mgmt.• Benutzerberatung
Digitale Medien• Medien-Mgmt.• Lernumgebungen• Audio-/Video-Technik
Basis-Infrastruktur• Server• Storage• Netze
„Nachwuchsgruppen“„Nachwuchsgruppen“„Nachwuchsgruppen“
Projekte und Plattformen(auch Forschungsdatenmanagement)
AG ePublishing(gemeinsam mit UB)
Basisinfrastruktur
10.09.2013
Computer- und Medienservice4
Zusammenarbeit mit den Einrichtungen
Servicetiefe des CMS
Plattformen
Dienste
Basis-Infrastruktur
Arbeitsplätze
Computer-und
Medien-Service
Computer-und
Medien-Service
Inst. 2
Computer-und
Medien-Service
Inst. 3
Computer-und
Medien-Service
ZentraleVerwaltung
Inst. 1
Breites Diensteportfolio
Identity Management Service
Image Editing
Mail Service
Mailing Lists
Map Service
Moodle
Printing Service
Public Computers
Publication Repository
Resource Management SW
Scanning Service
Shibboleth
Software Service
Statistics SW Service
Survey
Terminalserver Service Service
Professional Training
Trouble Ticket Systems
User Certificates
End User Support
Virtual Private Networks
Video Conference Service
Video Service
WLAN
Web Content Management
Web Files
Web Hosting
Website Management
Wiki and Blog Farms
Windows Domains
ePublishing
ADS
AFS
Accounts
Anti Virus Software
Audio / Video Equipment
Backup-Service
Calendar Service
Campus Management SW
Chat Service
Compute Clusters
Database Service
Managed Desktops
File Exchange
File Service
Graphics
Hardware Service
Breites Diensteportfolio
…aber noch nicht breit genug -> Forschungsdaten
Identity Management Service
Image Editing
Mail Service
Mailing Lists
Map Service
Moodle
Printing Service
Public Computers
Publication Repository
Resource Management SW
Scanning Service
Shibboleth
Software Service
Statistics SW Service
Survey
Terminalserver Service Service
Professional Training
Trouble Ticket Systems
User Certificates
End User Support
Virtual Private Networks
Video Conference Service
Video Service
WLAN
Web Content Management
Web Files
Web Hosting
Website Management
Wiki and Blog Farms
Windows Domains
ePublishing
ADS
AFS
Accounts
Anti Virus Software
Audio / Video Equipment
Backup-Service
Calendar Service
Campus Management SW
Chat Service
Compute Clusters
Database Service
Managed Desktops
File Exchange
File Service
Graphics
Hardware Service
Biographical Sketch(es), has been revised to rename the “Publications”
section to “Products” and amend terminology and instructions accordingly.
(c) Products
A list of: (i) up to five products most closely related to the proposed
project; and (ii) up to five other significant products, whether or not
related to the proposed project. Acceptable products must be citable
and accessible including but not limited to publications, data sets,
software, patents, and copyrights. Unacceptable products are
unpublished documents not yet submitted for publication, invited
lectures, and additional lists of products. Only the list of 10 will be used
in the review of the proposal.
Each product must include full citation information including (where
applicable and practicable) names of all authors, date of publication or
release, title, title of enclosing work such as journal or book, volume,
issue, pages, website and Uniform Resource Locator (URL) or other
Persistent Identifier.
National Science Foundation (NSF)
Biographical Sketch(es)
NSF 13-1 January 2013
http://www.nsf.gov/pubs/policydocs/pappguide/nsf13001/gpg_2.jsp#IIC2d
1. Recommended open access to scholarly papers of
publicly funded research
2. Recommended open access to all digital outputs of
publicly funded research
3. Mandated open access to scholarly papers of publicly
funded research
4. Mandated open access to all digital outputs of
publicly funded research
5. Enforced, mandated open access to scholarly papers
of publicly funded research
6. Enforced, mandated open access to all digital
outputs of publicly funded research
By Muriel (originally posted to Flickr as Muon Barrel) [CC-BY-SA-2.0 (http://creativecommons.org/licenses/by-
sa/2.0)], via Wikimedia Commons
By Hannes Grobe (Hgrobe 06:16, 26 April 2006 (UTC)), Alfred Wegener Institute for Polar and Marine Research
(Own work) [CC-BY-SA-2.5 (http://creativecommons.org/licenses/by-sa/2.5)], via Wikimedia Commons
Small Data
„Long Tail of Research Data“
Source: Ferguson et al. (2014): Big data from small data: data-sharing in the
'long tail' of neuroscience. DOI: 10.1038/nn.3838
Science 11 February 2011: Vol. 331 no. 6018 pp. 692-693 DOI: 10.1126/science.331.6018.692
The 2011 survey by Science, found that 48.3% of respondents were working with datasets that were less than 1GB in size and over half of those polled store their data only in their laboratories.
Simukovic, Elena; Kindling, Maxi; Schirmbacher, Peter (2013): Umfrage zum Umgang mit digitalen Forschungsdaten an der
Humboldt-Universität zu Berlin. Umfragebericht, Version 1.0. URN: urn:nbn:de:kobv:11-100213001n=457
https://www.cms.hu-berlin.de/de/ueberblick/projekte/dataman/home
Anpassungen und Abbildung
von Szenarien
z. B. Medien-Repositorien-Farm
Gemeinsame Projekte und
Software-Entwicklung
z. B. Laudatio Corpus Repositorium
Projekte in enger Zusammenarbeit mit den Instituten
(gemeinsame Drittmitteleinwerbung)
80 Individuelle Projekte
80 Installationen
80 Datenbanken
80 Konfigurationen
Individuelle Software
Fedora Repository
CakePHP
Elastic Search
PID-Server (Handle)
jQuery / Bootstrap
Apache Xerces
+RedMine for Tickets
Aus Sicht des Datenzentrums
... Nutzer erwarten dauerhaften Betrieb …
80 Individual Projets
80 Installations
80 Databases
80 Configurations
Individual Software
Fedora Repository
CakePHP
Elastic Search
PID-Server (Handle)
jQuery / Bootstrap
Apache Xerces
+RedMine for Tickets
Aus Sicht des Datenzentrums
... Pflegeaufwand …
Änderungen und Erweiterungen
(Maßkonfektion)
Überwiegende Eigenentwicklung(Maßgeschneidert)
Selbstadministration / Selbstbedienung
By Paul Hudson from United Kingdom (11" MacBook Air) [CC BY 2.0
(http://creativecommons.org/licenses/by/2.0)], via Wikimedia Commons
By HLundgaard (Own work) [CC BY-SA 3.0
(http://creativecommons.org/licenses/by-sa/3.0)], via Wikimedia Commons
By Sae1962 (Own work) [CC-BY-SA-3.0 (http://creativecommons.org/licenses/by-sa/3.0)], via Wikimedia Commons
Flexibilisierung der Infrastruktur
XaaS „Anything as a Service“
Selbstbedienungsfunktionen (cloud)
IaaSInfrastructure
PaaSPlatforms
SaaSSoftware /
Applications
HardwareServer / Storage /
Network
HardwareServer / Storage /
Network
HardwareServer / Storage /
Network
IaaSInfrastructure
IaaSInfrastructure
PaaSPlatforms
Cloud computing is a model for enabling ubiquitous,
convenient, on-demand network access to a shared pool of configurable computing resources (e.g., networks, servers, storage, applications, and services) that can be rapidly provisioned and released withminimal management effort or service provider interaction.
The NIST Definition of Cloud Computing
(US-National Institute of Standards and Technology SP 800-145)
Automatisierung /
Mehr Softwareentwicklung und –Technik
By Mixabest [CC BY-SA 3.0 (http://creativecommons.org/licenses/by-
sa/3.0), via Wikimedia Commons
Das Virtuelle Rechenzentrum
HEAT
my_port01:
type: OS::Neutron::Port
properties:
admin_state_up: true
name: Port01
network_id: { get_resource: my_first_network }
my_server07:
type: OS::Nova::Server
properties:
name: seaf07
key_name: { get_param: keypair_name }
flavor: { get_param: machine_flavor }
image: { get_param: image_id }
networks: [{ "port": { get_resource: my_port07 }, …}]
user_data: |
#!/bin/bash -v
….
cinder_volume07:
type: OS::Cinder::Volume
properties:
size: { get_param: volume_size }
volume_attachment07:
type: OS::Cinder::VolumeAttachment
properties:
volume_id: { get_resource: cinder_volume07 }
instance_uuid: { get_resource: my_server07 }
mountpoint: /dev/vdb
ANSIBLE Playbook
# update packet cache
- hosts: all
tasks:
- apt:
- name: apply common configuration to all nodes
hosts: all
roles:
- common
- name: deploy and configure memcached server
hosts: memcached
roles:
- memcached
- name: deploy MySQL and configure the databases
hosts: db
roles:
- db
- name: deploy seafile and apply configuration
hosts: nodes
roles:
- nodes
- name: deploy loadbalancer
hosts: loadbalancer
roles:
- loadbalancer
tags:
- loadbalancer
Computer- und Medienservice
Computer- und Medienservice
Horizon
Dashboard
Nova
Compute
Neutron
Network
Swift
Object Storage
Glance
Image Mgmt
Heat
Orchestration
Ceilometer
Metering
Keystone
Identity Mgmt
Cinder
Block Storage
Trove
Database Mgmt
Sahara
Hadoop
OpenStack Komponenten
• Preise [TEUR]
2016 2017 2018
WDM (Licht) 200 200
Linecards 10 Gbps x20
200
MLX (Router) Chassis 8x
175 175
Linecards(26) 10Gbps x8
286 286
Summe 661 661 200
Horizon
Dashboard
Nova
Compute
Neutron
Network
Swift
Object Storage
Glance
Image Mgmt
Heat
Orchestration
Ceilometer
Metering
Keystone
Identity Mgmt
Cinder
Block Storage
Trove
Database Mgmt
Sahara
Hadoop
Technische Komponenten der Cloud / Organisatorische Einbettung
Horizon
Dashboard
Nova
Compute
Neutron
Network
Swift
Object Storage
Glance
Image Mgmt
Heat
Orchestration
Ceilometer
Metering
Keystone
Identity Mgmt
Cinder
Block Storage
Trove
Database Mgmt
Sahara
Hadoop
Technische Komponenten der Cloud / Organisatorische Einbettung
Horizon
Dashboard
Nova
Compute
Neutron
Network
Swift
Object Storage
Glance
Image Mgmt
Heat
Orchestration
Ceilometer
Metering
Keystone
Identity Mgmt
Cinder
Block Storage
Trove
Database Mgmt
Sahara
Hadoop
Technische Komponenten der Cloud / Organisatorische Einbettung
Horizon
Dashboard
Nova
Compute
Neutron
Network
Swift
Object Storage
Glance
Image Mgmt
Heat
Orchestration
Ceilometer
Metering
Keystone
Identity Mgmt
Cinder
Block Storage
Trove
Database Mgmt
Sahara
Hadoop
Technische Komponenten der Cloud / Organisatorische Einbettung
Horizon
Dashboard
Nova
Compute
Neutron
Network
Swift
Object Storage
Glance
Image Mgmt
Heat
Orchestration
Ceilometer
Metering
Keystone
Identity Mgmt
Cinder
Block Storage
Trove
Database Mgmt
Sahara
Hadoop
Technische Komponenten der Cloud / Organisatorische Einbettung
Horizon
Dashboard
Nova
Compute
Neutron
Network
Swift
Object Storage
Glance
Image Mgmt
Heat
Orchestration
Ceilometer
Metering
Keystone
Identity Mgmt
Cinder
Block Storage
Trove
Database Mgmt
Sahara
Hadoop
Technische Komponenten der Cloud / Organisatorische Einbettung
Horizon
Dashboard
Nova
Compute
Neutron
Network
Swift
Object Storage
Glance
Image Mgmt
Heat
Orchestration
Ceilometer
Metering
Keystone
Identity Mgmt
Cinder
Block Storage
Trove
Database Mgmt
Sahara
Hadoop
Technische Komponenten der Cloud / Organisatorische Einbettung
Horizon
Dashboard
Nova
Compute
Neutron
Network
Swift
Object Storage
Glance
Image Mgmt
Heat
Orchestration
Ceilometer
Metering
Keystone
Identity Mgmt
Cinder
Block Storage
Trove
Database Mgmt
Sahara
Hadoop
Technische Komponenten der Cloud / Organisatorische Einbettung
Mein Virtuelles Rechenzentrum
ASAP
HEAT
my_port01:
type: OS::Neutron::Port
properties:
admin_state_up: true
name: Port01
network_id: { get_resource: my_first_network }
my_server07:
type: OS::Nova::Server
properties:
name: seaf07
key_name: { get_param: keypair_name }
flavor: { get_param: machine_flavor }
image: { get_param: image_id }
networks: [{ "port": { get_resource: my_port07 }, …}]
user_data: |
#!/bin/bash -v
….
cinder_volume07:
type: OS::Cinder::Volume
properties:
size: { get_param: volume_size }
volume_attachment07:
type: OS::Cinder::VolumeAttachment
properties:
volume_id: { get_resource: cinder_volume07 }
instance_uuid: { get_resource: my_server07 }
mountpoint: /dev/vdb
ANSIBLE Playbook
# update packet cache
- hosts: all
tasks:
- apt:
- name: apply common configuration to all nodes
hosts: all
roles:
- common
- name: deploy and configure memcached server
hosts: memcached
roles:
- memcached
- name: deploy MySQL and configure the databases
hosts: db
roles:
- db
- name: deploy seafile and apply configuration
hosts: nodes
roles:
- nodes
- name: deploy loadbalancer
hosts: loadbalancer
roles:
- loadbalancer
tags:
- loadbalancer
..…Danke…..
Fragen?
Computer- und Medienservice
Backup