Embed Size (px)
DESCRIPTION
Hokkaido university academic cloud which started services in 2011, is the largest academic cloud system in Japan. Its peak performance is 43TFlops and HPC/Hadoop cluster instances can be deployed automatically.
Citation preview
Hokkaido University Academic Cloud: Largest Academic Cloud System in Japan
Masaharu Munetomo
Information Initiative Center,Hokkaido University, Sapporo, JAPAN.
Cloud Technical Leadership Forum @San Francisco Intercontinental Hotel
Apr. 18th, 2012
Information Initiative Center, Hokkaido University
• A key institute of “high performance computing infrastructure” (HPCI) in Japan.
• Founded in 1962 as a national supercomputing center.
• University R&D center for Supercomputing, Cloud computing, Networking, IT systems for education
• Supercomputer (172TFlops, #95 in TOP500) & Cloud System (43TFlops)
@�(�*��[�( %!*��.9����
�7W1)�'*�%1�L�XN��"*��2G [@�(�*
��89=)+Y=)H4:<��"*���$*(�#�([,P=)
ZCQ=��"*���&(��#�(��B?������
;D�7S>3�RN EI�V��[57J������MO?A�/
-�Q0K�UF�@�(�*�6�T�����
���������� �
������
Hokkaido University Academic Cloud
• Largest Academic Cloud System in Japan: 43TFlops
• More than 2,000 VMs can be deployed
• High-performance cloud system: each physical node has 40-cores, 128GB memory. Network: 10GbE x 2, Shared Storage: 760TB
Hitach BladeSymphony BS2000Xeon E7 8870 2.4GHz (10-core) x 4
128GB memory / 10GbE x 2
Hitach NAS StorageAMS2300: 260TBAMS2500: 500TB
An overview of software architecture
@�(�*��[�( %!*��.9����
�7W1)�'*�%1�L�XN��"*��2G [@�(�*
��89=)+Y=)H4:<��"*���$*(�#�([,P=)
ZCQ=��"*���&(��#�(��B?������
;D�7S>3�RN EI�V��[57J������MO?A�/
-�Q0K�UF�@�(�*�6�T�����
���������� �
������
HW��Boot/VM Image: SAN�Data: NAS��
RedHat/CentOS� Hypervisor ( XenServer / VMWare)�
Cloud&Middleware&(CloudStack)�
Portal&system&(In9house&developed��Univ.&SSO&
CloudStack&API access�
Why we employ CloudStack?
Why we employ CloudStack?
Why we employ CloudStack?
• High quality and usability in user portal and management console
Why we employ CloudStack?
• High quality and usability in user portal and management console
• Easy to control resources via API
Why we employ CloudStack?
• High quality and usability in user portal and management console
• Easy to control resources via API
• Open source
VM Hosting Services: Hosting & Project servers
• Hosting servers: for web-hosting, etc. including CMS middleware packages (WordPress, MediaWiki, etc.)
• Project servers: for research projects
• Cluster packages are also available: Hadoop/MPI
Class Cores Memory Storage JPY/monthS 1 3GB 100GB 525M 4 12GB 100GB 2,100L 10 30GB 100GB 5,250XL 40 128GB 2TB 21,000
In-house developed cloud portal
Selection of service-level
Selection of packages
# of VMs for cluster
Authentication Config.
FW initial Config.
Network configuration
• Assign a VLAN to each user (laboratory)
Physical Node #1 Physical Node＃２
�������������
�������������
Virtual Router
for user A��������
eth0%
eth0%
eth0.200 eth0.201
������
eth0%
eth0.800
eth0%
eth0.200 eth0.201
Virtual Router
for user B�
eth1% eth0%
������
eth0%
�������
eth0%eth1% eth0%
VLAN+200!VLAN+800! VLAN+201! VLAN+200!VLAN+201!
eth0.800
VLAN+800!
Virtual%Interface�
Bridge�
Send%with%VLAN%tag�
�%Private%Network%for%User%A��%Private%Network%for%User%B�
�%Global%Network�
Virtual%NIC�
Automated Deployment of VM clusters
• Customizing scheduling policies in CloudStack to balance I/O overheads for cluster packages (Hadoop / MPI / Torque).
Storage #3�Virtual(Disk�
Storage #4�Virtual(Disk�
Storage #2�Virtual(Disk�
Zone!POD!
Shared Storage #1�Resource Pool #1�
HyperVisor #2�
HyperVisor #1�Virtual(Disk�VM(
Balancing!overheads!of!disk!I/O!with!round8robin!assignment!of!Virtual!disks.!�
Storage #1�
VM(
VM(
VM(
VM(
Virtual(Disk�Hadoop Cluster�Shared Storage #2�
Resource Pool #2�
HyperVisor #4�
HyperVisor #3�
Virtual(Disk�
VM(
Shared Storage #3�Resouce Pool #3�
HyperVisor #6�
HyperVisor #5�
Virtual((Disk�
VM(
Shared Storage #4�
Resouce Pool #4�
HyperVisor #8�
HyperVisor #7�
Virtual(Disk�
VM(
Shared Storage #1�Resource Pool #1�
HyperVisor #2�
HyperVisor #1�Virtual(Disk�VM(
Shared Storage #2�Resource Pool #2�
HyperVisor #4�
HyperVisor #3�
Virtual(Disk�
VM(
Shared Storage #3�Resouce Pool #3�
HyperVisor #6�
HyperVisor #5�
Virtual((Disk�
VM(
Shared Storage #4�
Resouce Pool #4�
HyperVisor #8�
HyperVisor #7�
Virtual(Disk�
VM(
CloudStack+(ManagementServer)+
CloudStack+DB+
Planner+
Storage+load+balancing�VM+Alloca<on�
Planner+
Request�
VM+Deployment+Job�
Deployment(Informa7on��Host��Storage����
Deployment(request�
Deploy(VMs�
CloudStack+Portal�
Deployment time for clusters
• Fast deployment: 5 min. for 1 VM, around 10 min. for 20 VMs
• A medium size cluster consistingof 257 VMs can be deployedin a little more than 1 hour !
• It greatly speedup startingresearch projects sinceconventional physicalcluster needs more thana week or even a monthto finish deployment includingpurchase process.
y"="0.2267x"+"4.1748"R²"="0.9989�
0"
10"
20"
30"
40"
50"
60"
70"
0" 50" 100" 150" 200" 250" 300"
Deploymen
t*+me*(m
in)�
#*of*VMs�
Heterogeneous hybrid cloud deployment test
• CloudStack@Hokkaido University - OpenNebula@TokyoTech
We developed a VPN client to establish connection via API
（Python + OpenSwan + l2tpd）
Gateway�Management Server�
Computing Node�
Gateway�
Virtual Router�
VPN Client�
VPN Server�
CloudStackVirtual Route VPN
SHINCLOM (Simple Heterogeneous INter-CLOud Manager): Inter-cloud manager for academic cloud systems
SHINCLOM (Simple Heterogeneous INter-CLOud Manager): Inter-cloud manager for academic cloud systems
SHINCLOM (Simple Heterogeneous INter-CLOud Manager): Inter-cloud manager for academic cloud systems
