Upload
lydang
View
213
Download
0
Embed Size (px)
Citation preview
18/06/2012
1
Cost effective data centre: How uOttawa leveraged a
relocation to deliver a modernized data centre
Joel Breton
Paul Mercier
University of Ottawa 1
Agenda
1. uOttawa Facts
2. Previous Data Centre
3. Relocation Process
4. Challenges
5. New and improved Data Centre
6. Accomplishments
7. Questions?
2
18/06/2012
2
uOttawa Facts
• World’s largest bilingual university (French/English)
• Located in Ottawa, Ontario
• 40,000+ students
• 250 undergraduate programs and 200 graduate programs in 10 faculties
• One of Canada’s top 10 research universities
• Over 5,100 staff members
• 280 IT staff in central IT (CCS) and in faculties and services (50-50 split)
3
Out with the Old, In with the New
4
18/06/2012
3
Previous Floor Plan
5
Some pictures…
6
18/06/2012
4
Power and Cooling• No Hot/Cold aisles concept
• UPS at Maximum Capacity
• No Power resiliency
• No standard way of distributing power
• Old equipment (Liebert, MGE, etc.)
• Weak Documentation on power
distribution (diagrams, labels, etc.)
7
Summer 2010 – It was HOT…
8
18/06/2012
5
Cable Distribution and Rack Location
• Years of technology changes
• No importance was given to the upkeep of the
cabling
• Was preventing good airflow and cooling
distribution
9University of Ottawa
Physical Server Access
• No standards for Remote Access to servers
• Multiple KVM and Console
• Incomplete Management Network
10University of Ottawa
18/06/2012
6
Networking
• Switches random location
• Distinct group for network and storage area
network
• No Central IP Management
• Maximum 1 Gbps
11University of Ottawa
Storage Area Network (SAN)
FastT 700 DS4800 DS5020 DS5100 EMC FCBR
Switch00 Switch01 NT1A NT1B Switch04 Switch05 Switch06 Switch07
BC07 BC07
BC01 BC02 BC04 BC05 Switch02 Switch03 BC06
12University of Ottawa
18/06/2012
7
Storage Area Network (SAN)
Storage Cabinet Server Cabinet
13University of Ottawa
Storage
• Distributed randomly
• Lack of technical expertise, documentation
and standards.
• Unable to retire Legacy
• No structure
• No tiered strategy
14University of Ottawa
18/06/2012
8
Strategy for the New Data Centre
• Address critical weaknesses: power and
cooling, capacity, scalability, sustainability, etc.
• Improve the efficiency and the reliability of
the Data Centre by modernizing the
equipment and by retiring legacy
15
Planning
• Multiple iterations of the floor plan to optimize
space and usage
• Hiring of a dedicated Project Manager
• Documentation and Inventory
• Server Virtualization
• Storage Virtualization
• Network Infrastructure
• Fabric Design & Renewal
16
18/06/2012
9
Relocation Process
• Server Relocation
• Storage Relocation
• Network/Fabric
Relocation
• Mainframe Relocation
• Change Management
• Defined Responsibilities
• Pilot the process
• Control Centre
• Check List
17
Challenges during the move
• Building still under construction
• Organizational issues: roles and staffing,
physical location of staff, multiple re-orgs
before and during the move.
• Operational issues
• Technical issues with fabric/SAN, load
balancers, etc.
• Process issues. Ex: forgot to turn on a server.
18
18/06/2012
10
New Building
19
Floor Plan of New Data Centre
20
18/06/2012
11
Hot Aisle Containment System
(HACS)
21
HACS.MOV
Power and Cooling
• Based on APC InfraStruxure
• Capacity of 500 kW, redundant
• Under-the-floor power distribution
• In-Rack Power resiliency (A/B Design)
• Current UPS Battery Time: ~110 minutes
• In-Row Chilled Water Cooling based on existing Campus
infrastructure, with redundant chillers for the building
• Cooling pipes are hidden under the floor to prevent leak
damages
22
18/06/2012
12
Top of Rack and Middle of Row
Cable Distribution
23University of Ottawa
Patch Panels
24
18/06/2012
13
Server Access
• IP KVMs to replace all local and KVM consoles
• End or Row Console Access
• Remote Access for geographically dispersed
staff
25University of Ottawa
Servers• Virtualization of servers
• Increased density of racks and servers
• FCoE implementation for virtual hosts
0
200
400
600
800
2008 2009 2010 2011 2012
Number of Virtual Servers
26University of Ottawa
18/06/2012
14
Networking
27University of Ottawa
Improved SAN
Edge Edge
Core Core
Edge Edge Edge Edge Edge Edge
DS4800
ServerA ServerB ServerC
DS5020 DS5100 DS5300
28University of Ottawa
18/06/2012
15
Storage Improvement
• Storage Physical Layout
• Storage Virtualization
– Thin provisioning
– Legacy retirement
– Storage tiers
• Storage growth
29University of Ottawa
Accomplishments
• More then 15,000 hours of labour
• Over three years for the planning, building of the
new data centre and execution of the move
• Documentation of existing infrastructure
– Labelling of all assets, including cables
– CMDB in service-now
• Cable Distribution
– 6 km of fibre
– 6 km of ethernet
30University of Ottawa
18/06/2012
16
Some Pictures
31
Some Pictures
32
18/06/2012
17
Overall: Success
• Energy Recycling: up to 80% of the heat generated by the
equipment is recuperated to heat up the building during
Winter months
• Energy Savings (power and cooling): from spending $700 per
year for a physical server to $20 for a virtual one
• Increased efficiency in the use of electricity
• Reduced data centre floor space
• Increased efficiency of data centre assets
• Standardized Infrastructure Architecture
33University of Ottawa
Lessons Learned
• Start virtualization as early as possible and virtualize
everything you can
• Start small, for testing
• Ensure Commitment throughout the organization
• Communication
• Involve all stakeholders early in the process
• Some things can and will go wrong. Ex: switch failure
• Minimize changes happening during the move.
34
18/06/2012
18
Next Challenges
• Foundation for the Cloud
• Disaster Recovery & Business continuity
• Co-location option
35
Questions?
Joel Breton
(613)562-5800 x4192
Paul Mercier
(613)562-5800 x3947
University of Ottawa 36