Who Hit The Turbo Button?

February 13, 2013

Presenter: Stephen Watson

ITS System Manager

Stephen F. Austin State University

Stephen F. Austin State University

Stephen WatsonSystems Manager ITS Previous jobs at SFASU

(Programmer Analyst in Student/HR/Budget)(Systems Programmer I and II)

18 Years at SFASU

Gabriel NguyenSr. Sales Consultant Oracle Corporation

Agenda

How SFA went from sluggish to turbo performance...

Introduction

Challenges

Architecture Selection Process

Exadata Implementation Process

Performance Results

Where We Are Today

Q and A

Stephen F. Austin State University

Academic and Research University

Undergraduate, Graduate and Doctoral

Enrollment of 13,000

Both Traditional and Online Courses

SFA Support Staff

3 DBA’s

7 Systems

11 Programming

Current Environment

Ellucian Banner 8

Student

Human Resources

Finance

Financial Aid

WebFOCUS Reporting

UC4 AppManager

Oracle 11g R2 DB and 11gR2 WLS/FMW

Why Did We Need A New DB Platform?

Registration Performance Issues

Future DB Platform Support from Oracle

and Ellucian

Problem 1: Registration Performance

2008 Banner 8 Go-Live

Registration load issues during orientations

and pre-registration periods

Limited to 250-300 simultaneous users

Had to break registration in very small

groups (<500)

Student Experience Impact

Problem 1: Registration Performance

University reputation (bad press, social media…)

Student satisfaction declined

Negatively impacted enrollment

Negative comments from parents about experience

Peripheral Impact

Making the News

Problem 2: Product End of Life

HP de-support Open VMS

Ellucian de-support of Open VMS

Oracle de-support of Itanium platform

HPUX/11G would be de-supported soon

soon as well

Technology Impact

Environment Before Exadata

HP Itanium rx6600 Running Open VMS 8.3 and

Oracle 10G R2 (3 identical servers)

Banner Production on one server

Banner Test instances (4) on second server

Banner Prod DR on third server, ready to come

up when needed

No high-availability, data protection, or

disaster recovery capabilities

Hardware Refresh Time

We could run HPUX/11G on the servers but this would be de-supported soon as well

After getting another year from the hardware we would have to buy new hardware

Why we selected Exadata

Investigation and research Oracle EX-CITE conducted on campus Key advantages

Fast transaction processing Extreme performance and redundant appliance type

architecture One throat to choke (so to speak)

One process to patch everything (OS, switches, storage and database homes)

Lots of storage capacity and processing power

for future growth

What is Exadata?

Oracle’s Engineered Database Machine Pre-configured, pre-integrated and highly efficient combination of hardware and software to provide a complete solution to the customer

Optimized for OLTP and DW database workloads

The Exadata Quarter “Server” or Rack containsMultiple database nodes (2)Multiple Cell or storage servers (3)Shared storage (12 3 TB disk per cell )All connected with a fast Infinband network (40 GB/sec)Designed to work as a unit to increase transaction performanceLots of RAID redundant storage. (108 TB raw )ASR Server monitors hardware failures continually

Solution: SFA Exadata Platform

X2-2 Quarter Rack• 2 Sun Fire x4170 DB Servers• 3 Exadata Storage Servers (SAS)• Raw Disk – 108TB (SAS Drives)• Raw Flash – 1.2TB• User Data – 40TB ( No Compression)ZFS

Backup Area – 20 TB

ProdX2-2 Quarter Rack

DR/Dev/Test/QAX2-2 Quarter Rack

Active Data Guard

(physical standby)

• 10g to 11g migration • RAC• 3 databases, 6

Instances• 700 GB size

• Stand-by PROD databases

• 5 databases, 10 Instances

• 3.0 TB size

OEM Grid Control

Grid ControlConsole

GigE Backup Network

ZFS Storage Appliance•Backups of database •Export dumps

Data Center A – Site 1 Data Center B – Site 2

Bottom Line

SFA needed a solution that “just works”

SFA enrollment growing – need a solution that will

keep up with demands of administration and students

Opportunity for IT consolidation to lower costs

Avoid further loss of confidence from faculty/staff

in our IT management and staff

Bottom Line

Loss of confidence from the students in SFA being

able to deliver on their mission promise: “...comprehensive

institution dedicated to excellence in teaching, research,

scholarship, creative work, and service”

To effectively recruit quality students in the future and

confidence from parents in SFA being the right university

for their family members

Project Timeline

A very rapid deployment

• DB Export Started on 4/5/2012 at 5:00 pm• DB import into Exadata Completed 4/7/2012• Testing and Data Validation 4/7 thru 4/8/2012• Go live Sunday 4/8/2012 at 7:00 pm

Summary of Implementation

Delivery to go live one of the shortest I have ever

been involved with Lots of long hours including evenings and

weekends (several 80-90 hour weeks) Oracle could not believe we implemented in this

short time frame Usual implementation of Exadata

(6 months to 1 year)

Ellucian and Oracle consulting were crucial

Where we are today

2 successful large registrations, fall and spring

5 successful orientations (registration)

Returned to larger groups for registration Classifications (about 3000 per group)

All Processes running faster

Better faculty/staff experience in Banner

Better student experience with

registration

Where we are today (continued)

Administration very pleased with the success of the

registration process

Strategically positioned for future IT consolidation of

databases on campus

No complaints from students

Benefits

Time to Value Return on Investment Performance

Before:Times

After:Times

Performance Improvement

Query run timeReporting

1 to 3 minutes 10 to 30 seconds 6 fold increase

AR ProcessesOther Processes

5 Min to 3 Hours 5 Sec to 10 min 60 fold increase

Backup Times Full Hot Backups4 Hours

Full Hot Backups2 hours

Time cut in half

Registration Less than 200 concurrent

Up to 1000 concurrent

5 Fold increase

Highlights

One Vendor for OS and DB issues

Oracle patches everything (OS, DB, Switches and

Storage)

More disk space than we know what to do with

FLASH CACHE storage on disk helps performance

Highlights

Lots of room for future growth

No system tweaking - delivered for performance

Faster backups

Faster cloning

Questions?

For additional information contact:

Stephen Watson

IT System Manager

Information Technology Services

swatson@sfasu.edu