Steve Peck Service Desk Manager Global Research NOC At Indiana University

Steve PeckService Desk Manager

Global Research NOC

At Indiana University

Greetings to the MAX Community !!!

The Global Research NOC at IndianaUniversity, in conjunction with the staffof the MAX, is here to provide you with

excellent

SERVICE&

SUPPORT!!!

Service Start-Up

This coming Monday June 29, 2009

Final stage of “BETA” support, where the GRNOC assumes full front-line support for the MAX network.

Support aspects are ready, putting the finishing touches on tools, etc.

GRNOC Supported Projects

OmniPoP

GRNOC Supported Projects

AMPATH

REN-ISAC

Open Science Grid (OSG)

Service DeskPerforms front-line network managementProvides customer interaction and service Provides service interface to engineeringStaffed by network techniciansStaff are network knowledgeable; they

understand network topology, cause & effect, and management processes

Operates 24 x 7 x 365Fail-safe operationsSingle point of contact for network

managementLeveraged resources for cost-effective

operations and management

Service Desk

Manager Steve Peck Supervisors Nizar Louly

Stacy Bengochea

Adam Williamson

Matt Robertson Front Line Staff 18 technicians Operations John Wilson &

Support Team 3 others

Service Desk

MAX Specialized Support Technician (SST): Terry Broadstreet

MAX NOC Contact Information:Email: noc@maxgigapop.net

Phone: (301) 314-6666

Web form to open a ticket: (coming very soon!!!)

Engineering GRNOC Network Engineering Team

provides Tier 2 support. Receives tickets via round-robin assignment, and provides 24x7 on-call service. Staff of 8, managed by Jay Duncan.

GRNOC Network Planning Team (NPT) provides Tier 3 engineering support.

Jeff Ambern is MAX NPT Assisted by Tom Johnson 6 other NPT engineers on staff

Engineering MAX Engineering team provides:

Tier 4 escalation supportNetwork designSchedules maintenance activityNew participant & peer

connectionsHands & Eyes support

Problem Management Cycle

AlertPro-active monitoringProblem reported by the MAX community

Problem verification & assessmentWHO is affected???

Trouble ticket creation Member or peer is contacted

Priority of ticket set in accordance with our Customer Impact Guidelines

Problem Management Cycle

Member & Peer NotificationSubject: MAX GigaPOP Layer 3 Core Node CLPK Maintenance Issue=127

SUBJECT: MAX GigaPOP Layer 3 Core Node CLPK Maintenance

AFFECTED: Layer 3 Core Node CLPK and its associated Peers and Participants

START TIME: Tuesday, June 25, 2009, 7:00 AM (0700) UTC (3:00 AM EDT)

END TIME: Tuesday, June 25, 2009, 8:00 AM (1000) UTC (6:00 AM EDT)

DESCRIPTION: Layer 3 Core Node CLPK will be unavailable while MAX Engineers perform a routing engine swap on the CLPK-T640. A brief outage is expected; however, the entire window is reserved.

TICKET NO.: 127:112

TIMESTAMP: 09-06-23 14:42:25 UTC

Problem Management Cycle

Ticket assignment and escalationGRNOC Network EngineeringVendor contact & ticket/case openedGRNOC Network Planning TeamMAX Engineering

Member and peer notification updates continue until…

PROBLEM RESOLUTION!!! Customer contact satisfaction

Change Management

Hardware and Software maintenance is scheduled with one week advance notice. Notification will be sent to the affected MAX community accordingly.

Emergency maintenance is performed as needed, for the HEALTH of the network. Notification is sent with as much advance notice as possible.

Customer Impact Guidelines

Customer Impact is a subjective scale of measure used to quantify the current impact of a problem or maintenance on the customer’s operations, performance, and usability.  Set in conjunction with the customer, it may change as the ticket progresses and is used for ticket escalation.  Customer Impact seeks to answer, “How high of a priority is the problem/maintenance to the customer?”   

Customer Impact Guidelines

1 - CRITICAL    A problem or issue for which the customer

needs immediate, undivided attention from NOC staff until resolved.

The customer is expected to be available immediately to commit full-time resources until the situation is resolved.

The NOC uses this by default when the network is monitored to have an outage of a non-redundant core network element.

Customer Impact Guidelines

2 - HIGH A problem or issue for which the customer

needs resolution within 1 business day. The customer is expected to commit

resources to resolve the situation between the hours of 1300 and 0100 UTC (1200 and 0000 UTC when Daylight Saving Time is in effect).

The NOC uses this by default when the network is monitored to have an outage of a redundant core network element.

Customer Impact Guidelines

3 - ELEVATED A problem or issue for which the customer does not

need immediate resolution, but needs NOC attention within 3 business days.

The customer is expected to be available to provide information or assistance when available during normal business hours.

The NOC uses this by default when a customer connection or session is monitored to have a problem or outage. This is also used by default for maintenance which is both NOC initiated and customer impacting.

Customer Impact Guidelines

4 - NORMAL  No impact to the customer’s operations,

performance, and usability  Non-urgent customer service requests Routine installation/provisioning tickets,

non-customer impacting maintenance, and customer initiated maintenance.

Systems Engineering

Provides all the wonderful systems and tools that the NOC uses, as well as tools that are available to the MAX community that are available on the MAX NOC homepage: http://maxnoc.grnoc.iu.edu/

(web pages coming very soon!!!) Staff of 12, managed by Luke Fowler

TOOLS: Operations Calendar

TOOLS: Ticket Viewer

TOOLS: Weekly Network Report

TOOLS: SNAPPSMNP Network Analysis and Presentation

Package

TOOLS: Real Time Atlas Map

TOOLS: Global NOC Database

The Global NOC Database containscomprehensive information regarding network elements, operations and problem management information for elements, network topology, and contacts.

Drill down linkages to the Database information has been developed for front-end tools such as Footprints and AlertMon.

TOOLS: Global NOC Database

TOOLS: AlertMon

The AlertMon front-end to monitoring systems (Nagios) consolidates alerts from network monitoring systems, provides an alert management interface, and links to ticketing and documentation for alerted elements. AlertMon receives input from Nagios and the MAX Database.

TOOLS: AlertMon

TOOLS: Footprints Ticketing

TOOLS: TickMon

Reminders !!!

Noc@maxgigapop.net is being monitored and answered by the MAX NOC Service Desk, not the MAX engineers.

Please respond to our contact information update requests. Very important!

We are still learning, but are 100% committed to doing our best!

Contact the MAX NOC

Email: noc@maxgigapop.net

Phone: (301) 314-6666

Web form to open a ticket: (coming very soon!!!)

MAX NOC Web Pages (coming very soon!!!)

http://maxnoc.grnoc.iu.edu/

Questions or Comments???

Steve Peck

speck@iupui.edu

(317) 274-4968

http://globalnoc.iu.edu/