An Instrumentation and Measurement Architecture Supporting Multiple Control

Monitoring Frameworks

XXX Brazilian Symposium on Computer Networks and Distributed SystemsIII Workshop de Pesquisa Experimental da Internet do Futuro (WPEIF 2012)May 4th, 2012

Authors

Marcelo M. Pinheiro14, Igor L. E. Macêdo1, Igor L. O. Souza1, Thiago S. Hohlenweger13, Paulo R. R. Leite1, Adriano L.

Spínola1, Herbert Monteiro1, Raphael A. Dourado2, Leobino N. Sampaio4, José A. Suruagy Monteiro2 and Joberto S. B.

Martins1

 1Salvador University (UNIFACS) 2Federal University of Pernambuco (UFPE)

3Federal Institute of Bahia (IFBA) 4Federal University of Bahia (UFBA)

• FIBRE-BR will possibly use three different control and monitoring frameworks in its nine islands– OFELIA Control Framework– cOntrol and Management Framework (OMF) and– ProtoGENI

• Each one takes a different approach in addressing I&M requirements and demands

• Each CMF has its own monitoring capabilities• How to put all these together?

Motivation

• An Instrumentation and Measurement Architecture Supporting Multiple Control Monitoring Frameworks

• Our target is: – to provide, possibly, with a maximum reuse of the

available CMFs I&M services over a new integrated and federated network structure;

– To provide instrumentation and monitoring considering different I&M Services through FIBRE-BR (Monitoring Orchestration);

– Multiple CMFs I&M data integration.

Goal

FIBRE-BR I&M ARCHITECTURE

FIBRE-BRI&M Persistent

Data Repository

OFELIAMonitoring

Data

MDIPMeasurement

Data Integration

Point

Aggr03

Aggr02

Aggr01

OFELIA CMF

OFELIA Control

Commands

ProtoGeni Monitoring

Data

MDIPMeasurement

Data Integration

Point

Aggr03

Aggr02

Aggr01

ProtoGENI CMF

ProtoGeni Monitoring Facilities

ProtoGeni Control

Commands

OMF Monitoring

Data

MDIPMeasurement

Data Integration

Point

Aggr03

Aggr02

Aggr01

OMF CMF

OMF Monitoring Facilities

OMF Control

Commands

Experimenter

Researcher

NetworkManager

Visualization/ Portal Service

Security Services

Orchestration & Configuration Service

FIBRE-BRPolicy

Repository

OFELIA Monitoring Facilities

FIBRE-BRI&M

SERVICES

FIBRE-BR I&MSERVICES

FIBRE-BR I&MSERVICES

FIBRE-BR I&MSERVICES

1

2

3

5

4

Measurement Data Integration Point (MDIP)- conforms the collected data

from the available CMFs to FIBRE-BR I&M standard format (NM-WG), representation and distribution (including visualization).

- This service includes all measurement data processing related aspects such as, message format, message transport protocol and/or service, access privileges and common data storage or on-the-fly data distribution.

1

The security and police service will use the global definitions implemented and controlled by the ClearingHouse(CH definition - is both an entity and a system consisting of software, operations, and policy to broker trust between federation partners.)

Some services that are going to be used by the I&M Architecture:• Trust relationship (CA, SASL, etc)• Identity credentials• Integrated authentication/authorization• Federation level policies• Slice behavior• Data access policy• Policy enforcement• FIBRE-BR policy document

2

The Orchestration and Configuration Services act on behalf of the users allowing them to configure, to define measurement points, and to orchestrate these measurement data collection facilities according to each individual CMF.

Implementation initial ideas:- Use XML pub/sub messaging service, based on

XMPP server- Currently supported by OMF- IMF @ GENI

3

The I&M Portal main functionality is to provide a user friendly interface to control and access the measured data, according to a defined policy.

Data visualization:- From real-time experiments- From data stored Persistent Data

repository in each individual CMF (I&M perspective)

- Verify privileges access- Available only to authorized users

4

The architecture has a storage strategy that allows users to retrieve data from their own or from others previous experiments, according to their access privileges. The persistent storage option is an experimenter decision that must comply with FIBRE-BR retention policy.

Implementation steps: • MDIP will be in charge of saving it

persistently• Data retention policy• MySQL/RRD/SQLLite3/PostgreSQL

database (access from I&M solution)• I&M Standard storage• Each CMF will keep its storage

mechanism. I&M will, eventually, store it centrally or access it based on users demand and/or privilege

• Logs storage

5

OML Overview and OML-MDIP Proposal

13

• OML was first developed as the measurement component of OMF• Today is a stand-alone project (independent)• Shortly, it is a framework (set of libraries) to collect and store

measurements

OML Introduction

14

• OML MDIP consists basically of one service: OML MA (OML Measurement Archive)

− It’s responsible for receiving and sending the requests and responses

− When it receives the requests, the MA queries OML Server DB and then uses standard perfSONAR messages to communicate with FIBRE-BR I&M Services

− The response is sent to the MDIP, in order to be stored persistently in FIBRE-BR I&M repository (if demanded)

OML MDIP Proposal(1)

15

OML MDIP Proposal(2)

16

- FIBRE-BR I&M Architecture is designed to configure and collected data from different CMFs.

- It is intended to be an instrumentation and measurement evolutionary architecture

- There are a lot of challenges to overcome!

Conclusions

Thank you / Obrigado

Backup slides

ProtoGENI I&M Architecture and Tools

• There are several GENI I&M related projects compatible with ProtoGENI. The most important ones are:– INSTOOLS– LAMP– OneTimeMeasure– S3Monitor

• Some of them are complimentary and will later be integrated to compose a GENI Integrated I&M Framework

Introduction

Instrumentation Tools (INSTOOLS)

• INSTOOLS’ high-level goal:– Make it easy for users to see what is going on in their

experiment – i.e., make it trivial to monitor a slice• What can INSTOOLS

measure?– Note that INSTOOLS is

concerned only about passive measurements

• INSTOOLS’ philosophy– Don’t reinvent the wheel

Overview

Architecture (overview)

Experimenter

Measurement Plane connections

Data Plane connections

MC Portal: single interface for all MCs in a slice

Slice’s nodes (instrumentized to actalso as MPs)

Measurement Controller (MC) = GENI MAP+MCAutomatically deployed by INSTOOLS (at least one per aggregate)

INSTOOLS Web Portal

LAMP

• LAMP stands for “Leveraging and Abstracting Measurements with perfSONAR”

• The main goal is to “create an instrumentation and measurement system, based on perfSONAR, for use by experimenters on ProtoGENI”

• Which tools does LAMP support?– OWAMP, BWCTL, Ganglia, PingER, NTP; ps-BUOY MA; etc

• So... what’s the difference between LAMP and pS-PSToolkit?– LAMP adapted perfSONAR-PS software suite to recognize GENI’s

Authentication and Authorization model and infrastructure– Added Ganglia as a host monitoring solution– Added distributed configuration through annotations in the topology

stored in UNIS– These annotations make it easy for users to save their slice

configuration and load it at a different slice

Overview

Architecture (a broader view)

ProtoGENIUNIS

Slice 1 Slice 2 Slice n

Node with LAMP tools

Node with LAMP tools + LAMP Portal enabledUNIS keeps

information about ALL slices Note that it is possible to

have more than one node running the Web Portal

Not every node has to be

“instrumentized”

This node has only “Measurement Plane

interfaces”

LAMP Portal

Each node has as set of (possible

different) measurement tools enabled

OneTimeMeasure andS3Monitor

• OnTimeMeasure– Is “an on-demand measurement service used in forecasting,

anomaly detection, and fault-location diagnosis in GENI experiments and GENI operations.”

– Can be integrated with INSTOOLS• S3Monitor

– Has a flexible design that allows easy “plug in” of new network measurement tools

– Based on (Scalable Sensing Service for PlanetLab)– Focused on measurement of large networked systems– Already integrated with INSTOOLS

OneTimeMeasure and S3Monitor

Integrating with FIBRE-BR I&M architecture

INSTOOLS MDIP

ProtoGENI Fibre I&M Architecture

Slice (INSTOOLS)

INSTOOLSMeas Controller

RRD

ProtoGENI

MDIP

RRD Collector

FIBRE I&M Data Repository

1 2

SQL DB

SQLCollector

1. MC collects measurement data from the MPs2. MDIP (through his collectors) collects measurement data, makes any

necessary format adjustments and 3. Stores the data in the permanent repository (if demanded) 4. Measurement data can be accessed through the Visualization Portal

3

Visualization Portal

4

LAMP MDIPProtoGENI

Fibre I&M Architecture

Slice ProtoGENI MDIP

FIBRE I&M DataRepository

NMWGGeneric

Collector

UNIS

MP

MP

LAMP Web Portal

SNMP MA

ps-BUOY MA

Ganglia MA

PingER MA

MA-specific defs

1. Fetch experiment description from UNIS2. Start copying measurement data from MAs3. Stores the data in the permanent repository (if demanded)

1

2

3

Visualization Portal

OFELIA I&M Architecture Igor Luiz (UNIFACS)

35

NOWADAYS

Backend Infrastructure Monitoring (servers, switches, links)- ZENOSS (not integrated to OCF)

OFELIA´s experiment monitoring:- Still under discussion and development

OFELIA I&M STATUS

36

• FIBRE I&M possible alternatives for OFELIA CMF are (in discussion):− Focus on FlowVisor and OF-related basic

measurements parameters (pragmatic approach)− Incorporate current OFELIA monitoring developments

(need to identify them)− Align with OFELIA current developments

OFELIA & FIBRE I&M

• OFELIA I&M technical possibilities/discussions:− OML − sFlow− perfSONAR− LAMP− ... !?