perfSONAR Information DiscoveryperfSONAR Information Discovery

February 11th 2010, APAN 29 – perfSONAR WorkshopJeff Boote, Assistant Director R&D

• Motivation• Information Services Architecture

– Global Lookup Service– Home Lookup Service– Topology

• LS Operations– Registration– Discovery– Synchronization

• Protocol

2 – 04/21/23, © 2009 Internet2

Outline

• Network measurements are useful for a local domain, but are also pivotal in solving end to end problems.– Many network problems are found to be at the demarcation of a

network– Most forms of communication must cross outside of the home

domain• Each domain should strive to make historical data available

– À la carte operation frees network staff from special requests– Data availability will drive the production of automatic tools

• Finding specific information can be challenging– Finding a specific type of measurement– Finding data for a given domain or IP subnet– Finding data classified by a VO or community

3 – 04/21/23, © 2009 Internet2

Motivation

• The perfSONAR information services were designed to record the location of data categorized on the following criteria:– Data Type, e.g. by classification such as latency or bandwidth. Or

by tool such as ping or iperf– Domain name– Topology reference (For L3: IP Address (v4 or v6))– Keyword, e.g. “tagging”. Users may assign arbitrary metadata to

measurement data to signify membership in a VO (USATLAS, eVLBI) or other forms of association (Internet2, DOE).

• Services are autonomous – they know to register to some (any close) information service.

• Information services are autonomous as well – they are self organizing to distribute information.

4 – 04/21/23, © 2009 Internet2

Motivation

• The architecture of the perfSONAR Information Services consists of the following components:– Global Lookup Service– Home Lookup Service– Topology– Clients and Services

• Each component is designed in the SOA fashion to do one specific job.

• All components speak the same XML protocols for interoperability

5 – 04/21/23, © 2009 Internet2

Information Services Architecture

6 – 04/21/23, © 2009 Internet2

Information Services Architecture

7 – 04/21/23, © 2009 Internet2

Information Services Architecture

8 – 04/21/23, © 2009 Internet2

Information Services Architecture

9 – 04/21/23, © 2009 Internet2

Information Services Architecture

10 – 04/21/23, © 2009 Internet2

Information Services Architecture

11 – 04/21/23, © 2009 Internet2

Information Services Architecture

12 – 04/21/23, © 2009 Internet2

Information Services Architecture

• The Global Lookup Service is a federation of individual instances– Recommended that the total number be kept small and

geographically placed– Form an ‘information cloud’– Each will synchronize with others to exchange information– Each server is a peer, and within a set amount of time knows what

other servers know• To “bootstrap” where to find the gLS services, a ‘hints’ file is

used– Similar to DNS– Current location: http://www.perfsonar.net/gls.root.hints– Each hints file = a new cloud, can be many at any given time

13 – 04/21/23, © 2009 Internet2

Global Lookup Service

• The gLS performs periodic operations:– Synchronizing with others– Summarizing internal data– Cleaning the databases of expired data

• Data has a TTL – it is automatically cleaned if not renewed• The gLS maintains 2 databases:

– ‘Registered’ information – this comes from hLSs– ‘Summarized’ information – this is a combination of everything

registered• Query Types are structured, or ad-hoc

– XQuery interface to arbitrarily query the databases– Well formed query description for ‘discovery’ need

• Domains• IP Addresses• Data Type• Keywords

14 – 04/21/23, © 2009 Internet2

Global Lookup Service

• gLS servers only accept registrations from hLS servers– MA/MP etc. services do not register to the gLS

• gLS servers respond to the following messages:– LSRegistrationRequest

• Register data from an hLS

– LSDeRegistrationRequest• Deregister data for an hLS

– LSKeyRequest• Retrieve the registration key for an hLS

– LSKeepaliveRequest• Renew a registration for an hLS

– LSQueryRequest• Query the database of the gLS

15 – 04/21/23, © 2009 Internet2

Global Lookup Service

• The Home Lookup Service manages the registration information of the services– Services register their name/address– Services register all of the measurement data they maintain

• Each domain should deploy an hLS– Use multiple hLSs if many services are required (e.g. load on hLS

will dictate splitting – recommended < 10 services per hLS)• hLSs will contact a ‘close’ gLS by using the hints file• hLSs may contact multiple gLSs if desired (gLS synchronization

does not make this a requirement)

16 – 04/21/23, © 2009 Internet2

Home Lookup Service

• The hLS performs periodic operations:– Synchronizing with others– Summarizing internal data– Cleaning the databases of expired data

• Data has a TTL – it is automatically cleaned if not renewed• The hLS maintains 2 databases:

– ‘Registered’ information – this comes from services– ‘Summarized’ information – this is a combination of everything

registered• Query Types are structured, or ad-hoc

– XQuery interface to arbitrarily query the databases– Well formed query description for ‘discovery’ need

• Domains• IP Addresses• Data Type• Keywords

17 – 04/21/23, © 2009 Internet2

Home Lookup Service

• hLS servers only accept registrations from services• hLS servers respond to the following messages:

– LSRegistrationRequest• Register data from a service

– LSDeRegistrationRequest• Deregister data for a service

– LSKeyRequest• Retrieve the registration key for a service

– LSKeepaliveRequest• Renew a registration for a service

– LSQueryRequest• Query the database of the hLS

18 – 04/21/23, © 2009 Internet2

Home Lookup Service

perfSONAR Information DiscoveryperfSONAR Information DiscoveryFebruary 11th 2010, APAN 29 – perfSONAR WorkshopJeff Boote, Assistant Director R&D

For more information, visit psps.perfsonar.net

19 – 04/21/23, © 2009 Internet2

• The topology service stores and normalizes network topology– XML format – similar to measurements– Each topology element features IDs and IDrefs to link state– Supported elements:

• domain• network• node• port• link• path• service

20 – 04/21/23, © 2009 Internet2

Topology Service

• Topology descriptions can be entered in two ways:– By hand via configuration files– Individual elements registered by services

• Queries are supported:– XQuery interface to answer arbitrary topology questions

• The TS registers the domains it has control over to the LS infrastructure.

21 – 04/21/23, © 2009 Internet2

Topology Service

• hLS servers respond to the following messages:– TSQueryRequest

• Query the TS for information

– TSAddRequest• Add new information to the TS

– TSUpdateRequest• Update the time to live for existing information in the TS

– TSReplaceRequest• Replace existing information in the TS

22 – 04/21/23, © 2009 Internet2

Topology Service

• The following operations will be described– Registration– Discovery– Syncronization

• Each operation may involve different actors– hLSs– gLSs– Services– Clients

• There is a time sensitivity to each operation, and it may be performed periodically.

23 – 04/21/23, © 2009 Internet2

LS Operations

• There are 2 Types of Registration:– Service to hLS registration– hLS to gLS registration

• Services will register:– Service name, contact information– All measurement “Metadata”, e.g. the description of each

measurement data set. • hLSs will register:

– Service name, contact information– Summarized data for each registered service (e.g. all “Metadata”

for a given service will be summarized into a set format)

24 – 04/21/23, © 2009 Internet2

LS Operations - Registration

25 – 04/21/23, © 2009 Internet2

LS Operations – Service Registration

26 – 04/21/23, © 2009 Internet2

LS Operations – Service Registration

27 – 04/21/23, © 2009 Internet2

LS Operations – hLS Registration

28 – 04/21/23, © 2009 Internet2

LS Operations – hLS Registration

• Registration accomplishes 2 goals:– Services identify themselves into the LS infrastructure– Services ‘renew’ the registration, e.g. establish themselves as

being active• Other messages go hand in hand with registration:

– DeRegistration• Remove an entire registration• Remove data from the registered data set

– Keepalive• Renew a registration

– KeyRequest• Request a key for a registered data set

29 – 04/21/23, © 2009 Internet2

LS Operations - Registration

• Discovery is the process of locating registered information• Discovery is normally a two part process:

– Find who to ask the question– Ask the question

• The first layer in the discovery process is the gLS:– gLS maintains a list of all hLSs and their summaries– Can answer which hLS has more information for a given discovery

query• The second layer is the hLS:

– hLS maintains a list of some services and their summaries– Can answer which service has more information for a given

discovery query

30 – 04/21/23, © 2009 Internet2

LS Operations - Discovery

• Typical data request:– “I want SNMP data for the Chicago -> New York link on the

Internet2 backbone”• Breaking down the components:

– SNMP Data– Internet2 Domain– Specific interfaces (OC192 between Chicago and New York)

• How to ask the question to the hLS/gLS:– Who has SNMP Data for the Internet2 Network?

• The answer from each will be different:– From gLS: which hLS(s) to ask– From the hLS: which service(s) to ask

31 – 04/21/23, © 2009 Internet2

LS Operations - Discovery

32 – 04/21/23, © 2009 Internet2

LS Operations - Discovery

33 – 04/21/23, © 2009 Internet2

LS Operations - Discovery

34 – 04/21/23, © 2009 Internet2

LS Operations - Discovery

35 – 04/21/23, © 2009 Internet2

LS Operations - Discovery

36 – 04/21/23, © 2009 Internet2

LS Operations - Discovery

37 – 04/21/23, © 2009 Internet2

LS Operations - Discovery

• hLSs and gLSs accept the same kinds of query message– XQuery

• Allows complex use of this language to request almost anything about the data

– Discovery• Structured to limit the querying to specific elements

– Domains– IP Ranges– Data Types– Keywords

• Full recursion is possible with the query model, but not currently supported.

38 – 04/21/23, © 2009 Internet2

LS Operations - Discovery

• gLS servers are independent yet federated– Any domain can deploy one– Services may register to any gLS– gLSs exchange data sets with each other, and ‘register’ things they

may not know about• Synchronization relies on:

– Hints file containing which gLSs participate in a “cloud”– gLSs maintaining a database of the services that have registered

directly with them (e.g. “authoritative” registration source for a given service)

– gLSs running a periodic “broadcast” to all other gLSs containing the hLS instances they are authoritative for. Other gLSs will then choose to either register or ignore this information.

39 – 04/21/23, © 2009 Internet2

LS Operations – Synchronization

40 – 04/21/23, © 2009 Internet2

LS Operations – Synchronization

• There are 5 Major messages in the gLS/hLS Protocol– LSRegistrationRequest– LSDeregistrationRequest– LSKeepaliveRequest– LSQueyRequest– LSKeyRequest

• Each is based on the standard perfSONAR message format:– Message container– At least one metadata element– At least one data element, linked to the metadata element

41 – 04/21/23, © 2009 Internet2

Protocol

• LSRegister(Request/Response)– Register content into the hLS (or gLS)– Metadata(s) = Service Structure– Data(s) = Metadata associated with service– Different operations based on context

• ‘New’ – If the service is not in the LS, it is added to the LS as a new entity

• ‘Update’ – Use the ‘key’ to add new items to the LS• ‘Clobber’ – Send the key and changes to the service element (N.B.

this nukes existing dataset)

Protocol - LSRegister

Protocol - LSRegister

Protocol - LSRegister

Protocol - LSRegister

• LSDeregister(Request/Response)– Remove content from the hLS (or gLS)– Metadata = Key for a service– Data = nothing, or specific service metadata• ‘nothing’ case is a data trigger• ‘Selective’ case is a pS-PS addition (EU *LS code does

not support this)

Protocol - LSDeregister

Protocol - LSDeregister

Protocol - LSDeregister

• LSKeepalive(Request/Response)– Updates the time in the control structure to allow

this service to still live in the hLS (or gLS)– Metadata = key– Data = nothing (trigger)

Protocol - LSKeepalive

Protocol - LSKeepalive

• LSQuery(Request/Response) has different types (varies by eventType):– XQuery (one for each ‘database’ the LS maintains)

• http://ogf.org/ns/nmwg/tools/org/perfsonar/service/lookup/discovery/xquery/2.0

• http://ogf.org/ns/nmwg/tools/org/perfsonar/service/lookup/query/xquery/2.0

– Discovery• http://ogf.org/ns/nmwg/tools/org/perfsonar/service/lookup/

discovery/summary/2.0

Protocol - LSQuery

• LSQuery(Request/Response) - XQuery– Query the database in a ‘raw’ format (need to

know a little about how the data is structured)– Metadata = XQuery/XPath statement– Data = nothing (trigger)

Protocol - LSQuery

Protocol - LSQuery

• LSQuery(Request/Response) - Discovery– Similar to query, search discovery set for specific items– Metadata = Discovery Items

• Domains• Keywords• Addresses• EventTypes

– Data = nothing (trigger)– More valuable in gLS case (N.B. discovery on an hLS works

only on the hLS dataset)

Protocol - LSQuery

Protocol - LSQuery

• LSKey(Request/Response)– Request a key (normally given at registration time)

for a service– Metadata = Service description– Data = nothing (trigger)– Currently not ‘safe’ due to no AA infrastructure

Protocol - LSKey

Protocol - LSKey

perfSONAR Information DiscoveryperfSONAR Information DiscoveryFebruary 11th 2010, APAN 29 – perfSONAR WorkshopJeff Boote, Assistant Director R&D

For more information, visit psps.perfsonar.net

58 – 04/21/23, © 2009 Internet2