1

Xrootd monitoring status update(Data collectors and Dashboard)

Julia Andreeva on behalf of xrootd monitoring developers:

Matevz Tadel, Artem Petrosyan, Daniel Dieguez Arias, Sergey Belov, Danila Oleynik, Domenico Giordano, Sergey Mitsyn, David Tucket, Alexandre Beche

2

Changes in the UCSD collector

• Matevz had created the new version of the collector which does publishing to ActiveMQ. In future no need for an additional component which does translation of UDPs into ActiveMQ reports.

To be done:• Crosscheck that data reported by the old and new

versions is consistent (Domenico Giordano)• New version has to be tested and properly

packaged (Sergey Belov, estimated time ~2 weeks)

3

Dashboard Message Broker and ActiveMQ collectors

• Tadashi confirmed today that he could consume from the xroot-related topics via virtual queues

• New Dashboard collector consuming xrootd messages (old collector) had been developed ( Daniel Dieguez Arias). For the moment data is recorded in the Dashboard Integration DB (ORACLE).

4

Topology

• Topology information for the moment is created manually. Server/client domains from the xrootd reports are manually maped to sites as they are defined in OIM/GOCDB. Used information sources : VO topology feeds and information from the CMS and ATLAS Dashboards

• Looking forward when this mapping will be provided by AGIS

5

Xrootd traffic in the WLCG Transfer Dashboard

• Thanks to Daniel Dieguez Arias the first prototype showing both FTS and xrootd traffic is available:

http://dashb-wlcg-transfers-dev.cern.ch/ui/• We still need to validate the content. Not yet

clear how we can do it. Investigate various possibilities (subscribe to UCSD ML repository or using Gratia reports)

• Please give a try and send you feedback to dashboard-support@cern.ch

6

WLCG Transfer Dashboard prototype (1)

Both FTS and XRootD are shown by

default

7

FTS vs xrootd traffic

8

Only xrootd

9

Xrootd by site using topology lookup table

10

DDM-like source/destination plots

11

Local vs remote traffic

12

Sorted by VO

Currently almost no activity on

ATLAS federation

13

Plans• Setup hadoop cluster at CERN. (Sergey Mitsyn). The request for

machines is sent • Test the new version of the UCSD collector and create an RPM (2

weeks) (Sergey Belov and Domenico Giordano)• Setup Federation Monitor at CERN based on the Hadoop backend

(1 month) (Artem Petrosyan)• Validate content of the xrootd transfer info in the WLCG Transfer

Dashboard prototype (2 weeks) (Daniel Dieguez Arias)• Setup new collector at CEERN in order to collect data form the EU

federation• Extend functionality following the list of requirements from people

running xrootd federation