Internet2 & perfSONAR-PSInternet2 & perfSONAR-PS

August 23rd 2012, OSG & perfSONAR-PS MeetingJason Zurawski – Senior Research Engineer

• Definitions• Historical Overview• Present Day Activities• “Wish List”• Future Directions

Agenda

2 – 04/20/23, © 2012 Internet2

• Measurement Middleware– Enables the sharing and management of data collected

from external sources (e.g. SNMP, Iperf)– Web Services based front ends – simple requests dump

back XML/JSON formatted raw measurement data– Location/Discovery services (find what you need)– (Our attempt at) GUIs

• Deployed community infrastructure, available on 100s of networks/domains

• Independent software implementations• Protocol to handle communication between

components

What is perfSONAR?

3 – 04/20/23, © 2012 Internet2

• A silver bullet– Performance debugging is still a dark art. – pS tools can assist in the process of identifying a

problem, but it is by no means fully automated• A complete product– Active development effort on protocol and software

• An effort to redefine measurement tools– Incorporate existing tools (e.g. SNMP monitoring, Ping,

Bandwidth etc.) when available. Middleware is there to share (and encourage the sharing) of data

– Ex: Think of a ‘protected’ MRTG web page. Only useful to those with a password. perfSONAR allows sharing of the raw data to be used in other systems

What isn’t perfSONAR?

4 – 04/20/23, © 2012 Internet2

• Definitions• Historical Overview• Present Day Activities• “Wish List”• Future Directions

Agenda

5 – 04/20/23, © 2012 Internet2

• Internet2 End-2-End Performance Initiative (E2EPI)– Fall of 2000– Founding premise for Internet2 is that QoS will solve

reliability problems experienced through the commercial internet. It is presumed that, until QoS is going, bandwidth overload will solve problems.

– Realization that bandwidth alone doesn't solve all problems. M. Mathis of PSC (now Google) presents the Wizard Gap; those with technical know-how can always use a system to full potential while the average user is experiencing end-to-end (E2E) problems or an insignificant improvement

– Performance Initiatives Performance Environment System (PIPES)• Regular testing on paths of interest• Dedicated machines to do the task• Discontinued as of 2005 in favor of perfSONAR

Historical Overview

6 – 04/20/23, © 2012 Internet2

• Global^H^H^H^H^H^H Open Grid Form– ~2003– Network Measurement working group – proposed a

ontology of common network measurements. Common representations and categories

– Created a format to enable programmatic sharing of this data.

– Reference implementation needed to strengthen the standard …

• perfSONAR– ~2004– Joint effort between Internet2 E2EPI, Members of GGF-

NMWG, GEANT2 JRA-1 Activity (Europe)– Focus on developing interoperable software for

measurement exchange (using well defined protocols)– Coordinate on international testing procedures

Historical Overview

7 – 04/20/23, © 2012 Internet2

• perfSONAR (cont)– Included ESnet and RNP (Brazilian NREN) in effort– First software – 2006 (Java based wrapper around RRD files;

enabled web service sharing of collected data). • perfSONAR-PS

– ~2007– Forked effort by U.S. Partners to produce interoperable

software based on sounder operational footing (e.g. emphasis on integration with existing Linux based measurement systems)

– ESnet, Internet2, University of Delaware, Georgia Tech, Indiana University, Fermilab, SLAC

– Producing software updates to current time

Historical Overview

8 – 04/20/23, © 2012 Internet2

• pS Performance Toolkit– ~2009– Observation that the largest barrier to adoption of an

end to end monitoring system is deployment– End to end monitoring needs observation points along

an entire path– Package the tools into a nearly 0 configuration

package:• Initially Knoppix, then CentOS based LiveCD/Netinstall

Linux distribution• Point and click to set up regular monitoring• Integration with outside alarming tools (NAGIOS/ZENOS)

Historical Overview

9 – 04/20/23, © 2012 Internet2

• 11+ Years of experience for some participants• Goal is simplify the identification and solution of end

to end performance problems• Software deployment bar has been lowered to

improve adoption – increased adoption facilitates faster solution space

• Related Efforts:– LHCOPN/LHCONE/USATLAS/USCMS Throughput Group– GEANT eduPERT– Fasterdata.es.net– Internet2 Research Support

• 25% of support requests (32 since April 2012) are related to E2E performance

Historical Overview - Conclusion

10 – 04/20/23, © 2012 Internet2

• Definitions• Historical Overview• Present Day Activities• “Wish List”• Future Directions

Agenda

11 – 04/20/23, © 2012 Internet2

• Software Development– Internet2 Core team = 3 developers, with

fractional FTE allotments– Core products• perfSONAR-PS Software - TL1/SNMP Status

monitoring, SNMP Measurement Archive, perfSONAR-BUOY Regular Testing system, Topology Service, Lookup Service– ESnet supports other current services

• pS Performance Toolkit (Linux Distro) – Support of the system guts and efforts to package software into RPMs– Joint effort with Esnet

Present Day Activities

12 – 04/20/23, © 2012 Internet2

• Software Support– “Community” Mailing lists. We encourage

everyone to be self supporting due to the man-power allotments

– Research Support via the CTO’s office (e.g. more personalized assistance in enabling a full deployment at a campus or VO)

– Soft Money – comes and goes• 2 DOE projects (OSC, GATech)• 1 GENI project (IU)• Internet2 pursuing NSF money – no ETA

Present Day Activities

13 – 04/20/23, © 2012 Internet2

• Network Observatory– Distributed set of test instances at 9 internet2 PoPs

• 2 x Throughput (bwctl.$CITY.net.internet2.edu)• 1 x Latency (owamp.$CITY.net.internet2.edu)• 1 x User tester (ndt.$CITY.net.internet2.edu)

– Used to collect baseline measurements of Internet2 network

– Release network data to researchers– Serves as a testing point for network debugging (e.g.

isolate the problem/divide and conquer).

Present Day Activities

14 – 04/20/23, © 2012 Internet2

• Related (Internal) Efforts– OWAMP

• Stable @ v 3.3• Additional releases not expected beyond bugfixes

– BWCTL• Stable @ 1.3• Additional releases are expected to support Iperf v3, no

ETA– NDT

• Stable, but still changing (MLab tie in).• Currently at 3.6.5. • Lots of questions on the future of this (see Web10G in

next section)

Present Day Activities

15 – 04/20/23, © 2012 Internet2

• Related (External) Efforts– IPERF

• Recent release of version 3 (complete re-write)• ESnet working to integrate into BWCTL• pS effort still using older version 2 of IPERF

– Web10G• New effort to redefine a TCP “MIB” for the Linux kernel• Follow on to Web100 – a necessary component for the

NPAD/NDT tools• pS Performance Toolkit uses Web100 kernels. Expects to

migrate to Web10G when:– Packages are available for CentOS Linux– A user tool (Iperf, NPAD, etc.) can consume the data

Present Day Activities

16 – 04/20/23, © 2012 Internet2

• Definitions• Historical Overview• Present Day Activities• “Wish List”• Future Directions

Agenda

17 – 04/20/23, © 2012 Internet2

• Not Ranked, not Comprehensive. Based on recent discussions and open issues– “Mesh” Configuration (e.g. management of

multiple nodes without having to configure each)– Framework Stabilization• Lookup/Topology Infrastructure is flakey. Re-

thinking information management and sharing• Data retrieval is slow and limited by XML

technology – considering REST API to encourage faster development of clients/GUIs

“Wish List”

18 – 04/20/23, © 2012 Internet2

• Cont.– More Open “Open Source”. Recent interest from

other communities about contributing (Medical, as well as OSG). How can we make that easier?

– Additional tools – pursuing soft funding from NSF/DOE/NIH to alter NDT/create new Web10G tool, or get funding to complete aforementioned items

– OS Upgrades – Support 32/64 Bit versions of CentOS 6 (we keep things simple, no real effort to support Debian based distros, or others).

– GUIS – although we want to get out of this game … Encourage others do do the GUI work, tell us what is needed to do it right.

“Wish List”

19 – 04/20/23, © 2012 Internet2

• Definitions• Historical Overview• Present Day Activities• “Wish List”• Future Directions

Agenda

20 – 04/20/23, © 2012 Internet2

• Internet2 does not want to see pS fade away, but …– Development does cost $ and resources (that we

are short on)– pS does not generate revenue– Community has a sense of ownership when it

comes to wanting free software, but not when it comes to supporting/advancing it. Change in perception needed

– Largest users are VOs, and many requests are not generalizable beyond the VO. VO resources (e.g. OSG) could be exercised on certain tasks

Future Directions

21 – 04/20/23, © 2012 Internet2

• Expected outcomes:– Increased community involvement in the direction

and action of development– Possible introduction of services on top of

measurement infrastructure (e.g. ‘RHEL model’, perhaps we offer a ‘Net+’ service out of measurement for institutions that would rather have a managed solution) – proceeds would advance the open source development

– Stepping away from GUIs and more into hardening the infrastructure (e.g. GUI development belongs with the users that need it most, not network engineers that can’t please everyone)

Future Directions

22 – 04/20/23, © 2012 Internet2

• ?

Questions?

23 – 04/20/23, © 2012 Internet2

Internet2 & perfSONAR-PSInternet2 & perfSONAR-PSAugust 23rd 2012, OSG & perfSONAR-PS MeetingJason Zurawski – Senior Research Engineer

For more information, visit http://psps.perfsonar.net

24 – 04/20/23, © 2012 Internet2