University of Illinois at Chicago Euro-Link HPIIS Performance Review October 25, 2000 Tom DeFanti Maxine Brown Jason Leigh Laura Wolf Linda Winkler

University of Illinois at Chicago

Euro-LinkHPIIS Performance Review

October 25, 2000

Tom DeFantiMaxine BrownJason LeighLaura Wolf

Linda Winkler


Euro-Link National Research Networks

Australia

China

Japan

Korea

Singapore

Taiwan

Canada

US Networks: vBNS, vBNS+, Abilene, ESnet, DREN, NREN/NISN

Chile Russia

CERN

Denmark

Finland

France

Iceland

Israel

Netherlands

Norway

Sweden

www.startap.net


Introduction to Euro-Link

• Cooperative Agreement ANI-9730202 to University of Illinois at Chicago

• To develop with European National Research Networks (NRNs) a US-European Network Consortium (Euro-Link) to provide US research institutions with high-performance access to outstanding R&E institutions in the European region connected by the European NRNs’ infrastructures.


Euro-Link $2.2M Annual Budget

• ~ 80% of funds go to cost-sharing the networks– $1.6M Direct payments) to 5 NRNs– $270K Ameritech AADS Local loops– No UIC overhead

• ~20% of funds go to staff– Overhead pays for part of business manager

• Euro-Link funds cover ~ 25% of the cost of international links


Euro-Link Management

• Tom DeFanti– Planning and Execution– Budget

• Maxine Brown – Management and Documentation– Events

• Jason Leigh, Andy Johnson, Dan Sandin– Visualization/Performance Analysis Tools

• Linda Winkler, Alan Verlo– Network Engineering


Euro-Link Technical Partners

• Tony Haeuser, AADS• Michael McRobbie and Jim Williams, Euro-Link

NOC, Indiana University• Joe Mambretti, iCAIR and MREN• Manuel Delfino and Olivier Martin, CERN• Ralphi Rom and Hank Nussbacher, IUCC• Peter Villemoes, NORDUnet• Dany Vandromme, RENATER2• Kees Neggers, SURFnet


Euro-Link US Partners

• HPIIS Team (IU, UIC, UTK)• Ameritech Advanced Data Services (AADS) NAP• STAR TAP• Indiana University• Argonne National Laboratory/ESnet• vBNS• Abilene/Internet2• MREN and International Center for Advanced

Internet Research (iCAIR) at Northwestern University

• NISN/NREN• DREN


Euro-Link Statement of Work

1. To create and maintain a high-performance connection between the vBNS and European NRNs via STAR TAP

2. To ensure that only approved institutions’ traffic uses the high-performance connection

3. To monitor Euro-Link performance and use

4. To develop testbeds of new networking protocols (IPv6, multicast, cache, etc.)

5. To maintain a Euro-Link web site

6. To provide user services in support of applications


Euro-Link Statement of Work#1: Connections

• To create and maintain a high-performance connection between the vBNS and European NRNs via STAR TAP– Provide high-performance circuits connecting

European NRNs to NGI networks via STAR TAP– Implement and manage a Level 3 service– Provide a coordinated NOC, NIC and user services

on a 24 x 7 basis


Euro-Link Statement of WorkConnections: CERN

Star Tap

155Mb/s


Euro-Link Statement of WorkConnections: Israel


Euro-Link Statement of WorkConnections: Nordic Countries

155Mb


Euro-Link Statement of WorkConnections: France


Euro-Link Statement of WorkConnections: The Netherlands

622Mb

SURFnet


Euro-Link Statement of WorkConnections: Level 2 and Level 3

• Once connected to STAR TAP, the NRNs, in addition to connecting to the vBNS, can peer with other US NGI networks, the Internet2, and advanced networks from other countries.

• NRNs can peer using the STAR TAP Router or by bilateral agreement.

• They may also connect to one or more ISPs at the AADS facility, which is outside the scope of Euro-Link, but a useful capability nonetheless.


Euro-Link Statement of WorkConnections: Level 2 and Level 3

Euro-Link STAR TAP International Direct Peering Matrix

Networks North & South America AsiaAb

ilene

CA

*net 3

DR

EN

ES

net

IPv6 R

ou

ter

MR

EN

NIS

N

NR

EN

ST

Ro

uter

vBN

S

RE

UN

A

AP

AN

CE

RN

ET

GE

Mn

et

Sin

gA

RE

N

TA

Net2

CERN

Israel IUCC

NORDUnet

RENATER2

SURFnet

Note: European peering is either direct or via TEN-155.


Euro-Link NOC

• http://noc.euro-link.org • HPIIS (Euro-Link, TransPAC, MIRnet) helps

support the Global Research NOC staff, network engineers and applications engineers

Jim Williams will present the Global NOC.


Euro-Link Statement of Work#2: Ensuring High Performance Traffic

• To ensure, to the extent supportable by prudent application of networking technology, that only approved institutions’ traffic uses the high-performance connection

– Implement routing and switching configurations consistent with AUPs of European NRNs and vBNS

– Update routing and switching hardware and software as new capabilities for implementing routing policies become available

– Document AUP traffic segregation measures and procedures on Euro-Link web site


Euro-Link Statement of WorkEnsuring High Performance Traffic

• Compliance with NSF vBNS AUP documented in Euro-Link annual report at http://www.euro-link.org/PUBLICATIONS/

• Most of the NRNs have a policy to connect all the universities in their countries without discrimination as to research need; however, to comply with the NSF AUP, they segregate traffic either at home or on the US East Coast prior to connecting to STAR TAP


Euro-Link Statement of Work#3: Monitoring Performance and Use

• To monitor Euro-Link performance and use– Monitor traffic parameters to manage the utilization of

Euro-Link networks and assure high-performance quality

– Develop a new generation of performance and analysis tools. Adopt and adapt emerging network performance monitoring technologies and develop advanced visualization, persistent databases and visualization/VR techniques to create new applications-oriented performance measuring tools for HPIIS


Euro-Link Statement of WorkMonitoring Performance and Use

• CERN provides its own Internet traffic statistics: http://sunstats.cern.ch/mrtg

• Performance monitoring – CERN hosts three Internet probes (Surveyor, RIPE and NIMI) and is also part of the Internet End-to-end Performance Monitoring (IEPM) PINGER project, supported by DOE MICS through the Stanford Linear Accelerator Center

• CERN has plans to install Netlogger and to study alternatives to TCP-based file transfer in order to reduce problems linked to error recovery on high-bandwidth delay paths



• IUCC statistical information about all its connections (Home page): http://www.internet-2.org.il/stats.html

• IUCC FastEthernet1/0 traffic analysis: http://noc.ilan.net.il/stats/TAU-GIGAPOP/tau-gp1-fe-i2.ilan.net.il.html

• STAR TAP IUCC peering statistics: http://noc.ilan.net.il/stats/ILAN-I2-uplinks/peers.html

• IUCC has, in addition, installed an NLANR passive monitor: http://moat.nlanr.net/Datacube/Data/TAU/



• NORDUnet NOC operations are carried out at KTH in Stockholm and by the national Nordic networks

• NORDUnet participates in the ANS Surveyor measurement infrastructure with UNINETT, operating a measurement station at the University of Oslo



• RENATER2 does not presently have a web page regarding STAR TAP traffic. Hopefully this will change at some time this year, once the current call for tenders is completed. A higher transatlantic link, along with associated services, will improve performance.



• STAR TAP SURFnet traffic monitoring: http://www.surfnet.nl/surfnet/persons/bos/startap-traffic/

• SURFnet monitors traffic on its STAR TAP connection, as it does on all its connections (external connections, backbone links, and customer links). Results are put on the web using MRTG or RRDTool. This information is typically protected by userid/passwds and access-lists



• EVL visualization network performance analysis tools: Motivation

• QoSIMoTo (QoS Internet Monitoring Tool) visualizes historic as well as real-time Netlogger data consisting of bandwidth, latency and jitter: www.evl.uic.edu/cavern/qosimoto

• Forward Error Correction schemes, both parity-based and XOR-based, are being developed as an alternative to TCP for transmitting real-time data, which has substantial latency over international links.

Jason Leigh will discuss this research this afternoon


Euro-Link Statement of Work#4: Engineering Advancements

• Cooperate with vBNS and NLANR to develop testbeds of new networking protocols (IPv6, multicast and Mbone, resource reservation, cache services, etc.)


Euro-Link Statement of WorkEngineering Advancements

• CERN participates in the Internet2 Distributed Storage Initiative

• IPv6 (with Lawrence Berkeley Lab via 6TAP)• Exploring DiffServ to deliver the required

grade of service to its users (with iCAIR and EVL using STAR TAP DiffServ Router)

– Working with Brian Carpenter, IBM/iCAIR, on the Virtual Room Video-conferencing System (VRVS) DiffServ testbed http://vrvs.cern.ch/About/VRVS_Paper.pdf

• High-speed file transfer with Caltech/SLAC/Fermilab



• IUCC is benchmarking and testing solutions that can negate the influence of large satellite delays that influence TCP throughput; results of testing two products at the Intelsat lab in Washington DC: www.internet-2.org.il/satellite-testing.html

• IUCC installed Mentat XH45 boxes in Israel and its co-location site in Chicago and found instances of a 160-fold performance increase for TCP streams. (NASA also tested the Mentat boxes; they help with both high-latency and low-latency (70 ms) links, which is applicable to the US’s high-performance networks. www.mentat.com/skyx/xh45.html

• Previously IUCC worked with researchers to TCP tune their systems, but this is not scalable: www.psc.edu/networking/perf_tune.html



• NORDUnet has set up native multicast connectivity with most of the STAR TAP-connected networks



• RENATER2 is increasing available transatlantic bandwidth. A call for tender is in progress. The transatlantic capacity should be increased above 622 Mbps, including an OC-3 path to STAR TAP.

• Experiments with QoS services, aiming at a more efficient (in terms of performance and of bandwidth allocation and usage) use of the costly transatlantic infrastructure, and make them operational as soon as possible.

• Experiment with IPv6 as part of the 6Bone activity.

• Experiment with end-to-end (campus to campus) QoS services



• EVL is collaborating with SARA in The Netherlands on network performance tests

• SURFnet is cooperatively planning with CA*net3/4 to bring 2.5Gb to STAR TAP as part of the I-WIRE/ Star Light effort



Euro-Link/ STAR TAP• A policy-free IPv4 router enabling STAR TAP-

connected networks to exchange routes and traffic with other STAR TAP-connected networks with whom they do not peer directly

• IPv6 efforts with ESnet• Web caching with NLANR and the Internet2

Distributed Storage Working Group• Performance measuring using the NLANR

Active Measurement Platform (AMP) instrument• DiffServ experiments with the DOE NGI-

supported EMERGE testbed• Unicast/multicast digital video QoS efforts with

iCAIR at Northwestern University • QBone QoS efforts with Internet2 and iCAIR


Euro-Link Statement of Work#5: Web Site

• To maintain a publicly-accessible Euro-Link web site about the European NRNs connections and research and education collaborations that enables:– Up-to-date international collaboration information,

points of contact, engineering links, documentation and mailing lists

– Up-to-date statistical information about the performance of the Euro-Link international connection

– Documented Euro-Link success stories, through up-to-date proposals, detailed technical requirements and results


Euro-Link Statement of WorkWeb Site

www.euro-link.org


Euro-Link Statement of Work#6: User Services

• In concert with the methodologies developed by NLANR DAST, provide consultative user services supporting the use of Euro-Link for high-performance computing and communications applications– Projects– Applications clearinghouse– Software tools– Training events– Presentations


Euro-Link Statement of WorkUser Services

• Projects: iGrid2000, etc: www.startap.net/igrid2000• Applications: www.euro-link.org/APPLICATIONS• Euro-Link annual meetings with HPIIS Team and STAR TAP

at INET 1998, 1999, 2000…• Software tools: CAVERNsoft, QoSIMoto• Training sessions

– Telecomm 2000, Israel, November 2000– Eurographics’ Workshop on Virtual Environments held in

Amsterdam in June, 2000

• Presentations– 19th Nordic Networking Conference, organized by NORDUnet, held

at Finlandia Hall, Helsinki, Finland in September, 2000– PDC Conference, Center for Parallel ComputersRoyal Institute of

TechnologyStockholm, Sweden, Dec 1999


Euro-Link Statement of WorkUser Services

European participation/collaborations at iGrid 2000

Scenes from iGrid 2000: http://www.startap.net/igrid2000


Scientific ApplicationsCERN

Networked experiments of the European Laboratory for Particle Physics (CERN)

CERN provides experimental facilities for particle physics experiments, mainly in the domain of high-energy physics.

Four very large experiments in man-made caverns intersect the LEP tunnel, constituting about half of CERN’s total experimental program for the 1990’s.

50 participating institutions, several hundred physicists, in five continents. CERN’s Large Electron Positron (LEP)

collider, in a 27km tunnel – the largest machine of this type in the worldhttp://www.cern.ch



The Globally Interconnected Object Databases (GIOD) Project

The Compact Muon Solenoid (CMS) detector will cleanly detect the diverse signatures of new physics at the Large Hadron Collider (LHC), due to be operational in 2005. It will identify and precisely measure muons, electrons and photons over a large energy range.

The data is overwhelming. Even though the data from the CERN CMS detector will be reduced by a factor >107, it will amount to over a Petabyte (1015 bytes) of data per year accumulated for scientific analysis.

Caltech, CERN, Hewlett Packard, others

Particle physicists are engaged in large international projects to address a massive data challenge, with special emphasis on distributed data access.http://pcbunn.cithep.caltech.edu



MONARC Project

LHC experiments have envisaged Computing Models (CM) involving hundreds of physicists doing analysis at institutions around the world.

Regional Centers are intended to facilitate access to the data; with more efficient and cost-effective data delivery to the groups in each world region, using high-speed national networks.

This project intends to study network-distributed computing architectures, data access and data management systems that are major components of the CM, and the ways in which the components interact across networks.

MONARC (Models of Networked Analysis at Regional Centers) will be created for data analysis from the Large Hadron Collider LHC experiments

http://www.cern.ch/MONARC/


Scientific ApplicationsCERN @ iGrid 2000

Distributed ParticlePhysics Research • California Institute of Technology,

USA, USA• CERN

for authentication and resource discovery, a distributed object database containing several terabytes of simulated events, and a component that enables queries issued by the front-end application to be matched to available computing resources in the system (matchmaking service). http://pcbunn.cacr.caltech.edu http://iguana.web.cern.ch/iguana/

http://vrvs.cern.ch http://cmsdoc.cern.ch/orca

This application demonstrates remote viewing and analysis of particle physics events. The application is the front end to an engineered object-oriented global system that incorporates grid middleware


Scientific ApplicationsIsrael

Interactive Simulation in the Field of Plant Nutrition

This US-Israel Bi-national Agricultural Research and Development (BARD) Foundation project involves hypotheses testing and result evaluation, using an interactive graphic model.

SimRoot, a 3D model developed at Penn State, graphically describes the deployment of plant root systems in soil – and predicts the performance of plants under various environmental conditions, depending on their physiological characteristics.

Tel Aviv University and Penn State University



Visualization of Acetylcholinesterase: Nature's Vacuum Cleaner

Acetylcholinesterase (AcChoEase) is an enzyme that plays a key role in the human nervous system. In vertebrates, nerve impulses travel from cell to cell by means of chemical messenger. When an electrical impulse reaches the end of one cell, messenger molecules – acetylcholine (AcCho), in this case – are released to diffuse though the fluid-filled, intercellular, synaptic gap. Upon reaching the destination cell, the AcCho molecules dock into special receptors triggering a new electrical impulse. Much like a vacuum cleaner, the enzyme AcChoEase is constantly sweeping up and hydrolyzing AcCho during this process, so that the whole cycle can begin again. Chemicals that inhibit the action of AcChoEase are being used in the treatment of glaucoma, myasthenia gravis and, experimentally, Alzheimer's disease. In spite of the ability to exploit the enzyme, its precise mechanism of operation is still a mystery.

Cornell University, USA; Weizmann Institute of Science, Israel



Visualization of Acetylcholinesterase: Nature's Vacuum Cleaner

Acetylcholinesterase (AcChoEase) is an enzyme that plays a key role in the human nervous system. In vertebrates, nerve impulses travel from cell to cell by means of chemical messenger. When an electrical impulse reaches the end of one cell, messenger molecules – acetylcholine (AcCho), in this case – are released to diffuse though the fluid-filled, intercellular, synaptic gap. Upon reaching the destination cell, the AcCho molecules dock into special receptors triggering a new electrical impulse. Much like a vacuum cleaner, the enzyme AcChoEase is constantly sweeping up and hydrolyzing AcCho during this process, so that the whole cycle can begin again. Chemicals that inhibit the action of AcChoEase are being used in the treatment of glaucoma, myasthenia gravis and, experimentally, Alzheimer's disease. In spite of the ability to exploit the enzyme, its precise mechanism of operation is still a mystery.

Cornell University, USA; Weizmann Institute of Science, Israel


Scientific ApplicationsNordic Countries

European Incoherent SCATter (EISCAT)

EISCAT studies the interaction between the Sun and the Earth as revealed by disturbances in the magnetosphere and the ionized parts of the atmosphere (these interactions also give rise to the spectacular aurora, or Northern Lights).

The Incoherent Scatter Radar technique requires sophisticated technology and EISCAT engineers are constantly involved in upgrading the systems.

The EISCAT Scientific Association operates radars and receivers in several Nordic cities. Several Incoherent Scatter facilities are distributed about the world, such as Millstone Hill Observatory (MHO), in Westford, Massachusetts.

http://www.eiscat.uit.no/


Scientific ApplicationsSweden @ iGrid 2000

Steering and Visualization of a Finite-Difference Code on a Computational Grid • Royal Institute of Technology,

Sweden • University of Houston, USA

This application enables computational steering of electromagnetic simulations across distributed resources using interactive visualization in a virtual-reality environment.

www.nada.kth.se/~erike/progress


Scientific ApplicationsFrance

The DØ Experiment is a worldwide collaboration of scientists conducting research on the fundamental nature of matter. The experiment is located at the world's premier high-energy accelerator, the Tevatron Collider, at the Fermi National Accelerator Laboratory.

Worldwide collaborations include Fermilab, Brookhaven National Lab; CERN; Cornell University; DESY, Germany; KEK, Japan; Lawrence Berkeley Lab and Stanford Linear Accelerator Center; several French institutions (DAPNIA/SPP, Centre de Physique des Particules de Marseille, Institut des Sciences Nucleaires de Grenoble, LPNHE, Universites Paris VI and VII, and Laboratoire de L’Accelerateur Lineaire.

http://www-d0.fnal.gov http://www-dapnia.cea.fr



iMAGIS—Models, Algorithms, Geometry for Graphics and Image Synthesis

The project develops new visualization techniques to enable the interactive manipulation of urban data. To achieve this goal, efficient image caching and interpolation techniques are being combined with traditional 3D techniques.

http://www.inria.fr/Equipes/IMAGIS-fra.html

This is important for applications such as project review, civil and military simulators, virtual tourism, and education, climate and environmental studies.

Participants: INRIA (Grenoble), MIT



Sloan Digital Sky Survey

The SDSS enables the automatic, systematic study and exchange of data of stars, galaxies, nebula, and large-scale structure.

Participants: Institut d’ Astrophysique de Paris, Johns Hopkins University, CFHT Corp. (Hawaii); University of Hawaii Institute for Astronomy, Honolulu.

http://www.sdss.org/sdss.html


Scientific ApplicationsThe Netherlands @ iGrid 2000

ALIVE: ArchitecturalLinked Immersive Environment• Academic Computing Services

Amsterdam (SARA), The Netherlands• Electronic Visualization Laboratory,

University of Illinois at Chicago, USA

ALIVE is used to evaluate the usability of collaborative virtual reality for architectural design.The ALIVE project started February 1999 at SARA in cooperation with EVL and OMA (Office for Metropolitan Architecture). In February 1998, Rem Koolhaas won the Richard H. Driehaus Foundation International Design Competition for the new Campus Center at Illinois Institute of Technology’s historic Mies van der Rohe campus

www.sara.nlwww.archfonds.nlwww.iit.edu/departments/pr/masterplan/mccortribucamcen.html


Scientific ApplicationsUSA, Canada, Japan, Singapore, Netherlands, Sweden,

CERN, Spain, Mexico, Korea @ iGrid 2000

GiDVN: Global Internet Digital Video Network • Digital Video Working Group, Coordinating

Committee for International Research Networks• CERN, Switzerland• APAN, Japan; KDD, Japan• APAN-KR, Korea; Seoul National University,

Korea• SURFnet, The Netherlands• DFSCA-UNAM, Mexico• SingAREN, Singapore• Universitat Politecnica de Catalunya, Spain• Royal Institute of Technology, Sweden• Int’l Center for Advanced Internet Research

(iCAIR), Northwestern, USA

GiDVN projects are enhancing media capabilities for the next-generation Internet, enabling new applications to interoperate throughout the world.

www.icair.org/inet2000


Euro-Link Future Plans

• Abilene, Ca*NET3, STAR TAP International Transit Networks

• Multi-Protocol Label Switching (MPLS) QoS as well as continuing DiffServ experiments)

• GryPhyN (Harvey Newman)• Globus and Grids (Ian Foster)• Access Grid deployment to Europe• Dense Wave Division Multiplexing (DWDM) and Optical

Switching with MREN/I-WIRE, CA*net3 and SURFnet• Continue Native Multicast deployment to Euro-Link

universities, following TransPAC lead• Continue IPv6 activities with ESnet, Abilene, Euro-Link

universities

Documents

University of Illinois at Chicago Euro-Link HPIIS Performance Review October 25, 2000 Tom DeFanti Maxine Brown Jason Leigh Laura Wolf Linda Winkler