Embed Size (px)
Citation preview
1 November 2010
Eric Boyd, Internet2 Deputy CTO
• Internet2 approached by 3 universi@es (Caltech, University of Michigan, Vanderbilt) and asked to lead a proposal on behalf of LHC and R&E community (July 2009) – MRI-‐R2 solicita@on capped the per university submissions at 3
• Issue was referred to advisory councils; recommenda@on to proceed from AOAC and RAC (July 2009)
• Internet2 convened a series of community calls (August 2009)
• With strong community support, submiUed proposal to Major Research Instrument – Recovery and Reinvestment (MRI-‐R2) (August 2009) – Request was for ~$2 million
• Roughly 50% for equipment, 50% for personnel
DYNES Background
2 – 11/1/10, © 2009 Internet2
• What is it?: – A na@onwide cyber-‐instrument spanning ~40 US universi@es and ~14 Internet2
connectors • Extends Internet2’s ION service into regional networks and campuses, based on
OSCARS implementa@on (developed in partnership with ESnet) of IDC protocol
• Who is it? – A collabora@ve team including Internet2, Caltech, University of Michigan, and
Vanderbilt University – Community of regional networks and campuses – LHC, astrophysics community, OSG, WLCG, other virtual organiza@ons
• What are the goals? – Support large, long-‐distance scien@fic data flows in the LHC, other leading
programs in data intensive science (such as LIGO, Virtual Observatory, and other large scale sky surveys), and the broader scien@fic community
– Build a distributed virtual instrument at sites of interest to the LHC but available to R&E community generally
DYNES Summary
3 – 11/1/10, © 2009 Internet2
AIM: extend hybrid & dynamic capabili@es to campus & regional networks. • A DYNES instrument must provide two basic
capabili@es at the Tier 2S, Tier3s and regional networks:
1. Network resource alloca@on such as bandwidth to ensure performance of the transfer
2. Monitoring of the network and data transfer performance
• All networks in the path require the ability to allocate network resources and monitor the transfer. This capability currently exists on backbone networks such as Internet2 and ESnet, but is not widespread at the campus and regional level.
• In addi@on Tier 2 & 3 sites require:
3. Hardware at the end sites capable of making op@mal use of the available network resources
DYNES System Descrip@on
Two typical transfers that DYNES supports: one between a Tier 2 and a Tier 3 and another between a Tier 2 and a Tier 1 site. The clouds represent the network domains involved in
such a transfer.
• Each DYNES (sub-‐)instrument at a Tier2 or Tier3 site consists of the following hardware, where each item has been carefully chosen to combine low cost & high performance:
1. An Inter-‐domain Controller (IDC)
2. An Ethernet switch
3. A Fast Data Transfer (FDT) server. Sites with 10GE throughput capability will have a dual-‐port Myricom 10GE network interface in the server.
4. An aUached disk array with a Serial AUached SCSI (SAS) controller capable of several hundred MBytes/sec to local storage.
DYNES: Tier2 and Tier3 -‐ Instrument Design
The Fast Data Transfer (FDT) server connects to the disk array via the SAS controller and runs FDT software developed by Caltech. FDT is an asynchronous multithreaded system that automatically adjusts I/O and network buffers to achieve maximum network utilization. The disk array stores datasets to be transferred among the sites in some cases. The FDT server serves as an aggregator/ throughput optimizer in this case, feeding smooth flows over the networks directly to the Tier2 or Tier3 clusters. The IDC server handles the allocation of network resources on the switch, inter-actions with other DYNES instruments related to network pro-visioning, and network performance monitoring. The IDC creates virtual LANs (VLANs) as needed.
5 Gbps with 2 Controllers
1
2
4
3
• Regional networks require
1. An Ethernet switch
2. An Inter-‐domain Controller (IDC), as shown.
The configura@on of the IDC consists of OSCARS, DRAGON, and perfSONAR just as in the Tier2/ Tier 3 cases. This allows the regional network to provision resources on-‐demand through interac@on with the other instruments
A regional network does not require a disk array or FDT server because they are providing transport for the Tier 2 and Tier 3 data transfers, not ini@a@ng them.
DYNES: Regional Network -‐ Instrument Design
At the network level, each regional connects the incoming campus connection to the Ethernet switch provided. Optionally, if a regional network already has a qualified switch compatible with the dynamic software that they prefer, they may use that instead, or in addition to the provided equipment. The Ethernet switch provides a VLAN dynamically allocated by OSCARS & DRAGON. The VLAN has quality of service (QoS) parameters set to guarantee the bandwidth requirements of the connection as defined in the VLAN. These parameters are determined by the original circuit request from the researcher / application. through this VLAN, the regional provides transit between the campus IDCs connected in the same region or to the global IDC infrastructure.
1 2
DYNES Solware Architecture
7 – 11/1/10, © 2009 Internet2
• As of August 1, 2010, DYNES has been funded! • Revised budget: ~$1.74 million
– Hardware budget unchanged – Personnel budget reduced ~25%
• Success due to strong community support • Going forward, we are looking for strong community engagement
• Need to adapt the plan in response to NSF requests and evolving network infrastructure
• hUp://www.internet2.edu/dynes
DYNES Update
8 – 11/1/10, © 2009 Internet2
• Internet2 received a total of 60 LeUers of Collabora@on – 44 Universi@es (some duplicates)
– 14 Regional Networks – 1 Virtual Organiza@on – 1 Federal Lab – LeUer format specified by solicita@on
• By signing below I acknowledge that I am listed as a collaborator and/or instrument user on this MRI proposal, en@tled “DYNES”, with Eric Boyd as the Principal Inves@gator. I agree to perform the tasks assigned to me, as described in the proposal, and I commit to provide or make available the resources therein designated to me.
– Proposal narra@ve text explained scope of commitment
DYNES Community Support
9 – 11/1/10, © 2009 Internet2
• ICO and Council no@fica@on • Community announcement
– LeUer, website – 2 Community Calls
• Set up external review panel (October, 2010) • CFP (November 1, 2010)
• Calls with individual regionals and with individual campuses (plus associated regional) upon request
• DYNES BoF (November 1, 2010, FMM)
• Site selec@ons announced (January, 2011) • Distributed virtual Instrument building begins in 2011
DYNES Roadmap
10 – 11/1/10, © 2009 Internet2
• Budget change (25% reduc@on in personnel effort) • Site Selec@on by External Review Panel
• Internet2 ION Service Defini@on and Connector Op@ons
DYNES Changes
11 – 11/1/10, © 2009 Internet2
• All sites will need to apply – Applica@on dral posted – Goal is to keep it lightweight, but fair
• External review panel – Composi@on and process s@ll TBD – Likely 6+ members
• CMS overall representa@ve • CMS Tier 3 representa@ve • ATLAS overall representa@ve • ATLAS Tier 3 representa@ve • 2 Non-‐LHC high performance networking community reps • Plus ex-‐officio members: PI and co-‐Pis
• Community feedback encouraged on site selec@on process
DYNES Site Selec@on Process
12 – 11/1/10, © 2009 Internet2
• For the latest announcements concerning DYNES, subscribe to the [email protected] mailing list. – hUp://lists.internet2.edu/sympa/subscribe/dynes.
• To ask ques@ons of the DYNES project team, email dynes-‐ques@[email protected].
• Five documents spelling out the documenta@on process are available: – DYNES: Applica@on Package – DYNES: Overview – DYNES: Regional Network and End-‐Site Par@cipa@on Requirements – DYNES: Criteria for Site Selec@on – DYNES: Frequently Asked Ques@ons (FAQ)
• This document and all of the above are on the DYNES web site: – hUp://www.internet2.edu/dynes
DYNES Site Selec@on Process
13 – 11/1/10, © 2009 Internet2
• The important dates associated with this submission are as follows: – Applica@on Submission Deadline: 30 November 2010
– Applica@on Review Period: 30 November -‐ 15 January 2011 – DYNES Site Selec@on Decision No@fica@on: 31 January, 2011
• Those interested in responding to this request are encouraged to contact the DYNES project team with any ques@ons at any@me prior to the applica@on submission deadline. Please forward any request for addi@onal informa@on to Eric Boyd: – dynes-‐ques@[email protected]
DYNES Applica@on Timeline
14 – 11/1/10, © 2009 Internet2
• How will your par@cipa@on in the DYNES project benefit researchers in your area?
• If your organiza@on is a Regional Network, list the End-‐Sites (Campuses) and associated researchers in your region who you expect will par@cipate in DYNES.
• If your organiza@on is an End-‐Site, list the researchers and campus IT organiza@on at your site who will par@cipate in DYNES. Also list the Regional Network you will collaborate with as part of DYNES project par@cipa@on.
• How will your par@cipa@on in the DYNES project contribute to the larger DYNES project goals? (Please refer to DYNES: Overview and DYNES: Regional Network and End-‐Site Par@cipa@on Requirements)
• Please provide any other informa@on you think is relevant to this applica@on.
DYNES Applica@on Ques@ons
15 – 11/1/10, © 2009 Internet2
• Iden@fy a DYNES point-‐of-‐contact person • Provide deployment plan and drawings • Confirm your willingness to receive and deploy the required DYNES
hardware components • Confirm your willingness to configure the equipment. For “standard” DYNES
components • Describe your plan to enable DYNES control capabili@es into the network
infrastructure • Confirm your willingness to provide local maintenance where physical
access is required • Describe your plan to work in a collaboraJve manner with DYNES end-‐sites
to ensure the regional network and associated end-‐site designs work well together to enable the DYNES func@ons
• Iden@fy DYNES Regional Team • Please include a leUer of support from every end-‐site applying to the DYNES
project indica@ng that they have reviewed the regional network implementa@on plan
DYNES Regional Applica@on Requirements
16 – 11/1/10, © 2009 Internet2
• Iden@fy a DYNES point-‐of-‐contact person • Provide deployment plan and drawings for how the DYNES equipment will be placed in the End-‐
Site facility • Confirm your willingness to receive and deploy the required DYNES hardware components • Confirm your willingness to configure the equipment • Describe your plan to enable DYNES control capabili@es into the network infrastructure • Confirm your willingness to provide local maintenance where physical access is required • Describe your plan to work in a collaboraJve manner with DYNES Regional Networks to ensure
the regional network and associated end-‐site designs work well together to enable the DYNES func@ons.
• Iden@fy DYNES Regional Team • Please include a leUer of support from the regional network you expect to connect through
indica@ng that they have reviewed your proposed end-‐site network implementa@on plan • Please include a leUer of support from your campus IT organiza@on indica@ng that they have
reviewed and support your proposed end-‐site implementa@on plan and that they have reviewed the proposed regional network implementa@on plan
DYNES End Site Applica@on Requirements
17 – 11/1/10, © 2009 Internet2
• Baseline Regional Configura@on – Ethernet switch, one of the following (Models below are under considera@on but not
yet final) • Dell PC6248 (48 1GE ports, 4 10GE capable ports (Select SFP+, CX4 or op@cal) • Dell PC8024F (24 10GE SFP+ ports, 4 “combo” ports suppor@ng CX4 or op@cal) • Custom variant for sites needing a different alterna@ve
– Inter-‐domain Controller (IDC) (Dell R610 1U, 1GE connec@vity) • Baseline End Site Configura@on
– Ethernet switch, one of the following (Models below are under considera@on but not yet final)
• Dell PC6248 (48 1GE ports, 4 10GE capable ports (Select SFP+, CX4 or op@cal) • Dell PC8024F (24 10GE SFP+ ports, 4 “combo” ports suppor@ng CX4 or op@cal) • Custom variant for sites needing a different alterna@ve
– Inter-‐domain Controller (IDC) (Dell R610 1U node, 1GE connec@vity) – A Fast Data Transfer (FDT) server. Sites with 10GE throughput capability will have a
dual-‐port Myricom 10GE network interface in the server. One op@on is Dell R510 which can accommodate Gen2.0 card x8 card along with 12 disks for storage.
– An aUached disk array with an LSI or equivalent Serial AUached SCSI (SAS) controller capable of several hundred Mbytes/sec to local storage.
DYNES Baseline Configura@on
18 – 11/1/10, © 2009 Internet2
• It is not required that the standard DYNES components be u@lized to realize the dynamic network capabili@es. – For instance, rather than receiving the DYNES standard ethernet
switch, it may be preferable for an End-‐Site to receive some interface cards for an exis@ng network element or other items to be integrated into exis@ng infrastructure.
• A customized solu@on will have addi@onal solware configura@on and deployment considera@ons.
• Ques@ons? dynes-‐ques@[email protected] • As part of the applica@on review process we an@cipate that
addi@onal informa@on may be requested by the site selec@on panel.
DYNES Custom Configura@ons
19 – 11/1/10, © 2009 Internet2
• The exact budget details and plan will be determined aler review of the par@cipa@on applica@ons.
• The general plan is to provide the DYNES baseline equipment to the Regional Networks and End-‐Sites as described above in the other DYNES project documents
• If a site would prefer to u@lize an equivalent amount of funds to purchase equipment which may extend their current infrastructure capabili@es in lieu of receiving the DYNES baseline equipment
• Equivalent fund value will also be determined aler review of applica@ons • Expected equivalent fund value to be:
– 10 GE connected end-‐sites: $18k – 1 GE connected end-‐sites: $15k – Regionals: $26k
• These numbers are based upon DYNES budget planning which assumed 40 end-‐sites distributed 45% 10GE end-‐sites, 55% 1GE end-‐sites and 14 regionals.
• If your applica@on includes a request for equipment other then the DYNES standard equipment, please iden@fy model numbers and provide cost es@mates.
DYNES Budget
20 – 11/1/10, © 2009 Internet2
• It’s been a year since the proposal was wriUen • In that @me, Internet2 has:
– Migrated ION service from dedicated Ciena network onto the Juniper MX plazorm
– Updated the Connector Op@ons • Old way: Contrac@ng for an IP connec@on gives you a second ION
connec@on of the same size • New way: For a modest fee increase, connector can purchase 2 connec@ons
to use as they choose – Widespread adop@on to date by connectors
• Prac@cal impacts – Many (but not all) connectors are using second connec@on for
redundant IP/CPS connec@vity • These changes change how DYNES will be implemented
• Strawman versions of various connec@on strategies follow
Changes to ION Service and Impact on DYNES
21 – 11/1/10, © 2009 Internet2
DYNES Minimum Connec@on Scenario A
22 – 11/1/10, © 2009 Internet2
DYNES Minimum Connec@on Scenario B
23 – 11/1/10, © 2009 Internet2
DYNES Single Campus Link Scenario
24 – 11/1/10, © 2009 Internet2
DYNES Dual Campus Link Scenario
25 – 11/1/10, © 2009 Internet2
DYNES Full Dynamic Control Scenario
26 – 11/1/10, © 2009 Internet2
• PI: Eric Boyd; Co-‐PI: Ana Preston – [email protected] – [email protected]
• Division of Responsibili@es: – Eric: Execu@on of IRIS and DyGIR proposals, direct engagement with
other PIs on deliverables – Ana: Overall coordina@on, Internet2 engagement with IRNC projects
and partners • IRIS: Interna@onal Research Instrumenta@on System
– Support performance monitoring via perfSONAR-‐PS – The “performance / perfSONAR” one
• DyGIR: Dynamic Gateways for Interna@onal Research – Support automa@c provisioning of circuits via OSCARS / DRAGON – The “dynamic circuits / DCN / IDC / ION/ OSCARS” one
IRIS and DyGIR
27 – 11/1/10, © 2009 Internet2
• Objec@ve: provide the infrastructure necessary for the iden@fica@on, diagnosis and eventual correc@on of network performance problems for paths traversing IRNC links
• IRIS will provide a solware framework to enable performance monitoring services on IRNC links • Produce a set of easy-‐to-‐install, tailored solware packages of the
perfSONAR-‐PS solware suite for use by the IRNC link operators
• Develop new func@onality specific to interna@onal exchange points • Work with the IRNC link awardees to help them deploy the solware
28 – 11/1/10, © 2010 Internet2
IRIS
• Objec@ve: enable researchers to reserve dedicated bandwidth over IRNC links • Allow for more distributed collabora@on
• DyGIR will provide a prototype solware framework to enable dynamic circuit services for a pair of IRNC links • Produce a set of easy-‐to-‐install, tailored solware packages for the
OSCARS solware suite
• Prototype new func@onality specific to interna@onal exchange points
• Work with 2+ of the IRNC link awardees to help them deploy the prototype solware
29 – 11/1/10, © 2010 Internet2
DyGIR
By 2013 … • Integrated end-‐to-‐end performance monitoring is a “normal part
of networking” – Preserve local autonomy and enable diversity of networking
prac@ces
– Beginning to see monitoring directly supported on networking devices
• Integrated end-‐to-‐end automa@c provisioning of circuits is a “normal part of networking” – Preserve local autonomy and enable a diversity of networking
technologies and prac@ces
Aggregate Goal of IRIS, DyGIR and DYNES
30 – 11/1/10, © 2009 Internet2
• Eric Boyd – [email protected]
• DYNES – dynes-‐ques@[email protected]
Ques@ons
31 – 11/1/10, © 2009 Internet2
