Upload
others
View
7
Download
0
Embed Size (px)
Citation preview
Advanced Communication Exchange Facilities:
Architecture, Services, and Design, Technologies,
Implementation and Operations (i.e., Creating 21st Century
Communications Services and Technology)
Joe Mambretti, Director, ([email protected])
International Center for Advanced Internet Research (www.icair.org)
Northwestern University
Director, Metropolitan Research and Education Network (www.mren.org)
Hong Kong Workshop on Next Generation Open Communication
Exchanges for Science Research
University of Hong Kong
January 18, 2012
Creating the Future of Communications
• Close Examination of Emerging and Future Requirements
• New Architecture
• New Services
• New Technology
• Testing With Simulations and Models
• Testing via Network Research Testbeds At Scale!
• Best Window To the Future = Data Intensive Science Research
• 30+ Year History of Communication Innovations Has Been Driven
Primarily By Data Intensive Sciences
• These Sciences Encounter Technology Issues Many Years Before
Other Domains
Sloan Digital Sky
Survey
www.sdss.org
Globus Alliance
www.globus.org
LIGO
www.ligo.org TeraGrid
www.teragrid.org
ALMA: Atacama
Large Millimeter
Array
www.alma.nrao.edu
CAMERA
metagenomics
camera.calit2.net
Comprehensive
Large-Array
Stewardship System
www.class.noaa.gov
DØ (DZero)
www-d0.fnal.gov
ISS: International
Space Station
www.nasa.gov/statio
n
IVOA:
International
Virtual
Observatory
www.ivoa.net
BIRN: Biomedical
Informatics Research
Network
www.nbirn.net
GEON: Geosciences
Network
www.geongrid.org
ANDRILL:
Antarctic
Geological
Drilling
www.andrill.org
GLEON: Global Lake
Ecological
Observatory
Network
www.gleon.org Pacific Rim
Applications and
Grid Middleware
Assembly
www.pragma-
grid.net
CineGrid
www.cinegrid.org Carbon Tracker
www.esrl.noaa.gov/
gmd/ccgg/carbontrack
er
XSEDE
www.xsede.org
LHCONE
www.lhcone.net
WLCG
lcg.web.cern.ch/LCG/publi
c/
OOI-CI
ci.oceanobservatories.org
OSG
www.opensciencegrid.org
SKA
www.skatelescope.o
rg
NG Digital
Sky Survey
ATLAS
Compilation By Maxine Brown
A Brief Overview of the Future of
Communications • Transition From Monolithic, Static Services and Infrastructure To
Ad Hoc Specialized, Customized, and Even Personalized
Resources:
– E.g., Personal Global Networks
– Individualized Communication Services
– Note: Historic Progression From Monolithic To
Individualized Environments (Part of the IT DNA)
• Personal Computer vs Mainframe
• Smart Phone vs Personal Computer
• Intelligent Devices vs Smart Phones
• Etc.
– Advanced Network Testbeds Are Being Designed And
Implemented To Accomplish These Goals, Including By The
National Science Foundation
Invisible Nodes,
Elements,
Hierarchical,
Centrally Controlled,
Fairly Static
Traditional Provider Services:
Invisible, Static Resources,
Centralized Management,
Highly Layered
Distributed Programmable Resources,
Dynamic Services,
Visible & Accessible Resources,
Integrated As Required, Non-Layered
Limited Services, Functionality,
Flexibility, Expandability
Unlimited Services, Functionality,
Flexibility, Expandability
Paradigm Shift – Ubiquitous Services Based on Large Scale
Distributed Facility vs Isolated Services Based on Separate
Component Resources
Releasing the Fully Potential of Digital Technologies
Next Phase = A Global Platform For Customized
Networks, Led By Science Networks
• Migration from Designing, Creating, Implementing,
Operating Private Networks As Individual Projects To
Designing, Creating, Implementing, Operating
Distributed Environment (Platform) Within Which Such
Individual Networks Can Be Readily Created
• Existing Building Blocks. Prototypes, Including the
StarLight National/International Communications
Exchange Facility, Metropolitan Research and
Education Network (MREN, Regional), National Lambda
Rail (NLR, National), Global Lambda Integrated Facility
(GLIF, International), GLORIAD (Global Ring for
International Application Development, International)
Distributed Programmable Network
Platforms
• Programmable Network Platforms = Capabilities for Creating
Instant New and Enhanced Services vs Legacy Multi-Year Schedule
of Design, Development, and Deployment
• Advanced =
– Dynamic vs Static
– Highly Customizable, Including At Edge
– High Level of Abstractions, Including APIs
– Flexible Middleware Processes That Can Be Dynamically Provisioned
– Highly Distributed Processes vs Centralized Command and Control
– Etc
• Programmable =
– All Resource Elements As Objects
– Discoverable/Integratable
– Programmability Extending To Hardware Components
– Rich Semantics for Resource Discovery and Integration
Ad Hoc Specialized Networks
• APN Approach: Create Private Network
– Based On Controllable Resources, e.g., Private Optical
Fiber/Lambdas/L2 VLANs
– Advanced Control Planes/Advanced Control Frameworks
– Advanced Management Planes
• Possible To Obtain Leverage From
– IaaS/NaaS
– PaaS
– SaaS
– OaaS
– XaaS
• Additional Leverage from
– Dynamic Clouds Closely Integrated With Dynamic Networks (Ref TransCloud,
Note Demo At GEC10)
Control Framework Architecture and
Implementations
• Control Frameworks Enable Resources Programming
• Many Approaches To Control Framework Architecture Currently
Being Created By Network Research Communities
• Many More Will Be Created By Research Communities
• One Approach Currently Being Transitioned From Research
Communities To A Standard Through a Joint Project Between the
GLIF and the Open Grid Forum (a Standards Organization)
• Network Services Interface (NSI)
• Based In Part On GLIF GOLE Control Frameworks and an API
(FENIUS)
t
Fenius GLIF
Demonstrations
Global Lambda Grid Workshop
SC10
HK GLIF Technical Workshop
Automated GLIF Open Lambda Exchange
Demonstration at SC11
Other Network Control Frameworks
• Many Other Network Control Frameworks Are
Being Created (About 24 Major Initiatives)
• The US National Science Foundation Has
Established an Initiative That Incorporates 4
Additional Major Frameworks
• The Initiative Is The Global Environment for
network Innovations (GENI)
National Science Foundation’s Global
Environment for Network Innovations (GENI)
• GENI Is Funded By The National Science Foundation’s Directorate
for Computer and Information Science and Engineering (CISE)
• GENI Is a Virtual Laboratory For Exploring Future Internets At
Scale.
• GENI Is Similar To Instruments Used By Other Science Disciplines,
e.g., Astronomers – Telescopes, HEP - Synchrotrons
• GENI Creates Major Opportunities To Understand, Innovate and
Transform Global Networks and Their Interactions with Society.
• GENI Is Dynamic and Adaptive.
• GENI Opens Up New Areas of Research at the Frontiers of Network
Science and Engineering, and Increases the Opportunity for
Significant Socio-Economic Impact.
• GENI Is Both
– A Research Instrument and
– A Prototype of Future Programmable Network Platforms
iGENI: The International GENI
• The iGENI Initiative Is Designing, Developing, Implementing, and Operate a Major New National and International Distributed Infrastructure.
• iGENI Is Placing the “G” in GENI Making GENI Truly Global.
• iGENI Is a Unique Distributed Infrastructure Supporting
Research and Development for Next-Generation Network
Communication Services and Technologies.
• This Infrastructure Has Been Integrated With Current and
Planned GENI Resources, and Is Being Operated for Use by
GENI Researchers Conducting Experiments That Involve
Multiple Aggregates At Multiple Sites.
• iGENI Infrastructure Connects Its Resources With Current GENI
National Backbone Transport Resources, With Current and
Planned GENI Regional Transport Resources, and With
International Research Networks and Projects.
•
iGENI/GCDnet National/International Fabric
SC11 SCinet Research Sandbox OpenFlow Demonstrations
Seattle, Wash November 2011
TransCloud Experiments Alvin AuYoung, Andy Bavier, Jessica Blaine, Jim Chen, Yvonne Coady, Paul Muller, Joe Mambretti,Chris
Matthews, Rick McGeer, Chris Pearson, Alex Snoeren, Fei Yeh, Marco Yuen
Demo: http://tcdemo.dyndns.org/
Example of working in the TransCloud [1] Build trans-continental applications spanning clouds: • Distributed query application based on Hadoop/Pig
• Store archived Network trace data using HDFS • Query data using Pig over Hadoop clusters
[2] Perform distributed query on TransCloud, which currently spans the following sites:
• HP OpenCirrus • Northwestern OpenCloud
• UC San Diego • Kaiserslautern
• Use By Outside Researchers? Yes
• Use Involving Multiple Aggregates? Yes
• Use for Research Experiments? Yes Also Ref: Experiments in High Perf Transport at GEC 7
Digital Media TransCoding Demonstration
• TransCloud: Advanced Distributed
Global Environment Enables Dynamic
Creation of Communication Services,
Including Those Based on Rapid
Migration of Virtual Network
and Cloud Resources
•TransCloud: Set of Protocols,
Standards, Management Software
Enables Interoperation of Distinct
Cloud and Network Resources
•Example: Dynamic Cloud+Dynamic
Network for Digital Media Transcoding
Using Single Platform vs Multiple
Infrastructures
Transcoding
Cloud 1 Transcoding
Cloud 2
Video
Sources
OpenFlow
Switches Transcoding
Cloud 3
JGN-X Network Topology
National Institute for Information
Communication Technology (NICT)
National International Testbed
10 Gbps Connection via
GLIF/TransLight StarLight
CNIC/Chinese Academy of Sciences
iCAIR Signing Ceremony
Another Driver For Programmable Network Platforms With
Advanced Control Frameworks
• Advanced Digital Media
• Many Services Have Requirements Similar To
Large Scale Science
• Visualization Is a Key Application.
• Currently, Many Major Innovations In Advanced
Digital Media
Programmability + High Capacity
• Advanced Programmability Provides A Wide Range of
Capabilities
• As Basic Resources, Network Resource Objects Can Be
Encapsulated and Self-Advertised
• Also, Capacity Is a Key Resource
• Currently, 10 Gbps Is Basic Resource Unit
• In 2012, 100 Gbps Will Be a Basic Programmable
Resource Unit
• 100 Gbps Not Merely As A capability For Aggregation of
Multiple Small Flows, But As a Single Programmable
Resource, e.g., For a Single Stream
StarWave: A Multi-100 Gbps Facility
• StarWave, A New Advanced Multi-100 Gbps Facility and Services
Will Be Implemented Within the StarLight International/National
Communications Exchange Facility
• StarWave Is Being Funded To Provide Services To Support Large
Scale Data Intensive Science Research Initiatives
• Facilities Components Will Include:
• An ITU G. 709 v3 Standards Based Optical Switch for WAN
Services, Supporting Multiple 100 G Connections
• An IEEE 802.3ba Standards Based Client Side Switch,
Supporting Multiple 100 G Connections, Multiple 10 G
Connections
• Multiple Other Components (e.g., Optical Fiber Interfaces,
Measurement Servers, Test Servers
DOE Office of Science ESnet ANI 100 Gbps Testbed
StarLight
Evaluations/Demonstrations of 40-
100 Gbps IPv4/IPv6
Disk-to-Disk File Transfer
Performance Across LANs and
WANs
CA*net/Ciena/StarLight/iCAIR 100 Gbps
Testbed Sept-Oct 2010, Oct 2011
Source: CANARIE
100 Gbps
~850 Miles, 1368 Km
CVS 2010/11 Terrestrial Demo Content & Applications – 100G
Chicago
Ottawa MEN LAB 10
100G Connectivity from Ciena
Ottawa to Starlight Gigapop in
Chicago.
Ability to show simultaneous
data-flows sourced from
major collaborators.
MEN LAB 10 Tile Display
Canadian Brain Imaging
Network
CBRAIN portal & Metadata
Montreal
Selected Ad Hoc Testbeds and Protoypes
• Ad Hoc Network Science Research Testbeds
• New Sevices and Technology Experiments
• Service Prototypes
• New Architecture Protypes
• New Science Network Prototypes
FILE
Ampath
GOLE USP
T-LEX
GOLE
U
Mackenzie
vlan 2712
vlan 2711 C-wave
vlans 2711/2
FILE
Phospherous
StarLight Facility Chicago
Ampath
EU Asia
GreenLight International
• GreenLight explores how researchers can take advantage of data
centers linked by high-speed networking in an era of carbon-thrifty
computing
• Partnered with Canada’s GreenStar network – researching a “follow the
sun/wind” distributed computing environment.
• Recent studies migrating virtual machines to green energy sites
indicate that 100 Gb/s networks are far superior to 10 Gb/s to make this
transparent.
10GE CAVEwave
on the National LambdaRail
UIC
NU
NCSA
UCSD
Summary
• Transition From Monolithic, Static Services and Infrastructure To
Ad Hoc Specialized, Customized, and Even Personalized
Resources:
– Individualized Communication Services
– Part of Historic Progression From Monolithic To Individualized
Environments (Part of the IT DNA)
– Design, Implementation, Operation of Distributed Network
Platforms
– Integrated With Sophisticated Control Frameworks
– Addressed via Advanced Programming Methods
– Provided With Extremely High Capacity
– Capabilities Will Be Centered At Advanced Communication
Exchanges
www.startap.net/starlight
Thanks to the NSF, DOE, DARPA
Universities, National Labs,
International Partners,
and Other Supporters