Taiwan Advanced Research and Education Network (TWAREN) -

Current status & Future Plan

Dr. Te-Lung LiuDr. Te-Lung LiuResearcherResearcher

National Center for High-Performance ComputingNational Center for High-Performance Computingtlliu@nchc.narl.org.twtlliu@nchc.narl.org.tw

22

Outline

TWAREN Network OverviewDevelopment and Research Technologies

33

TWAREN Network OverviewDevelopment and Research Technologies

44

TaiWan

Advanced

Research and

E

ducation

Network

TWARENTWAREN

55

What is TWARENWhat is TWARENA physical network serves multiple purposes and logical networks

TANet, connects to commodity Internet

TWAREN research network

experiment, testbed, special research

Provisioning services on multiple layers

L1 lightpaths

L2 VLAN

L3 IP

has been successfully migrated from old backbone in Oct 2006

66

• 4 core nodes

• 20G backbone

• 12 GigaPops

• Connects HPC resources in

North and South Taiwan

TWAREN Architecture

77

TWAREN is part of “Challenge 2008”, a comprehensive six-year national development plan formulated by the government

Build a highly reliable, stable and flexible R&E network for academic and research community in TW

Provide advanced network services to satisfy the needs of academia field in TW.

Increase the International and domestic collaboration

Future infrastructure drives today’s research agenda

Goals of TWAREN

88

TWAREN GigaPoPsTWAREN GigaPoPs

99

TWAREN Services

■ Broadband Connection Service

■ International Research Network Transit (Internet2)

■ Measurement / Network Management

■ Multimedia / Multicast■ Lightpath provisioning■ Virtual Private Network(VPN)■ Native IPv6 Service ■ Internet access

MCU Proxy Server SourceForge File Download Center Consultation Applications support

1010

High reliability & availability (99.9% 99.99%) fault tolerance

automatic protection if possible

automatic failure detection and locating

Better performance: minimum number of routers between GigaPoPs

Flexible: can be easily and quickly to set up a logical network per user’s request

People skills: Optical network OAM

TWAREN Achivements

1111

STM-64STM-16

NSYSU

NCHU

NCTU

NTHU

ASCC

NCKUCCU

TP

HC

TN

TC

NIU

NDHU

NCU

NTU

ONS15600

ONS15454

Optical Backbone

1212

Interconnecting with L2/L3 devicesSTM64STM1610GEGE

NSYSU

NCHU

NCTU

NTHU

ASCC NCCU

NCKU CCU

Taipei

Hsinchu

Tainan

Taichung

NCNU

NIU

NDHU

NHLTC

NTTU

NCU

ONS15600

ONS15454

GSR

7609

6509

3750

NTU

1313

Protection Mechanism

Circuit break:

2 levels of protection

By carriers: SDH protected

By architecture:Link b/w core nodes: VLAN are reconfigured with

rapid spanning-tree protocol. (5s)Link b/w GigaPOP and core node: the backup

SNCP lightpaths are configured for automatic fail-over. (50ms)

1414

Protection Mechanism

Equipment protectionCore node failure: Manually configure emergency

lightpaths to re-route traffic from affected GigaPoPs to another core node. Emergency lightpaths need to be designed and documented.

GigaPoP failure: Spare line cards

1515

STM64STM1610GEGE

NSYSU

NCHU

NCTU

NTHU

ASCC NCCU

NCKU CCU

Taipei

Hsinchu

Tainan

Taichung

NCNU

NIU

NDHU

NHLTC

NTTU

NCU

ONS15600

ONS15454

GSR

7609

6509

3750

NTU

Normal Traffic Flows

1616

STM64STM1610GEGE

NSYSU

NCHU

NCTU

NTHU

ASCC NCCU

NCKU CCU

Taipei

Hsinchu

Tainan

Taichung

NCNU

NIU

NDHU

NHLTC

NTTU

NCU

ONS15600

ONS15454

GSR

7609

6509

3750

NTU

In case of circuit break...

1717

STM64STM1610GEGE

NSYSU

NCHU

NCTU

NTHU

ASCC NCCU

NCKU CCU

Taipei

Hsinchu

Tainan

Taichung

NCNU

NIU

NDHU

NHLTC

NTTU

NCU

ONS15600

ONS15454

GSR

7609

6509

3750

NTU

In case of core node failure...

1818

NOC (Network Operation Center) Located at NCHC southern business unit in Tainan Science ParkGoals: To ensure the 7x24 network operationMajor works:

Providing 7x24 network maintenance and operation Enhance the security capacity Provide network service

Peering Light path provision

Network architecture design

TWAREN NOC

TWAREN NOC

1919

TANet VPN

TANet VLAN

NTU6509

NCCU6509NDHU6509

TP7609CL2 Switch

TC7609CL2 Switch

HC7609CL2 Switch

TN7609CL2 Switch

NCHU6509

NTHU6509

NCTU6509

CCU6509NTTU6509

NCKU6509

NSYSU6509

NHLUE6509

TN7609P

MOEcc6509

TC7609

HC7609

NCU6509

One Subnet L2 VLAN

2020

TWAREN Research VPN

ResearchVLAN

NTU7609P

ASCC7609P

NDHU7609P

TP7609CSwitch

TC7609CSwitch

HC7609CSwitch

TN7609CSwitch

NCHU7609P

NCNU7609P

NTHU7609P

HC7609P

NCTU7609P

CCU7609PTN7609P

NCKU7609P

NSYSU7609P

TN12816R

TP12816R

TC12816P

HC12816R

NCU7609P

TN12816P

TP12816P

TC12816R

HC12816P

NIU7609P

TAIWANLight TANet (MOEcc6509)

TWGATE Internet

ISP Peering

ASCC APAN

TAIWANLightTAIWANLight

ISP Peering

iBGP RR

iBGP RR

2121

VPN ServicesMultipoint-to-Multipoint Layer2 VPN (VPLS)Multiple VPNs over single architectureCross-area campuses and offices can

be connected within single administrative domainProvide dynamic creation of VPNs for National-wide integrated projects

User-based SSL VPN AccessAccess to different VPN according to login name and password authenticationResearchers and Professors could access their own research resources from home or outside

2222

VPLS Architecture

2323

User-Based SSL VPN Access

SSL VPN

SSL VPN

TWARENVPLS

Backbone

Core node@ HsinChu

Org 1

Org 2

Org 3

Org n

。。。

Web Browser

Users

Core node@ Tainan

2424

TWAREN’s International Connections

Pacific Crossing to USA’s west coast upgraded to 5 Gb/s Connections between LA, Palo Alto, Chicago, and New York are 2.5 Gb/s Connects to the rest of the world via the U.S.’s Abilene NetworkConnection expanded to Europe in 2006 (IEEAF donated 622 Mbps of bandwidth/fiber optic cable)

2525

NCU-15454

TP-15600

TN-15600 TC-15454

HC-15600HC-15454

TN-15454

NCHU-15454CCU-15454NCKU-15454NCSYSU-15454

ASCC-15454

NIU-15454

NDHU-15454

NTU-15454NCTU-15454

NTHU-15454

TP-15454

TWARENOptical Network

Palo Alto 15454

Chicago 15454

LA 15454

NY15454

TAIWANLight

Combined TWAREN/TAIWANLight Lambda Testbed

2626

TWAREN’s International Peerings

TWAREN made peerings with international NRENs at Los Angeles, Chicago, New York and Seattle (through Pacific Wave).

2727

TWAREN’s Direct Peerings Coverage

TWAREN's direct peering covers most area in America, Asia, Australia and New Zealand, and will soon be expanded to Europe.

2828

TWAREN/TAIWANLight and GLIF

TWAREN is a member of GLIF (Global Lambda Integrated Facility)TAIWANLight is an official optical exchange - GOLE (GLIF Open Lightpath Exchange)

2929

TWAREN Network OverviewDevelopment and Research Technologies

3030

Future Internet Testbed @ Taiwan

3131

Future InternetThere are many serious limitations in current Internet. ScalabilitySecurityQoSVirtualization

Future Internet is a summarizing term for worldwide research activities dedicated to the further development of the original Internet. (From Wiki)

3232

Future Internet Testbed

For innovations and researches in Future Internet, the testbed requires some advanced concepts:ProgrammabilityVirtualizationEnd-to-end slice

3333

OpenFlow

Make deployed networks programmableMakes innovation easierNo more special purpose test-bedsValidate your experiments on production network at full line speed

3434

TWAREN OpenFlow Testbed in 2010

TWARENL3

Network

NOX

OpenFlow

Switch

iCAIR

Capsulator

CapsulatorCapsulator

OpenFlow

Network@NCKU

OpenFlow

Network@KUAS

NCHC

NCKU and KUAS are pilot universities that connected with the TestbedThe OpenFlow Testbed is extended to iGENI@iCAIR Capsulator (Ethernet-in-IP tunnel) is used to emulate pure L2 network for OpenFlow

34

3535

TWARENVPLS

KUAS

35

OpenFlow

Switch

NCKU

OpenFlow

Switch

CHT-TL

OpenFlow

Switch

NCU

OpenFlow

Switch

NCHC

OpenFlow

Switch

NTUST

OpenFlow

Switch

OpenFlow

Switch

Capsulator

Capsulator

TWAREN OpenFlow Testbed in 2011

NTUST, NCU and CHT-TL joined the Testbed.For TWAREN connectors (NCKU, KUAS and NCU), a dedicated VPLS VLAN is allocated for better transmission performance.

lightpath

3636

Emulab/ProtoGENI TestbedTWISC (Taiwan Information Security Research and Education Center) operats 206 nodes of Emulab Testbed in Taiwan.Third largest Emulab in the

worldTestbed@TWISC is operated by NCKU team and co-located in NCHC

A portion of the testbed is planned to try ProtoGENI test with University of Utah.A lightpath is provisioned

between NCHC and iCAIR shared by both OpenFlow and Emulab/ProtoGENI

Experiment Network

Experiment Network

Experiment Switch

BOSS OPS

Control Network

Experiment Network

Experiment Network

Experiment Switch

BOSS OPS

Control Network

BOSS OPS

Control Network

BOSSBOSS OPSOPS

Control Network

FirewallFirewall

FirewallFirewall

FirewallFirewall

FirewallFirewall

FirewallFirewallFirewallFirewall

FirewallFirewall

36

3737

Lightpath and VLAN setup

NCHCOF sw A OF sw B

iCAIR

7609V NCKU

Vlan 462Vlan 1548

7609P@HC

NCKU

7609V NCKU EE

Emulab/ProtoGENI – Vlan 462

Lab

Vlan 2782

NCKU OF (with iCAIR) – Vlan 1548

Vlan 462Vlan 1548

Trunk

Vlan 462Vlan 2782

iCAIR OF (with NCKU) – Vlan 2782

Trunk port

Vlan 2782

7609P@TNEmulab/ProtoGENI – Vlan 462

Vlan 462Vlan 2782

Vlan 462Vlan 2782

Emulab @NCHC

Vlan 1548Vlan 1555

Vlan 1548Vlan 1555

Vlan 462

37

3838

iGENI - Taiwan Integrated Research Network

38

3939

Multi-Domain OpenFlow Management

Each network domain has its own OF Controller Each Controller manages topology and flow provisioning inside the domain

Inter-domain flow could be made by connecting partial flows provisioned by controllers of each cloudLack of global view for inter-domain flowsNo loops allowed for inter-domain topologyDifficult to support QoS or SLA functions across

domains

Inter-domain topology auto-discovery is required for multi-domain management

39

4040

OpenFlow Controller just only knows its directly connected switches.

ENVI is a useful GUI tool to show OpenFlow topology under single controller.

40

Controller1

OFA OFB OFC OFD

OFA OFB

Topology of Domain1

Controller2

OFC OFD

Topology of Domain2UI

Domain

Inter-Domain Topology Discovery (I)

4141

We add additional contents in LLDP packet to let Controllers have its neighbors’ connectivity details.

ENVI is also modified to show the whole topology.

41

Controller1

OFA OFB OFC OFD

Controller2

OFA OFB OFC OFD

UI

Domain

Topology of Domain1 & 2

Inter-Domain Topology Discovery (II)

4242

Results

42

Physical OpenFlow Network Topology

Multi-Domain Network Topology shown in GUI

4343

GLIF & SC11 Demo

Joint Demo among NCHC/TW, iCair/US, and CRC/Canada

4444

Information Security Activity Detection over High-Speed Backbone

4545

Security Detection over High-Speed Backbone

Normally, we don’t install IDS/IDP in backbone for performance issue.IDS/IDP are placed at user’s local sites

Backbone traffic is hard to mirroring due to its large amount and high-speedIt’s impossible to do packet analysis Packet header analysis is available with

Netflow/sFlow

Information Security Activity Detection over High-Speed BackboneIntegrate fast packet header analysis with

attack information from user’s local site

4646

Invasion and attack info from user’s local sites

Users’ IDS/IDP

Users’ HoneyPot

Users’ Log analyzer

Security Detection@Backbone

Collect

Search

Orientation Trace-back

Notification Block

Backbone’s Netflow data

Netflow Data from Backbone/User Routers

Users’ Netflow data

Notify User with Suspicious Activities

Backbone network, peering partner, User network

System Architecture

4747

Design Concepts

Distributed ComputingFor monitoring netflow data in real-time

Fast SearchEffective Tree-Searching algorithm

ExpandableSimply add more machines when larger data analysis is required

Remote BackupSeparate different computing nodes in order to provide robust analysis service

Single PortalAll input can be submit to single portal with Global Server Load-Balancing technology

Cooperate with Researchers/DevelopersWill design an open API for developers to contribute their own ideas

4848

Design Blocks

Controller 2

Distributor 1 Distributor 2

Filter 1 Filter 2 Filter 3 Filter N

Analyzer 1 Analyzer 2 Analyzer 3 Analyzer N

Controller 1

Router1 Router2 Router3 RouterN

IDS/IDP

Honey...Syslog

IP Port Type Analyzer Analyzer Port ……

A.A.A.A 1234 botnet 1 3333

B.B.B.B 4321 Fake-IP 2 4444

C.C.C.C 1122 Cracker 3 5555

Blacklist

Analyzer 1 P3333

Analyzer 2 P4444

Analyzer 3 P5555

Blacklist Search Tree

Update Blacklist

Update Search Tree

Netflow packet

Netflow packet

Matched Netflow raw Matched Netflow raw

Netflow packet

result result result

4949

Numerical Results of Tree Creation

5050

Numerical Results of Real-time Matching

5151

5252

5353

5454

5555

5656

5757

5858

Thank You !For more information, please see :

www.twaren.net

- 2011 -