1

Network topologies

Pehr SödermanPehrs@kth.se

KTH-NOC/CSC/NADA

2

This lecture● We will cover some common network

topologies● We will have a look at several existing networks● We will talk about some common terms and

issues when dealing with network design● We will drift into peering and transit traffic, this

is discussed more closely in IPRO2

3

Fundamentals● The structure of the network should suit the

needs– Current needs– Future needs

● Redesigning a network is complex● Don't forget to plan for growth

– This is the most common mistake!● Do a good job from the start!

4

Religion● Design of networks is a very religious matter● You will frequently hear that “We do it this way”● Try to keep an open mind and don't get in fights

– At least not until you can recognize the camps

5

Common requirements● Uptime and fault tolerance● Performance● PoPs● Peering● Cost

6

The 3 tier topology● Edge: Connecting individual systems● Aggregation: Collecting many streams● Core: Center of the network● Almost all networks are built this way

7

PoP (Points of Presence)● This is how most ISP's design their network:

– Choose the PoPs– Design a core network– Deploy PoP where needed– Connect customers

● It can be used for most company networks● A modular design makes it much easier to

maintain and upgrade the network

8

Building tools: Core Connections● Ethernet over fiber: 10 gbit● T-Carriers: 565 mbit● Sonet optical carriers: 15.9 gbit● Range is mostly a matter of amplifying

9

Building tools: Edge Connections● DocSis2: 40/30 mbit● DocSis3: 160/120 mbit● ADSL2+: 24/3.5 mbit● VDSL: 100mbit● IEEE 802.11 (WiFi): 54 mbit● IEEE 802.16 (WiMax): 70 mbit● Ethernet over copper: 1 gbit

10

Dark fiber● When laying fiber in the ground the price of

actual fiber is very small● Most companies therefor add lots of extra fiber● This fiber is called “dark fiber”● You can frequently buy dark fiber and rack

space from an ISP● This way you can get a dedicated connection

almost anywhere in the world

11

Core routers● The center of the modern internet● Very very fast● Not very smart, but runs BGP● Dedicated hardware for everything● Very expensive● Cisco CRS-1● Juniper T-Series

12

Aggregation routers● To connect the major parts of your network● Large routers

– Many interfaces– High capacity backplanes

● Typically “smart” routers● Used for advanced network management● Cisco 7000 series● Juniper mx series

13

Edge routers● To connect end users to your network● Typically specialized for various technologies● Relatively low speed● Access control and filtering functionality● Cheap● Cisco 2000 series● Juniper J and M series

14

Who handles the Internet traffic?● Traffic providers are typically split in three tiers● Tier 1 Handles transit traffic globally

– AT&T, GBLX, Level 3, Verizon, NTT, Qwest, Sprint, Savvis and ATDN

● Tier 2 Handles regional traffic with peering, but uses a tier 1 for global transit– Telia, BT, Tele2 etc

● Tier 3 handles local traffic and lets a tier 2 do their peering. Most companies are here.

15

Who pays for the traffic?● Who pays, and how much, is only a matter of

negotiation!● But typically:

– Tire 3 pays a tier 2 to handle their traffic– Tier 2 peers traffic for free– Tier 2 pays a tier 1 for global transit– Tier 1 peers with other Tier 1 for free

16

IX/IXP points● Often a majority of your traffic goes to a local

neighbor● Sending it over a transit peer is expensive● Therefor you want to peer with your local

neighbors● Even if fiber is cheap you don't want to build a

full mesh over the area...

17

IX/IXP points● An IX point is a dedicated facility to exchange

traffic● Typically anybody can connect, as long as they

pay a fee● Three large IX in Stockholm:

– Netnod– SOL-IX– STHIX

18

Service Level Agreements● When you buy a transit connection you get an

SLA● For a home connection the SLA is “Whatever

we decide to give you”● For a company it's usually something like

– Guaranteed bandwidth 99% of the time– Guaranteed uptime 99.95%– Guaranteed service within 4 hours

19

How do we handle remote sites?● Tunnel them over the common Internet?

– Cheap?– No guarantees

● Buy dedicated connections...● Let them be completely independent?● Advantages?

20

Basic topologies: Bus network● Most early networks● This topology is rarely

seen today

21

Basic Topologies: Ring network● Provides redundancy● Requires multiple

connections● A good balance

between cost and redundancy

22

Basic topologies: Star● Cheap, easy to

deploy● Vulnerable central

node● Commonly used in

small networks● No redundancy

23

Basic topologies: Extended star● Frequently the result

when starting with a star topology

● No redundancy● Cheap● Adding a few cross

connections can often improve the network

24

Basic topologies: Mesh● Expensive● High performance● Full mesh is rare● Lose mesh is

regularly used● Can handle a large

number of outages

25

Combining topologies● It's common to

combine the mesh and ring topologies to form a lose mesh

● This is a very common topology today

26

Example network: Small company

27

Example network: Medium company

28

Example network: Large company

29

Example core: ARPA

30

Example core: ARPA

31

Example core: Nordunet

32

Example core: SUNET 155mbit● Dual links● Extended star with

redundant links

33

Example core: GigaSunet● Large ring network● No single point of

failure● This map doesn't

really show the complexity of Stockholm

34

Example core: OptoSunet● Large ring topology

again● All lines are

redundant● Virtual routers

35

Example core: BT

● Typical mesh network

● Very high level of redundancy

36

Example core: KPNQwest

● Typical multiple ring network

37

Example core: Golden Telecom

38

The limiting factor for global networks

39

Losing a submarine cable (or two)● From January 23 to February 4, 2008, a total of

5 submarine cables were cut● This resulted in a major Internet outage in

Egypt and India● Many companies got their networks split

– With the backup route over the same cable– Client offices split from the back end servers

● How do you handle this?

40

What is the security model?● Firewalls

– Secure inside-insecure outside– Outdated concept– Still popular in company networks– Requires a design with DMZ

● Open network– Let each PoP handle security on it's own– This is how ISP networks work

41

Network management● Arrange OoB (Out of Band) management

– Typically modems or a separate network● Make sure your NOC is well located

– Multiple connections to the core– No single point of failure

● Plan for a backup NOC!● Several companies had their NOC in New

Orleans...● And their backup NOC's to...

42

Conclusions● A good design can save you a lot of headaches● Plan for the future● Make sure you know where your traffic goes● Make disaster plans for equipment can, and

will, fail.● Simulate failures to find out if your network

topology is good enough● This is a religious matter for many people

43

The exam● The exam will be in two parts. You have to pass

both.● One practical part where you will be required to

configure routers– This will be done individually– Expect all subjects of the labs to show up

● One written exam● If you pass both your combined score will be

used to grade you. ● Questions?