Cisco Catalyst 3750 / 3560 and 2960 Series Switching ...d2zmdbbm9feqrf.cloudfront.net/2013/usa/pdf/BRKARC-3437.pdf · Cisco Catalyst 3750 / 3560 and 2960 Series Switching Architecture

Cisco Catalyst 3750 / 3560 and 2960 Series Switching Architecture BRKARC-3437

Muhammad A Imam

TME, CCIE #27739

Enterprise Networking Group

© 2013 Cisco and/or its affiliates. All rights reserved. TECCRS-3437 Cisco Public

Abstract This session discusses the Cisco Catalyst 3750,3560 and 2960 product line architectures. It includes an in depth analysis of the differences between the various platforms focusing on ASICs, TCAMs, modules and architecture based packet walks. It covers the evolution of Cisco Stacking solutions by comparing Cisco StackWise, StackWise Plus and FlexStack. Packet walks are provided for better understanding. The session then moves on to stacking functions. After an in-depth discussion on centralized and distributed functions in a stack, the session moves on to configuration management and best practice deployments of stacks. Further discussions center on QoS on the 2960 and 3750 series and software licensing and feature sets. This session is for network designers and network operation engineers who have deployed or considering deployment of the 3750, 3560 or 2960 series switches. A basic understanding of switching and routing would be beneficial.

3


Agenda

Fixed Switching Portfolio

Architecture Overview

Packet Walks

Stacking Architecture – Data & Power

Summary Quality of Service

Summary

4


Product Portfolio – Overview

Catalyst Compact

Switches Catalyst 3K Series Switches

Stackable FAMILIES

3750-X Series

3750-V2 Series

3850 Series

Modules

C3KX-NM-1G/10G C3KX-NM-10GT C3KX-SM-10G

Standalone FAMILIES

3560-X Series

3560V2 Series

FAMILIES

Layer 3:

3560C Series

Layer 2 only:

2960C Series

Catalyst 2K Series

Switches

Modules

FAMILIES

2960-S Series

2960-SF Series

2960 Series (Non-Stackable)

2960-X Series

2960-XR Series

C2960S-STACK

Wall Mountable

5


Catalyst 3K Product Portfolio

Stand-Alone 3K Switches

Fast Ethernet Gigabit Ethernet

Catalyst 3560 v2

Data or PoE

Fixed 1G Uplinks

Single PS

LLW

Catalyst 3560-X

Data/PoE(+)/ UPoE

Mod 1G/10G Uplinks

Dual PS

E-LLW

Stackable 3K Switches


Catalyst 3750 v2

Data or PoE

StackWise

Fixed 1G Uplinks

Single PS/LLW

Catalyst 3750-X

Data/PoE(+)/ UPoE

StackWise+

StackPower

Mod 1G/10G Uplinks

Dual PS / E-LLW

C3KX-NM-10G/1G

C3KX-NM-10GT C3KX-SM-10G

LAN Base IP Base IP Services

6

Fiber 3K Switches

Catalyst 3750-

V2

24 Port

Stackwise/LLW

Catalyst 3750-X

12 Port / 24 Port

Stackwise+/E-

LLW


Built on Cisco’s Innovative “UADP” ASIC

Wireless CAPWAP Termination

Integrated Controller: Up to 50 APs

Up to 2000 Clients per Stack

40 Gbps Uplink Bandwidth

Stackpower

Line Rate on All Ports

480 Gbps Stacking Bandwidth

Full POE+

FRU Fans, Power Supplies

Granular QoS/Flexible NetFlow

New Arrival – Catalyst 3850

BRKARC-3438 - Cisco Catalyst 3850 Series Switching Architecture @ 3 to 4:30 (Tue) & 8 to 9:30 (Thu) BRKCRS-2889 - Converged Access System Architecture @ 12:30 to 2:30 (Tue) BRKCRS-2888 - Converged Access Design Options @ 1:30 to 3:30 (Wed)

7


Stackable


Catalyst 2960-SF

Data / PoE(+)

Fixed 1G Uplinks

Single PS

E-LLW

LAN Lite LAN Base IP Lite*

C2960S-STACK

Catalyst 2960

Data or PoE

Fixed 1G Uplinks

Single PS

LLW

Catalyst 2960-S

Data / PoE(+)

Fixed 10G Uplinks

Single PS

E-LLW

Catalyst 2K Product Portfolio

8

Stand-Alone

Fast Ethernet

Catalyst 2960Plus

Data or PoE

Fixed 1G Uplinks

Single PS

LLW

Fast Ethernet


2960-C Portfolio

Fast Ethernet

LAN Base IP Base

NO Modules

3560-C Portfolio

Gigabit Ethernet/Fast Ethernet

Compact Switch Product Portfolio

9

Gigabit Ethernet/Fast Ethernet

Data/PoE

8/12 Ports

PoE Pass-through

Fixed 1G Uplinks

Fanless

MACSEC

Data/PoE

8/12 Ports

PoE Pass-through

Fixed 1G Uplinks

Fanless

Architecture Overview


Switch Components – 3750-X

SDRAM

CPU

Stack PHY

Flash

Serial

Port

ASIC Port

ASIC

Port

ASIC

Switch Fabric

Dual Mode PHY

10/100

2X10G or 4X1G 12X1G 12X1G 12X1G 12X1G

StackWise,

StackWise

Plus

24X1G POE 24X1G POE

Two Stack

Cables

8 Port

PHY

8 Port

PHY

8 Port

PHY 8 Port

PHY

8 Port

PHY

8 Port

PHY

Simple Design

Key components:

- ASICs

- CPU

- Phys

Slight Variations in some Families/SKUs

Some additional components might be:

- Switch Fabric

- Stack Ports& Phys

- PoE Controllers

11


Two Stack

Cables

SDRAM

CPU

Stack PHY

Flash

Serial

Port ASIC Port ASIC Port ASIC

Switch Fabric

Dual Mode PHY

10/100

2X10G or 4X1G 12X1G 12X1G 12X1G 12X1G

StackWise,

StackWise

Plus

24X1G POE 24X1G POE

8 Port

PHY

8 Port

PHY

8 Port

PHY 8 Port

PHY

8 Port

PHY

8 Port

PHY

Port ASICs

Port ASIC is the BRAIN of the Switch

– Intelligence

– Memory

Processes packets by pre-pending 24 B header for internal use

Processing includes

– Packet Modifications

– Decision to forward or drop the packets

Lookups include

– TCAM/Hash Tables/Forwarding/etc

12


TCAM/SRAM

Stores vital information including IPv4, IPv6 and MAC addresses, ACES etc.

The 3750-X TCAM/SRAM is incorporated into the Port ASIC

Hardware Merge process to pack entries into TCAM

Two Stack

Cables

SDRAM

CPU

Stack PHY

Flash

Serial


Switch Fabric

Dual Mode PHY

10/100

2X10G or 4X1G 12X1G 12X1G 12X1G 12X1G

StackWise,

StackWise

Plus

24X1G POE 24X1G POE

TCAM

SRAM

TCAM

SRAM

TCAM

SRAM

8 Port

PHY

8 Port

PHY

8 Port

PHY 8 Port

PHY

8 Port

PHY

8 Port

PHY

13


Switch Fabric

128Gbps switching Fabric

Provides line rate and local switching within a switch and stack connectivity – 48G + 2X10G + 32 Stack-ports (100Gbps FDX)

64 Gbps Ring Stackwise Plus

1 Gbps Ring Inter-connect control path to the Port ASICs to the CPU

Point-to-Point, 32 Gbps ring connecting each Port ASIC

Two Stack

Cables

SDRAM

CPU

Stack PHY

Flash

Serial

Port

ASIC Port

ASIC

Port

ASIC

Switch Fabric

Dual Mode PHY

10/100

2X10G or

4X1G 12X1G 12X1G 12X1G 12X1G

StackWise,

StackWise

Plus

24X1G POE 24X1G POE

8

Port

PH

Y

8

Port

PH

Y

8

Port

PH

Y

8

Port

PH

Y

8

Port

PH

Y

8

Port

PH

Y

14


CPU

Switch-to-Switch communication and synchronization

Updates the MAC and Routing caches attached to each port ASIC

Performs CPU Software-based forwarding when the TCAM is over its limits for MACs, Routes, ACL entries etc.

The CPU communicates with the Port ASICs via a dedicated management ring (the yellow ring in the diagram)

Two Stack

Cables

SDRAM

CPU

Stack PHY

Flash

Serial

Port

ASIC Port

ASIC

Port

ASIC

Switch Fabric

Dual Mode PHY

10/100

2X10G or

4X1G 12X1G 12X1G 12X1G 12X1G

StackWise,

StackWise

Plus

24X1G POE 24X1G POE

8

Port

PH

Y

8

Port

PH

Y

8

Port

PH

Y

8

Port

PH

Y

8

Port

PH

Y

8

Port

PH

Y

15


PHY – Includes Link Encryption

MACsec link encryption in hardware – Line rate (3750X/3560X)

All media conversion

– 10/100/1000 Mbps

FX, LX/LH, SX, ZX, BX (1490 & 1310Nm), CWDM, DWDM

– 10G, Supported:

LR (SMF 10km), LRM, SR (MMF), CX1, CX3, CX5

Two Stack

Cables

SDRAM

CPU

Stack PHY

Flash

Serial


Switch Fabric

Dual Mode PHY

10/100

2X10G or 4X1G 12X1G 12X1G 12X1G 12X1G

StackWise,

StackWise

Plus

24X1G POE 24X1G POE

8 Port

PHY

8 Port

PHY

8 Port

PHY 8 Port

PHY

8 Port

PHY

8 Port

PHY

16


Ring View of the Switch Fabric

Physically, the ring is a series of switch fabrics strung together by stack cables

The switch fabric performs token generation and ring control

Two Stack

Cables

Stack PHY Switch Fabric

Stack PHY Switch Fabric

SDRAM

CPU

Stack PHY

Flash

Serial


Switch Fabric

Dual Mode PHY

10/100

2X10G or 4X1G 12X1G 12X1G 12X1G 12X1G

StackWise,

StackWise

Plus

24X1G POE 24X1G POE

8 Port

PHY

8 Port

PHY

8 Port

PHY 8 Port

PHY

8 Port

PHY

8 Port

PHY

17


Catalyst 3750v2 Overview Fast Ethernet Models

• 3750 and 3750-X Main Architectural Differences:

• 3750 Does not have a second tier switch fabric like the 3750-X and can not locally switch without sending packets on the ring

• 3750 has external TCAMs

• All port-ASIC are part of the Stackwise internal ring!

• 3750 only runs in StackWise mode – 32G

2 Stack

Cables

Ports

Port ASIC

TCAM

SRAM

SDRAM

CPU

Stack PHY

Flash

Serial

Port ASIC

TCAM

SRAM

Port ASIC

TCAM

SRAM

POE POE POE

Ports Ports

8 Port

PHY

8 Port

PHY

8 Port

PHY 8 Port

PHY

8 Port

PHY

8 Port

PHY

8 Port

PHY

8 Port

PHY

8 Port

PHY

18


2960S Architectural Overview

Single ASIC with capacity of 88Gigs. Most energy efficient design

All Gig downlinks and 2 10-gig Uplinks

FlexStack

TCAM – embedded in ASIC

Single Port ASIC SDRA

M

CPU

Flash

Serial

USB

Serial

8

Por

t

PH

Y

8

Por

t

PH

Y

8

Por

t

PH

Y

8

Por

t

PH

Y

8

Por

t

PH

Y

8

Por

t

PH

Y

X

2 X

2

X

2 X

2

48 Port Switch with 2 X 10G uplink

10G 10G

2 Stack

Cables

19


Compact Switches Architectural Overview

SDRAM

CPU

Flash

Serial

Port ASIC

10/100 8 Port

PHY

PSE

module

8XPOE 1G 1G

GE

PHY

GE

PHY

PoE

System DC/DC

12V

SDRAM

CPU

Flash

Serial

Port ASIC

10/100 8 Port

PHY

PD/PSE

module

8XPOE 1G 1G

GE

PHY

GE

PHY

PoE \ PoE+ \UPoE

PoE

System DC/DC

12V

Aux Power External Power

AC / DC

Compact Switch model Comparison

Power : Models that are PoE/PoE+/UPoE powered and AC Powered

Speeds :10/100 or 10/100/1000.MACSec Phys on the 3K

Capability : 3560c runs IPBase Only. No IPServices

ASICs: Two types of ASICs – Choice depends on speed and capability

TCAM – embedded in ASIC

20


Service Module Architectural Overview

CPU

FPGA MACSec Phy SFP+

SFP+

Mem

Mem

To Switch ASIC

3KX-SM-10G : 3750X Service Module

FPGA : Contains logic to implement Netflow engine. Supports 32K flows

Phy : Helps perform switch to switch MACSec in H/W.

CPU : Quad Core Processor with 600 Mhz Frequency. Runs a Linux based environment over IOS

Upgrades : Separate image that can be downloaded from cisco.com

21

Packet Walks


Within the ASIC

MAC Port 1

MAC Port 2

MAC Port 4

MAC Port 3

MAC Port 16

TXT Queues

Forwarding Controller

RCV FIFO

TXT FIFO

RCV Buffer

TXT Buffer

To CPU

From Ring/Fabric

MAC Port 5

TCAM SRAM

To Ring/Fabric

Ingress Path Egress Path

23


Within the ASIC

MAC Port

1

MAC Port

2

MAC Port

4

MAC Port

3

MAC Port 16

TXT Queues


RCV FIFO

TXT FIFO

RCV Buffer

TXT Buffer

To CPU

From Ring/Fabric

MAC Port

5

SRAM

To Ring/Fabric

L2

L3

QOS ACES

SEC ACES

IPv6

These spaces can re-

allocated using SDM

templates

24


Within the ASIC

MAC Port

1

MAC Port

2

MAC Port

4

MAC Port

3

MAC Port 16

TXT Queues


RCV FIFO

TXT FIFO

RCV Buffer

TXT Buffer

To CPU

From Ring/Fabric

MAC Port

5

TCAM SRAM

To Ring/Fabric

1

1 Packets enter the

switch and is received

by the Receive FIFO

25


MAC Port

1

MAC Port

2

MAC Port

4

MAC Port

3

MAC Port 16

TXT Queues


RCV FIFO

TXT FIFO

RCV Buffer

TXT Buffer

To CPU

From Ring/Fabric

MAC Port

5

TCAM SRAM

To Ring/Fabric

2 2 The whole packet is

sent to the Receive

Buffer

Copy of the first 200

bytes is sent into the


for processing

(forwarding, ACL,

QOS lookups)

Packet Walk - Ingress On the Way In

26



MAC Port

1

MAC Port

2

MAC Port

4

MAC Port

3

MAC Port 16

TXT Queues


RCV FIFO

TXT FIFO

RCV Buffer

TXT Buffer

To CPU

From Ring/Fabric

MAC Port

5

TCAM SRAM

To Ring/Fabric

3

3 Submit packet to

VLAN

decap/parsing/proces

sing in the forwarding

controller and the

native frame is

obtained

4

4

27



MAC Port

1

MAC Port

2

MAC Port

4

MAC Port

3

MAC Port 16

TXT Queues


RCV FIFO

TXT FIFO

RCV Buffer

TXT Buffer

To CPU

From Ring/Fabric

MAC Port

5

TCAM SRAM

To Ring/Fabric

5

5 Search Engine in the


sends the packet to

the TCAM for L2

lookup and receives

the index

The Forwarding

Controller queries the

SRAM with the index

to get the desired

look up.

6 6

28



MAC Port

1

MAC Port

2

MAC Port

4

MAC Port

3

MAC Port 16

TXT Queues


RCV FIFO

TXT FIFO

RCV Buffer

TXT Buffer

To CPU

From Ring/Fabric

MAC Port

5

TCAM SRAM

To Ring/Fabric

7

7

Search Engine


sends QOS and ACL

Look up sent to the

TCAM. Index

returned


queries the SRAM for

the respective Ingress

ACL and QOS

response

8 8

29


MAC Port

1

MAC Port

2

MAC Port

4

MAC Port

3

MAC Port 16

TXT Queues


RCV FIFO

TXT FIFO

RCV Buffer

TXT Buffer

To CPU

From Ring/Fabric

MAC Port

5

TCAM SRAM

To Ring/Fabric

9

9 Look up to the policer.

How much policing to

do?

Policing information

returned

10

10


30 30


MAC Port

1

MAC Port

2

MAC Port

4

MAC Port

3

MAC Port 16

TXT Queues


RCV FIFO

TXT FIFO

RCV Buffer

TXT Buffer

To CPU

From Ring/Fabric

MAC Port

5

TCAM SRAM

To Ring/Fabric

11

11 Search Engine in


sends the L2/L3

Forwarding Look up

to the TCAM. Index

returned


sends Index to the

SRAM for destination

details. Destination

information returned

12 12


31 31


MAC Port

1

MAC Port

2

MAC Port

4

MAC Port

3

MAC Port 16

TXT Queues


RCV FIFO

TXT FIFO

RCV Buffer

TXT Buffer

To CPU

From Ring/Fabric

MAC Port

5

TCAM SRAM

To Ring/Fabric

14 24 byte descriptor

sent to the Receive

buffer. Appended to

the original packet.

14

Native

Packet

Descriptor

Descriptor


32


MAC Port

1

MAC Port

2

MAC Port

4

MAC Port

3

MAC Port 16

TXT Queues


RCV FIFO

TXT FIFO

RCV Buffer

TXT Buffer

To CPU

From Ring/Fabric

MAC Port

5

TCAM SRAM

To Ring/Fabric

15 Packet with the extra

24 byte descriptor is

sent to the Switching

Fabric/Ring

15

Native

Packet

Descriptor


33


Within the ASIC

MAC Port 1

MAC Port 2

MAC Port 4

MAC Port 3

MAC Port 16

TXT Queues


RCV FIFO

TXT FIFO

RCV Buffer

TXT Buffer

To CPU

From Ring/Fabric

MAC Port 5

TCAM SRAM

To Ring/Fabric

Egress Path

34


MAC Port

1

MAC Port

2

MAC Port

4

MAC Port

3

MAC Port 16

TXT Queues


RCV FIFO

TXT FIFO

RCV Buffer

TXT Buffer

To CPU

From Ring/Fabric

MAC Port

5

TCAM SRAM

To Ring/Fabric

1 Packet with the extra

24 byte descriptor is

sent to the Switching

Fabric/Ring

1 Native

Packet

Descriptor

Packet Walk - Egress On the Way Out

35


MAC Port

1

MAC Port

2

MAC Port

4

MAC Port

3

MAC Port 16

TXT Queues


RCV FIFO

TXT FIFO

RCV Buffer

TXT Buffer

To CPU

From Ring/Fabric

MAC Port

5

TCAM SRAM

To Ring/Fabric

2 Packet is stored in

the Transmit buffer.

Packet location

stored in the TXT

Queue

2


36


MAC Port

1

MAC Port

2

MAC Port

4

MAC Port

3

MAC Port 16

TXT Queues


RCV FIFO

TXT FIFO

RCV Buffer

TXT Buffer

To CPU

From Ring/Fabric

MAC Port

5

TCAM SRAM

To Ring/Fabric

3 Packets egresses

and is stored in the

Transmit FIFO for

egress processing

3


37 37


MAC Port

1

MAC Port

2

MAC Port

4

MAC Port

3

MAC Port 16

TXT Queues


RCV FIFO

TXT FIFO

RCV Buffer

TXT Buffer

To CPU

From Ring/Fabric

MAC Port

5

TCAM SRAM

To Ring/Fabric

4 First 200 bytes sent

to the Forwarding

Controller for egress

processing 4


38 38


MAC Port

1

MAC Port

2

MAC Port

4

MAC Port

3

MAC Port 16

TXT Queues


RCV FIFO

TXT FIFO

RCV Buffer

TXT Buffer

To CPU

From Ring/Fabric

MAC Port

5

TCAM SRAM

To Ring/Fabric

5 Search Engine in


sends egress ACL

lookup query to

TCAM. TCAM returns

index

5 6 6


uses index to get the

ACL info


39 39


MAC Port

1

MAC Port

2

MAC Port

4

MAC Port

3

MAC Port 16

TXT Queues


RCV FIFO

TXT FIFO

RCV Buffer

TXT Buffer

To CPU

From Ring/Fabric

MAC Port

5

TCAM SRAM

To Ring/Fabric

7 Packet Header

prepared in the


7

8

8


sends the header info

to the TXT FIFO

where the final packet

is assembled


40 40


MAC Port

1

MAC Port

2

MAC Port

4

MAC Port

3

MAC Port 16

TXT Queues


RCV FIFO

TXT FIFO

RCV Buffer

TXT Buffer

To CPU

From Ring/Fabric

MAC Port

5

TCAM SRAM

To Ring/Fabric

9 Final packet sent to

the egress port.

9


41



Switch Fabric

3750X Unicast Packet Walk Locally Switched (Stacked or Standalone Switch)

The packet is sent to the switch Fabric and locally switched to the destination Port ASIC

Simple switching with, no ACK necessary

Does not disrupt the Stack rings

42

Source

Destination

Packet

Packet is locally switched. Never get forwarded to Stackwise rings

42


Switch Fabric

3750X Unicast Packet Walk Destination Across the Stack – Destination Strip

The Source Port ASIC sends the packet to the Source Switch Fabric and it is switched to the Destination Switch Fabric

The Destination Switch Fabric removes the packet and sends a 16 bit ACK

The Originating Switch Fabric receives and removes the ACK

43


Switch Fabric


Switch Fabric


Source

Destination

Packet

ACK

1. Forward To the Stack

ring

2. Copies packet – sends it to

port-ASIC

3. Removes packet off the

stack ring

4. Send 16 bits ACK

5. Remove the ACK

43


3750v2 Unicast Packet Walk Source Strip

All types of packets are passed all the way around the ring, copied at the destination(s) and returned to the sender for stripping

All packets are sent to the stack ring, the Port ASICs can not locally switch traffic

44

Source

Destination

Source

Destination

Packet




1. Forward To the Stack

2. Copy Packet by Dest. Port-

ASIC

3. Pass Packet to Dest. port

4. Original Packet rotates around the

ring

5. Src Removes Packet off the

ring

44


Smart Multicast Packet Walk All Catalyst 3K Models

The Switch Fabrics with multicast ports in that group copy the packet

The originating Switch Fabric removes the packet from the ring

Note: There is only one packet on the ring per multicast flow, replication only occurs at the local level

Note: if the sender and all of the receivers are on the same switch no packets are sent to the ring

45

Source

Destination

Source

Destination

Packet




Switch Fabric

Switch Fabric

Switch Fabric

1. Multicast Packet Must forward to the

stack ring!

2. Copy Multicast Packet: forward to

the stack ring and to local port-ASICs

3. Replicate Packet and forward to the

port-ASICs that have listeners

5. Original Multicast Packet continues on the ring in case there are more listeners in the

stack. 4. Replicate in case of multiple

listeners

6. Remove Packet off the ring. Dest. Strip. – –

45

Stacking Architecture – Stackwise & Stackwise Plus


What is a Stackable Switch?

Allows access to all switches with a single IP address

Provides the means to manage the stack via CLI or MIB

Can connect all switches in a physical ring topology

Traffic flows in either direction of the ring, some Resiliency

Automatic Master selection & backup 1:N

Automatic IOS versioning and Update!

Automatic configuration of new members

Automatic unit replacement (configuration of old switch retained)

Stateful switch over in case of master failures

Ring resiliency similar to FDDI, provides HA and Resiliency

Sub-millisecond Master failover

Smart Multicast

Cross-stack features (Etherchannel and QoS)

47


Cisco StackWise (Plus)

Logical and physical ring architecture of the Catalyst 3k

Comprises two 16 Gbps counter-rotating rings

– Data on both rings when fully connected:

Stackwise - 32 Gbps

Stackwise Plus - 64 Gbps

Internal Connection: On 3750v2 : The Stack Ring is the switching fabric

3750X : The Stack Ring only interconnects the individual Switch Fabrics

StackWise

Rings/Cable

48


Understanding the Stack Cable

Eight TX/RCV pairs per ring

That is 16 pairs (8 channels per ring)

Each TX/RCV pair has 2 traces that use differential signaling.

That is 32 traces in total

Each TX/RCV pair runs at 2.5 Gbps

8B/10B encoding is used.

(for every ten bits sent, eight bits are data and two bits are overhead)

Cable

RCV Pair

TX Pair Trace

49


Stack Cable 1

TXT/RCV Pairs

Trace

Stack Cable 2

16 Gbps 16 Gbps 16 Gbps 16 Gbps

Understanding the Stack Ring Speed

16 Channels x 2.5 Gbps x 8B/10B = 32Gbps

Or bidirectional 16 Gbps per cable = 32Gbps

Or Two Rings running at 16Gpbs each = 32Gbps

50


Stackwise vs. Stackwise Plus Spatial Reuse

No Spatial Reuse

(Source Strip)

3750v2 StackWise Only 2 Flows

Access-based tokens

Spatial Reuse

(Destination Strip)

3750-X StackWise Plus Up to N by 2 Simultaneous Flows

Credit-based Tokens

Note: These are packets not tokens. There is only 1 token per ring.

Stackwise

32 Gbps

Stackwise Plus

N by 32 Gpbs

51





Switch Fabric

Switch Fabric

Switch Fabric

Loops

3750v2 3750-X

Ring Healing

The Switch Fabric or Port ASIC closest to cable detects link down – Criteria is coding violations in a period of time – Loss of at most one packet that was being transmitted when ring broke – Just microseconds for hardware to detect failure

Each switch signals a bad link to stack its partner

Both ends of the cable loop back on themselves

Loop

52

Stacking Architecture – Flexstack


Flexstack on the Catalyst 2960-S New Stacking Mechanism

Stack Bandwidth – 20 Gbps

Up to 4 members in a stack

Not a ring Architecture – Hop by Hop

Local switching support for unicast packets

All members of the stack see the unknown Unicast, Broadcast, and Multicast packets

54


C2960-S FlexStack Packet Flow, BCAST

Packet flows in C2960-S stack are hop by hop.

L2 Destination Unknown, MCAST, BCAST all the same

Drop Table does not fwd packet between member 3 & 4

Member 1

Member 4

Member 3

Member 2

Bcast Packet

ingresses member 1

BCAST packet egresses on

all interfaces FWDing on

that vlan for all members

Passive Link for Switch

1 prevents

Fwd of packet between

members 2 & 3

55


2960-S FlexStack Ease of Use

3750-X StackWise Plus Ease of Use and High Availability

Device Limit 4 units 9 units

Stack Bandwidth 20G 64G

Architecture HW Drop Table Ring (Destination stripping)

Dynamic Ring Load Balancing No Yes

Stack Convergence 1-2 seconds Few milliseconds

Stack QoS Applied hop by hop Applied on ingress

Management Single IP address, SNMP, SYSLOG Single IP address, SNMP, SYSLOG

Configuration Single config and CLI, auto image and config update Single config and CLI, auto image and config update

Show and Debug Commands Unified Unified

Single Forwarding and Control Plane Synchronize ARP, MAC Address, IGMP, VLAN tables Synchronize ARP, MAC Address, IGMP, VLAN,

Routing tables

Cross-Stack Features Yes Yes

Single Bridge-ID Yes Yes

Preprovison members Yes Yes

Redundancy Stack master 1:N redundancy Stack master 1:N redundancy

Easy member replacement Yes Yes

FlexStack Vs. StackWise Plus

56

Stacking Architecture – Stack Functions


Stack Master Election Criteria

1) The stack (or switch) whose master has the higher user configurable mastership priority 1–15

2) The stack (or switch) whose master is not using the default configuration

3) The stack (or switch) whose master has the higher software priority

Cryptographic IP Services

Cryptographic IP Base

4) The stack (or switch) whose master has the longest uptime

5) The switch or stack whose master has the lowest MAC address

58


Switch Priority for Master Role Default Is 1, Don’t Participate Is 0

Switch (config)# switch 3 priority 10

Switch (config)# switch 4 priority 9

Switch (config)# exit

Switch# show switch

Switch# Role Mac Address Priority State

-----------------------------------------------------------------

1 Member 000a.fdfd.0100 1 Ready

2 Member 000a.fdab.0100 1 Ready

3 Master 000a.fd22.0100 10 Ready

4 Member 0003.fd63.9c00 9 Ready

Master

Back-up Master

59


When Does the Stack Master Change?

A Stack Master Can Change If:

The stack master fails or reboots

The stack master is removed from the switch stack

The stack master is power cycled or powered off

There is a Stack Merge

– Stack merge occurs when a new switch is powered up before being connected to the stack cables, or when two cables are disconnected from the stack

60


Functions of the Stack Master

The Stack Master:

Controls all centralized functions

Builds and propagates the L3 FIB

Manages and Propagates the configuration file to the stack

Controls the console

Controls the CDP neighbor table

Controls the VLAN database

Upgrades the stack

Config FIB IOS

Config

FIB

IOS

Config

FIB

IOS

61


Centralized and Distributed Functions

Centralized functions Those that are reside on the

master node

Those that are forwarded to the master node

Those that are controlled or synchronized by the master node

Ex: CDP, VLAN Database mgmt,Routing

Distributed functions Those that are performed locally

by each node

These functions are synchronized or updated between the nodes

Ex: MAC address management, STP

Master

Master

62


Stack Configuration Management

Master:

– Copies of the startup and running config files are kept on all members in the stack

– The current running-config is synched from the master to all members

– On a switchover, the new master re-applies the running-config so that all switches are in sync

Member:

– Keeps a copy of startup and running config at all times

– On boot-up waits for config file from master and parses it

Config

Config

Config

63


Adding a New Switch Example

The stack has three members - 1, 2, 3

A new switch with #3 is added to the stack

Conflict detected, number changes based on the rules used for numbering (ID).

It is assigned the #4 and reloads switch #4

Configuration commands in the config file which apply to interfaces 4/0/* apply to the new switch

Switch #2

Switch #3

Master #1

Switch #4

64


Switch Pre-provisioning Example

1) Pre-provision Switch #4 in advance

Switch(config)#switch 4 provision WS-

C3750x-48P

2) Enter the port configuration for Switch #4.

3) Add Switch #4 when required

Master #1

Switch #2 Switch #3

Config Config

Switch #4

65


Switch Removal

The stack has three members — 1, 2, 3

Member #3 is removed or powered down – Neighbor loss is detected by Switch

#1 and Switch #2

– Layer 2 and Layer 3 convergence may need

– to happen

– Now there is a stack of two switches—Switch #1 and Switch #2

– Switch#1 is still the master

Master #1 is removed or powered down – Switch #2 takes over as master

– Layer 2 and Layer 3 convergence may need to happen

– Now there is a stack of one switch—#2 which is the master

Master #2

Master #1

Switch #2

Switch #3

66


Replacing a Switch

In this case, the failed switch is a Cisco Catalyst WS-C3750X-48TD

If replaced by another Cisco Catalyst WS-C3750X-48TD, the new switch will receive the port-level configuration of the original unit

If replaced by a different switch, the original configuration is lost and the new switch receives all stack global configuration

Config Config

67


Types of Stack Mismatch Homogeneous Stack: 3750 or 3750E/X:

Version Mismatch:

level and feature IOS revision set i.e. LAN Base, IP Base, and IP Services

SDM Mismatch:

All members of the stack must run the same SDM template as the master

Version Mismatch has priority over SDM mismatch

Hardware Mixed Stack: 3750 and 3750E/X:

Same as above

Feature Mismatch

Hardware features (POE, Jumbo frame routing) 3750

3750-X

3750-E

3750-E

3750-X

3750-E

3750-E

68


Stack IOS Upgrade Process Automatic Upgrade involves two processes:

Auto-Upgrade and Auto-Advise

The auto-upgrade processes runs first and it consists of:

auto-copy process and auto-extract process

Auto-copy copies a running image of any stack member into a switch if this process fails, then:

Auto-extract searches through all FLASH devices for a TAR file suitable for the switch in VM mode

If auto-extract fails, Auto-Advice provides a recommendation on how to upgrade manually!

Recommend:

Store Universal and Reformation TAR images in the master and a backup master for auto-extract to work

Configure a url for last resort: (point to the image repository)

boot auto-download-sw tftp://10.1.1.15/images/fall06/c3750-universal-tar

– Use to upgrade a mixed hardware stack:

– archive download-sw /directory tftp://10.1.1.10/ c3750-ipservices-tar.122-35.SE.tar c3750e-universal-tar.122-35.SE2.tar

69


LAN Base Stack Exception: No Mixed IOS Feature Set

A form of Feature mismatch in a mixed Hardware stack

No HW Encryption, no Stackpower support

No mixed IOS feature set support for LAN Base

Catalyst 3750X models running LAN Base feature set can only stack among themselves

Common mistake…Don’t call TAC..!

– Stacking LAN Base models with IP Base / IP Services

70

Stacking Architecture – Stackpower


StackPower Overview

Aggregates and shares available input power capacity in a Stack

Flexible arrangement of power supplies in a stack

– Up to 8.8Kw in a power stack of 4 switches (ring topology)

– Up to 22Kw in a power stack of 9 switches (Star topology using an XPS(eXpandable Power System) 2200)

Stackpower decouples a Power supply from its physical location in a switch/stack!

Provides RPS functionality (Zero-footprint RPS)

Intelligent Load shedding

Up to four switches can be participate in a power stack

More than one power stack within one Data stack (Stackwise Plus)

72


StackPower Modes of Operation

Available Power – Total Power available – show power inline

Budgeted Power – Power allocated to all devices - show power inline

Consumed Power – Power being consumed - show power inline police

StackPower operates in two modes: Power share – All PS shared, No redundancy

Redundant – Largest PS is available for redundancy

73


Lost PS or

Power source

Shed Load

Dropped PD

Lost PS or

Power source

Lost Shed Load

another PS

Dropped PD

Loose and Strict modes Control the behavior of Load shed

Loose mode allows for a negative power budget

Strict mode sheds load as soon as the power budget goes below the Allocated power level

BUDGET Available Power Pool

ALLOCATED Committed Power

ACTUAL Drawn Power

BUDGET Available Power Pool

ALLOCATED Committed Power

ACTUAL Drawn Power

Power-sharing Strict mode Power-sharing Loose mode Default

74


Best Practice Balance Power supplies across the stack

Total Input Power = 5,400w

Total Output Power = 4,400w

The right half generates only 20A but consumes 80A

Stackpower rated for ~40A

In failure scenario, Stackpower could be oversubscribed; console messages will warn about the condition and Intelligent load shed will occur.

500w

500w

A B

1,100w

1,100w

A B

1,100w

1,100w

A B

2,000w

A B

2,000w

200w

200w

X

30 A

60 A

30 A

Recommendation:

1. Balance PS across all systems, and

2. insist on filling up PS slot A on every switch in

the stack, before using slot B on any switch! 75

Quality of Service


Catalyst Switches - Line Rate Switches

77

Why QoS ?

Congestion can still happen

Bursty Traffic

Traffic Prioritization

Speed Mismatches

Traffic variation


Network

Interface

Catalyst 3K - QoS Model

78

Trust

Status

Service-

Policy

Policer/R

eMarker

Input Q

Map

Q1

Q2

SRR

(Shared)

WTD

WTD

Output Q

Map

Stackwise

Q1

Q2

Q3

Q4

WTD

WTD

WTD

WTD

SRR

(Shaped/Shared)

1P3Q3T/

4Q3T


Trust, Classification & Marking

79

Trust

Status

Service-

Policy

Policer/R

eMarker

QoS Label

Markings trusted by default – ‘no mls qos’

‘mls qos’ enabled – all markings are set to BE

Trust Config

Trust COS/DSCP

Conditional Trust

Mark without trust

MQC

AutoQoS

C3750(config)#no mls qos

C3750(config)#

C3750(config)#mls qos

C3750(config)#interface GigabitEthernet1/0/11

C3750(config)#mls qos trust dscp



C3750(config)#mls qos trust dscp

C3750(config)#mls qos trust device cisco-phone



C3750(config)#mls qos cos 5

C3750(config)#mls qos cos override

C3750(config)#access-list 101 permit tcp any eq www any

C3750(config)#class-map match-all http

C3750(config-cmap)#match access-group 101

C3750(config-cmap)#policy-map web-server

C3750(config-pmap)#class http

C3750(config-pmap-c)#police 500000 8000 exceed-act drop

C3750(config-pmap-c)#int gig1/0/11

C3750(config-if)#service-policy input web-server

C3750(config-pmap-c)#int gig1/0/2

C3750(config-if)#auto qos voip cisco-phone

C3750(config-if)#do sh run int gig1/0/2

interface GigabitEthernet1/0/2

...

mls qos trust device cisco-phone

mls qos trust cos

auto qos voip cisco-phone

service-policy input AutoQoS-Police-CiscoPhone


Ingress Queuing & Scheduling

80

Input Q

Map

Q1

Q2

SRR

(Shared)

WTD

WTD

Stackwise

Congestion on the ingress less likely but possible

– Speed Mismatches

– Aggregation Points

Queuing

– Two user-configurable ingress queues/ ASIC

– One queue is an expedite queue – by default maps to voice traffic

Dropping

– WTD with 3 thresholds is used for congestion avoidance

Scheduling

– DeQueuing of packets from the queues

– Ingress Queues can be shared but not shaped

C3750#sh mls qos maps dscp-input-q

Dscp-inputq-threshold map:

d1 :d2 0 1 2 3 4 5 6 7 8 9

------------------------------------------------------------

0 : 01-03 01-03 01-03 01-03 01-03 01-03 01-03 01-03 01-01 01-02

1 : 01-02 01-02 01-02 01-02 01-02 01-02 02-01 02-01 02-01 02-01

…

4 : 02-03 02-03 02-03 02-03 02-03 02-03 02-03 02-03 02-02 02-02

5 : 02-02 02-02 02-02 02-02 02-02 02-02 02-02 02-02 02-02 02-02

6 : 02-02 02-02 02-02 02-02

C3750#sh mls qos input-queue

Queue : 1 2

----------------------------------------------

buffers : 67 33

bandwidth : 90 10

priority : 0 10

threshold1: 8 34

threshold2: 16 66

Queue Number

Threshold Number

No Ingress Queuing on 2Ks


Egress Queuing & Scheduling

Output Q

Map

Q1

Q2

Q3

WTD

WTD

WTD

SRR

(Shaped/Shared)

Queuing

– Four egress queues/port

– Queues assigned based on QoS label

– 2 Queue-sets – 2 Queue configurations

Dropping

– WTD used for congestion avoidance

Scheduling

– Per Interface configuration

– Strict Priority

– SRR used to manage the queues

Q4 WTD

1P3Q3T/

4Q3T

81

C3750#sh mls qos maps dscp-output-q

Dscp-outputq-threshold map:

d1 :d2 0 1 2 3 4 5 6 7 8 9

------------------------------------------------------------

0 : 04-03 04-03 04-03 04-03 04-03 04-03 04-03 04-03 04-01 04-02

1 : 04-02 04-02 04-02 04-02 04-02 04-02 03-03 03-03 03-03 03-03

...

4 : 01-03 01-03 01-03 01-03 01-03 01-03 01-03 01-03 02-03 02-03

5 : 02-03 02-03 02-03 02-03 02-03 02-03 02-03 02-03 02-03 02-03

6 : 02-03 02-03 02-03 02-03

C3750#sh mls qos queue-set 1

Queueset: 1

Queue : 1 2 3 4

----------------------------------------------

buffers : 10 10 26 54

threshold1: 138 138 36 20

threshold2: 138 138 77 50

reserved : 92 92 100 67

maximum : 138 400 318 400


Queueset: 2

Queue : 1 2 3 4

----------------------------------------------

buffers : 16 6 17 61

threshold1: 149 118 41 42

threshold2: 149 118 68 72

reserved : 100 100 100 100

maximum : 149 235 272 242


Queueset: 1

Queue : 1 2 3 4

----------------------------------------------

buffers : 10 10 26 54

threshold1: 138 138 36 20

threshold2: 138 138 77 50

reserved : 92 92 100 67

maximum : 138 400 318 400

Q1

2 MB/ASIC

1 Buffer = 256B

8192 Buffers

Q2

Q3

Q4

Q1

Q2

Gig1/0/1

Gig1/0/2 CPU Pool

140 KB

Common Pool Reserved Pool

C3750#sh mls qos int gig1/0/1 queueing

GigabitEthernet1/0/1

Egress Priority Queue : disabled

Shaped queue weights (absolute) : 25 0 0 0

Shared queue weights : 25 25 25 25

The port bandwidth limit : 100 (Operational Bandwidth:100.0)

The port is mapped to qset : 1

C3750#sh mls qos int gig1/0/3 queueing

GigabitEthernet1/0/3

Egress Priority Queue : enabled

Shaped queue weights (absolute) : 25 0 0 0

Shared queue weights : 10 10 60 20

The port bandwidth limit : 85 (Operational Bandwidth:100.0)

The port is mapped to qset : 2


Queueset: 1

Queue : 1 2 3 4

----------------------------------------------

buffers : 25 25 25 25

threshold1: 138 138 36 50

threshold2: 138 138 77 75

reserved : 92 92 100 100

maximum : 138 400 318 100


Cisco Catalyst 2960/3560/3750 QoS Design Queuing Design (1P3Q3T)

Q1

Priority Queue

1P3Q3T

Queue 4 (5%) CoS 1

Network Management

Call Signaling

Streaming Video

Transactional Data

Interactive Video

Voice

Application

Bulk Data

AF21

CS3

CS4

AF41

EF

CS2

AF11

Scavenger CS1

Best Effort 0

Internetwork Control CS6

Mission-Critical Data AF31

DSCP

Network Control –

CoS 2

CoS 3

CoS 4

CoS 4

CoS 5

CoS 2

CoS 1

CoS 1

0

CoS 6

CoS 3

CoS

Queue 2

(70%)

CoS 7

CoS 5

CoS 2

CoS 4

Queue 3

(25%) CoS 0

Q2T3

Q2T2

Q4T2

Q4T1

Q2T1

CoS 3

CoS 6

CoS 7

82

Wrap up


Deployment Scenarios

84

IP IP

• Data Redundancy using

StackwisePlus (64 Gbps)*

• Power Redundancy using

StackPower*

• Field replaceable PS and Fans*

• 9 member switch stack**

• Enhanced video features, e.g.,

Mediatrace, built-in traffic

generation

• Enhanced Security features, e.g.,

SGT, Device Sensor

**available on 3750, *available on 3K-X


Summary

85

Fixed Switch

Platform Architecture

ASIC based Architecture

StackPower

QoS Packetwalks

Stackwise & Stackwise+


Maximize your Cisco Live experience with your

free Cisco Live 365 account. Download session

PDFs, view sessions on-demand and participate in

live activities throughout the year. Click the Enter

Cisco Live 365 button in your Cisco Live portal to

log in.

Complete Your Online Session Evaluation

Give us your feedback and you could win fabulous prizes. Winners announced daily.

Receive 20 Cisco Daily Challenge points for each session evaluation you complete.

Complete your session evaluation online now through either the mobile app or internet kiosk stations.

86

Documents

Cisco Catalyst 3750 / 3560 and 2960 Series Switching ...d2zmdbbm9feqrf.cloudfront.net/2013/usa/pdf/BRKARC-3437.pdf · Cisco Catalyst 3750 / 3560 and 2960 Series Switching Architecture