Upload
sybil
View
30
Download
0
Tags:
Embed Size (px)
DESCRIPTION
Experiences from SLAC SC2004 Bandwidth Challenge. Les Cottrell, SLAC www.slac.stanford.edu/grp/scs/net/talk03/bwc-04.ppt. Sponsors/partners. SLAC/SC04 Bandwidth Challenge (plan C). PSC SciNet. 2 Sun Opteron/Chelsio-10GE. SLAC/FNAL booth (2418). 6 Boston file servers 1 GE. - PowerPoint PPT Presentation
Citation preview
1
Experiences from SLAC SC2004 Bandwidth Challenge
Les Cottrell, SLACwww.slac.stanford.edu/grp/scs/net/talk03/bwc-04.ppt
2
Sponsors/partners
3
SLAC/SC04 Bandwidth Challenge (plan C)
ESnet/QWest OC192/SONET
Sunnyvale/Qwest/ESnet
Juniper T320
Sunnyvale/Level(3)/NLR
SLAC/FNAL booth (2418)
1400 Kifer
1380 Kifer
10Gbps from NLR (via SEA, DEN, CHI)
PSC SciNet
SiSi
LoanedCisco Rtr
NLR-PITT-SUNN-10GE-17 NLR-PITT-SUNN-10GE-17
1 Sun file server 1 GE
2 Sun Opteron/Chelsio-10GE
SiSi
1554015808
NLR demarc 15808
15454
NLR demarc
SLACCisco Rtr
6 Sun Opteron/Chelsio-10GE
6 Boston file servers 1 GE 2 Sun file server 1 GE2 Sun Opteron/S2io-10GE
1 Sun Opteron/Chelsio-10GE
4
SC2004: Tenth of a Terabit/s Challenge
Joint Caltech, SLAC, Joint Caltech, SLAC, FNAL, CERN, UF, FNAL, CERN, UF, SDSC, BR, KR, ….SDSC, BR, KR, ….
10 10 Gbps waves to 10 10 Gbps waves to
HEP on show floorHEP on show floor Bandwidth Bandwidth
challenge: aggregate challenge: aggregate throughput throughput of 101.13 Gbpsof 101.13 Gbps
FAST TCPFAST TCP
5
Components10 Gbps NICs
S2io
Chelsio
1982 10Mbps3COM
v20z
v40z
3510 disk array
SR XENPAK
SVL/NLR
6
Challenge aggregates from SciNet
• Aggregate Caltech & SLAC booth, in & out
• 7 lambdas to Caltech, 3 ro SLAC
7
Challenge aggregates from MonALISA
• Sustained ~ 10Gbps for extended periods
8
Weathermap showing 8.7Gbps on ESnet
9
To/From SLAC booth• NLR: 9.43Gbps (9.07 goodput) + 5.65Gbps
(5.44Gbps goodput) in reverse– Two hosts to two hosts
• ESnet: 7.72Gbps (7.43Gbps goodput)– Only one 10Gbps host at SVL
• Single V40Z host with 2*10GE NICs to 2*V20Z across country got 11.4Gbps
• S2io and Chelsio (& Cisco & Juniper) all interwork
• Chelsio worked stably on uncongested paths
10
TOE• Chelsio had TCP Offload Engine
– Utilization factor of throughput & parallel streams
– Reduced cpu c.f. S2io non0TOE by factor ~ 3
11
Challenges• Could not get 10Gbps waves to SLAC only SVL
– Equipment in 3 locations• Keeping configs in lock-step (no NFS, no name
service)• Security concerns, used iptables• Machines only available 2 weeks before, some not
until we got to SC04• Jumbo frames not configured correctly at SLAC booth,
used 1500B frames mainly• Mix of hdw/swr: Opterons with various GHz & disks,
Xeons; Solaris 10, Linux 2.4, 2.6• Coordination between booths (sep by 100 yds)• Everything state of art (Linux 2.6.6, SR XENPAKs,
NICs
12
Award