TRIUMF a TIER 1 Center for ATLAS CanadaSteven McDonald

TRIUMF Network & Computing Services

steven.mcdonald@triumf.ca

iGrid 2005 – San Diego Sept 26th – 29th 2005

* *******

*

*

**

Centre - Country ALICE ATLAS CMS LHCb MB/s

ASCC - Taipei X X 110

CNAF - Italy X X X X 220

PIC - Spain X X X 200

CC-IN2P3 - France X X X X 220

FZK - Germany X X X X 220

RAL - England X X X X 220

BNL –United States X 154

FNAL – United States X 50

TRIUMF - Canada X 75

NIKHEF - Netherlands X X X 175

NORDIC GRID X X X 90

# TIER1 Sites 6 10 7 71,60

0

The TRIUMF Facility

• Located in Vancouver, Canada

• Research is performed locally using the world’s largest 500MeV Cyclotron

• As well as internationally at• CERN-ATLAS• BNL-KOPIO• KEK-Neutrino Experiment

•~400 full-time employees

• Nuclear Physics• Particle Physics• Astrophysics• Material Science

• Proton & PET Therapy• Radiopharmaceuticals • Radioactive beams• Microchip design

Canada’s National Laboratory for sub-atomic physics

Politics & Funding

• TRIUMF 5 yr budget cycle runs 2005-2009, TRIUMF ATLAS Tier1 centre is one of the main TRIUMF programs to be undertaken in that 5 year plan

• However, the Canadian political climate has been somewhat turbulent for the last year with an election expected to be called at the end of the year, consequently funding for ATLAS at TRIUMF has not been fully secured?

• TRIUMF’s 5 yr budget request for an ATLAS Tier1 centre was $8M Capital, with 9 more additional FTE’s

• Two FTE’s were hired last year for ATLAS at TRIUMF

• Until Funding is fully secured TRIUMF’s Core Computing & Networking Group is providing the initial 10G infrastructure, along with CANARIE and HEPnet Canada.

The ATLAS Experiment

• 2000 physicists in 34 countries

• ATLAS Canada ~0.6 TIER1• ~80 Physicists• 4.5% of ATLAS Colab.

• Size of 5 story Building

• millions of channels of data

• Discover the Higgs particle

• 30 million events per second

Year 2007 2008 2009 2010

CPU (kSI2K) 7899 26502 47600 81332

DISK (PB) 5.5 15.5 23 42

TAPE (PB) 3 10 18.5 31

Totals for all 10 TIER1’s integrated over the years

1 kSI2K = 3GHz CPU

Tier0 – Tier1 network meeting in Amsterdam Jan ‘05 made it clear all Tier1 sites would require a 10 Gbps network connection to CERN

This would accommodate the expected sustained data rates with head room for efficiency, burst and recovery traffic.

Sept 1 ‘05

3rd week Sept ‘05

End Oct ‘05

Amsterdam

The TRIUMF-CERN 1GbE Lightpath(s)

• TRIUMF• BCNET• CANARIE• SURFnet• CERN

• 1 GbE circuit • established April 18th 2005

• 2nd GbE circuit established July 19th 2005

ATLAS Tier2 in Canada

TRIUMF

Tier 1

Eastern

Tier2

CERN TIER 0

10G

1G

Dedicated e2e LightpathsCan be lightpathsor on the routed 1Gbps CA*net4research network

~5G ? ?Tier 1

There will be two Tier 2’s Centres in Canada, one in the East and one in the West. Each centre will be a distributed across several sites, with one contact point

Western

Tier2

1G

Tier2’s

CWDM & Lightpaths at TRIUMF

/CA*net4

10 GbE @1550nm

2x 1 GbE @1530nm

2x 1 GbE @1510nm

2x 1 GbE @1690nm

Aug 2002

TRIUMF Local loop to Gigapop & CA*net4 PoP ~ 22km

TRIUMF WAN CWDM Equipment

BCNET 22km

10GbEFoundry Switch -> CERN

MRV CWDM

1610 nm1590 nm1570 nm1550 nm

Potential to Add 2 more1GbE channels

Single Pair Fiber

4 1GbE channelsPassport 8600• ORAN• WESTGRID• 2x CERN

SFP 4 Port Optical Mux

2x GbE TDM

Transfer results over 10GbE circuit

TRIUMF Tier1 → Canada Tier2’s

Participating Canadian Tier2 sites in SC3Simon Fraser UniversityUniversity of VictoriaUniversity of AlbertaUniversity of Toronto

T1-T2 network activity Aug 17 - Aug 24 : All four Tier 2 sites participating

A Daily snap-shot of SC3

10 GbE Lightpaths to CERN 2004-05

TRIUMF CERN

Atlantic Crossing√

√√

√

√

√X

Summary …• Currently,

• 2 x 1 GbE circuits TRIUMF to CERN (since April 18th / July 19th 2005)

• 10 GbE Foundry NI40G’s & NI1500 on loan

• In Progress – To be completed by end of year

• Circuits

• Sept 1st 2005 CANARIE/CA*net4 was delivered a 10GbE circuit between Amsterdam and New York

• 3rd week Sept 2005 CA*net4 add 4th OC192 circuit across Canada

• End of October add a 5th OC192 circuit across Canada

• Still waiting on final Lambda between Amsterdam-CERN (GEANT2 ?)

• 10 GbE Equipment

• Last mile DWDM optics for TRIUMF/BCNET expected by Oct/Nov ‘05

• 10G Foundry Rx-4 switch ordered, expect delivery Oct/Nov ‘05