Amsterdam May 19-23,2003 Site Report Roberto Gomezel INFN - Trieste

AmsterdamMay 19-23,2003

Site ReportRoberto Gomezel

INFN - Trieste

2AmsterdamMay 19-23,2003

Outline of Presentation• Introduction

• Environment today

• Services

• Network

• AFS

• Condor

• Tier 1 at CNAF

• INFN Windows Report (Gian Piero Siroli)


Computing Environment and security

• 90% of boxes are PCs running Linux or Windows • Mac OS boxes keep on living and growing• Commercial unix boxes only used for specific tasks or

needs• VPNs not yet available but they are supposed to be

configured almost everywhere by the end of this year (Cisco – NetScreen boxes using IPsec)

• Network Security– No dedicated Firewall machines– Implemented with access lists on router connected to WAN

INFN Site Report – R.Gomezel


Desktop• PCs running Linux and Windows

• Automatic installation using Kickstart for Linux and RIS for Windows

• Metaframe Citrix or Vmware used to reduce the need to install Windows OS on all PCs for desktop applications

• A few sites chose to outsource support for desktop environment due to lack of personnel



Backup• Tape Libraries used:

– DLT - widespread– AIT2 – a few sites– Exabyte Mammoth2 – poor used– IBM Magstar – just used at LNF– LTO – many sites are moving from DLT into LTO technology

• Backup tools:– IBM Tivoli – quite used– HP Omniback – quite used– Atempo Time Navigator – just a few sites– Domestic tool - widespread



Wireless LAN• Access point running standard 802.11b

• All sites are using wireless connection as meeting or conferences are running

• Most of them use it to give connection to laptop computers

• Security issues:– Permission based on Secure Port filtering (MAC Address) –

poor security

– No encryption used

– Open problem



E-mail • Mail server

– Sendmail – widespread and more used

– Postfix – a few sites

• Mail access protocol – POP3

– IMAP – IMAP over SSL

– Secure Webmail



INFN network• LAN backbone network mainly based on

Gigabit Ethernet

– Layer 2 and 3 switching

– No layer 4 switching

• The INFN WAN network is completely integrated into the GARR, nation-wide infrastructure, providing a backbone connectivity at 2.5 Gigabit

• Since the 6th of May 2003 GEANT, the Pan-European Research Network, and GARR have been interconnecting at 10Gbit/sec.

• 3 * 2.5Gbit/sec to North America via GEANT

network for research traffic to USA and global internet



How we share data today• INFN sites heavily utilize AFS services to share data and software

throughout sites and both AFS and NFS within local site

• Local cells have already moved or are moving server functionality to Linux boxes running OpenAFS software

• Authentication and file servers of the nation-wide cell INFN.IT are moving from Tru64 boxes to Linux boxes running OpenAFS by the end of summer

• Almost everywhere disks are organized in RAID array system (SCSI and EIDE) to provide storage to farm and central computing facility

• There is an increasing usage of NAS and SAN architecture in order to rule over the complexity and to improve the reliability of data



INFN Condor Pool

• Condor converts a collection of unrelated workstations into a high-throughput computing facility.

Minimize Wait while IdleAnd:

• …increase throughput.

• …do housekeeping.

• …improve reliability

by P.Mazzanti – F. Semeria



The ‘Condor on WAN’ INFN Project

• Approved by the Computing Committee on February 1998.• Goal: install Condor on the INFN WAN and evaluate its

effectiveness .• Collaboration INFN-CS Madison-Wisconsin

• It has been running as a production tool since 1999




The INFN-WAN Pool

• Used by many INFN researchers.• The first example in Europe of a national

distributed computing environment.• More than 200 CPUs in the INFN WAN

Condor Pool• Avarage Pool Utilization (last few years)

~ 80 cpu years




GARR-B Topology

155 Mbps ATM based Network

access points (PoP)

main transport nodes

radio wave bridge 34M user access E1-E3

TO PD

BARI

PALERMO

FIRENZE

PAVIA

MI

19

GENOVA

NA

CAGLIARI

ROMA

PISA

L’AQUILA

CATANIA

BO

UDINE

TRENTO

PERUGIA

LNF

LNGS

SASSARI

LECCE

LNS

LNL

Europe/US

155Mbps

T3

SALERNO

COSENZA

S.Piero

FERRARA

A

CNAF87

Central Manager

CKPT domain # hosts

INFN Condor Pool on WAN: checkpoint domainsApril 2003

ROMA2

32

12

25

11

42

Default ckptdomain in cnaf

PV 11

71

US

Ts6




Condor At Large

• Growing use in commercial world– Oracle: automated software building & testing

– Micron: chip design, simulation, defect analysis, testing

– Leica Geosystems: image analysis

– many others…

• Over 400 Condor pools in production worldwide– USA, Italy, Mexico, Brazil, UK, Germany, Spain, France,

Poland, Hungary, more…

– More than 14,000 CPUs




INFN – TIER1 • INFN computing facility for HNEP community

– Location: INFN-CNAF, Bologna (Italy)• One of the main nodes on GARR network

– Ending prototype phase this year– Fully operational next year

• Multi-experiment– LHC experiments, Virgo, CDF– BABAR (3rd quarter 2003)– Resources dynamically assigned to experiments

according to their needs• Main (~50%) Italian resource for LCG

– Coordination with Italian TIER2s, TIER3s– Participation to grid test-beds (EDG,EDT,GLUE)– Participation to CMS, ATLAS, LHCb Data Challenge

by L. dell’Agnello – F. Ruggieri



LAN TIER1

FarmSW1 (*)

FarmSW2(*)

FarmSWG1 (*)

FarmSW3(*)

Switch-lanCNAF (*)

SSR2000

Catalyst6500

Fcds1 Fcds2

8TF.C.

2TSCSI

NA

S2

131.154.99.192

NA

S3

131.154.99.193

Fcds3

LHCBSW1 (*)

LAN CNAF 1 Gbps

GARR

1 Gbps link

(*) vlan tagging enabled




TIER1 Resources (1)

• Computing servers (CPU farms)– 150 (320 next summer) 1U bi-processors Pentium

III/IV 800-2400 MHz – System installation & administration

• Linux RedHat (6.2, 7.2, 7.3)• Experiment specific library software• LCFG (WP4 EDG)

• Access to on-line data (DAS, NAS, SAN)– 35 TB (> 70 TB next summer)– Study of Large File System solutions

• GFS• GPFS

– “SAN on WAN” tests (collaboration with CASPUR)– Test of several Hw technologies (EIDE, SCSI, FC)




TIER1 Resources (2)• Mass storage/tapes

– StorageTek library with 9840 and LTO drives: 180 tapes (100 GB each)

– New library with 2000-5000 tapes next summer– CASTOR as front-end software for archiving

• Direct access for end-users• Oracle as back-end

• Helpdesk– Support for our users– Synergy with EDG testbed sitemanagers– GOC (Deployment in progress)

• Personnel: ~ 10 FTE’s




INFN Windows Infrastructure• Estimated total number of Windows nodes: ~6-7000

• Overall platform distribution (large variations among different sites)– W2K: 50%

– XP: 16%

– W/NT: 9%

– W9x: 25%

W2K

XP

W/NT

W9x

INFN Windows Report – G.P.Siroli


INFN Windows Infrastructure• Domains:

– About 1/3 of the sites have just standalone systems

– The majority of sites have more than one domain (W2K, W/NT, specific services or groups) collecting the majority on nodes; 1 site using SMS

– User generally has local Admin privileges

– Metaframe to integrate non-Windows world

– About 30% of the sites use some (re)installation or cloning tool (ghost, drive image, RIS or home made)



INFN Windows Infrastructure• Storage:

– About 50% of the sites have a central server with backup (or RAID)

– A few sites use SAMBA and AFS

– Still many sites have no central storage management



INFN Windows Infrastructure• Security and WAN access:

– Mostly no WAN access; use of Windows Terminal Server (WTS) and Metaframe; 1 VPN/Windows

– LAN access open; a few sites use VNC for remote management

– Antivirus s/w almost everywhere (standalone or local server)

– Windows Update regularly used by about 30% of the sites (in same cases irregular use or suggested to the user)

– Only very few sites don’t enforce some level of security (user self management)


Documents

Amsterdam May 19-23,2003 Site Report Roberto Gomezel INFN - Trieste