Tier2 Centre in Tier2 Centre in PraguePrague
JiJiří Chudobaří Chudoba
FZU AV ČR - Institute of Physics of tFZU AV ČR - Institute of Physics of the he Academy of SciencesAcademy of Sciences of the Czech Republic of the Czech Republic
19.10.2005, FZK19.10.2005, FZK [email protected]@fzu.cz22
OutlineOutline
Supported groups Supported groups HardwareHardware Middleware and softwareMiddleware and software Current statusCurrent status
19.10.2005, FZK19.10.2005, FZK [email protected]@fzu.cz33
Particle Physics in the Particle Physics in the Czech RepublicCzech RepublicGroups located atGroups located at Charles University in PragueCharles University in Prague Czech Technical University in PragueCzech Technical University in Prague Institute of Physics of the Academy of Sciences of the Czech Institute of Physics of the Academy of Sciences of the Czech
RepublicRepublic Nuclear Physics Institute of the Academy of Sciences of the Nuclear Physics Institute of the Academy of Sciences of the
Czech RepublicCzech Republic
Main ApplicationsMain Applications Projects ATLAS, ALICE, D0, STAR, TOTEMProjects ATLAS, ALICE, D0, STAR, TOTEM Groups of theoreticiansGroups of theoreticians Approximate size of the communityApproximate size of the community
6060 scientistsscientists, 2, 200 engineers engineers, 2, 200 technicianstechnicians andand 4 400 students students and PhD students and PhD students
19.10.2005, FZK19.10.2005, FZK [email protected]@fzu.cz44
HardwareHardware
2 independent computing farms2 independent computing farms– Golias located in FZUGolias located in FZU– Skurut from CESNET Skurut from CESNET (= “academic network provider”)
older hardware (29 dual nodes, PIII700MHz) older hardware (29 dual nodes, PIII700MHz) offered by CESNEToffered by CESNET
part used as a production farm, some nodes for part used as a production farm, some nodes for tests and support for different VOs (VOCE, tests and support for different VOs (VOCE, GILDA, CE testbed)GILDA, CE testbed)
contribution to Atlas DC2 at a level of 2% of all contribution to Atlas DC2 at a level of 2% of all jobs finished on LCGjobs finished on LCG
19.10.2005, FZK19.10.2005, FZK [email protected]@fzu.cz55
Available Resources - Available Resources - FZUFZU Computer hall Computer hall
– 2 x 9 racks2 x 9 racks– 2 air condition units2 air condition units– 180 kW electrical power 180 kW electrical power
available from UPS, backed available from UPS, backed up by Diesel generatorup by Diesel generator
– 1 Gbps optical connection to 1 Gbps optical connection to CESNET metropolitan networkCESNET metropolitan network
– direct 1 Gbps optical direct 1 Gbps optical connection to CzechLight connection to CzechLight
– shared with other FZU shared with other FZU activitiesactivities
19.10.2005, FZK19.10.2005, FZK [email protected]@fzu.cz66
Hardware @ FZUHardware @ FZU Worker NodesWorker Nodes
(September 2005)(September 2005)– 6767x x HP DL140, HP DL140, dual dual Intel Intel
Xeon 3.06Xeon 3.06 GHz withGHz with HT HT (enabled only on some (enabled only on some nodes), 2 or 4 GB RAM, 80 nodes), 2 or 4 GB RAM, 80 GB HDDGB HDD
– 11x dual AMD Opteron 1.6 x dual AMD Opteron 1.6 GHzGHz, 2 GB RAM, 40 GB HDD, 2 GB RAM, 40 GB HDD
– 24x HP LP1000r, 2xPIII1.13 24x HP LP1000r, 2xPIII1.13 GHz, 1 GB RAM, 18 GB SCSI GHz, 1 GB RAM, 18 GB SCSI HDDHDD
– WN connected via 1 Gbps WN connected via 1 Gbps (DL140) or 100 Mbps (DL140) or 100 Mbps (LP1000r)(LP1000r)
Network componentsNetwork components– 3x HP ProCurve Networking 3x HP ProCurve Networking
Switch 2848 (3x48 ports)Switch 2848 (3x48 ports)– HP 4108GL HP 4108GL
~ 30 KSI2K will be added this year
19.10.2005, FZK19.10.2005, FZK [email protected]@fzu.cz77
Golias Farm Hardware - Golias Farm Hardware - serversservers
PBS server: HP DL360 – 2xPBS server: HP DL360 – 2x Intel Xeon 2.8Intel Xeon 2.8, 2 GB RAM, 2 GB RAM CE, SE, UI: HP LP1000, 2xCE, SE, UI: HP LP1000, 2x1.13Ghz PIII, 1 GB RAM, 100 1.13Ghz PIII, 1 GB RAM, 100
Mbps (SE should be upgraded to 1 Gbps soon)Mbps (SE should be upgraded to 1 Gbps soon) NFS serversNFS servers
– 11x x HP DL145 – 2xHP DL145 – 2x AMD Opteron 1.6 GHz AMD Opteron 1.6 GHz, connected , connected disc array 30 TB (raw capacity), ATA discsdisc array 30 TB (raw capacity), ATA discs
– 1x HP LP4100TC, 1 TB disc array, SCSI discs1x HP LP4100TC, 1 TB disc array, SCSI discs– 1x embedded server in EasySTOR 1400RP (PIII), 10 TB, 1x embedded server in EasySTOR 1400RP (PIII), 10 TB,
ATA discsATA discs dCache serverdCache server
– HP DL140 upgraded by raid controller, 2x300GB discsHP DL140 upgraded by raid controller, 2x300GB discs– not used for productions, reserved for SC3 not used for productions, reserved for SC3
Some other servers (www, SAM)Some other servers (www, SAM)
19.10.2005, FZK19.10.2005, FZK [email protected]@fzu.cz88
Middleware, Batch Middleware, Batch SystemSystemGOLIAS: GOLIAS:
– LCG2 (2_6_0): LCG2 (2_6_0): CE, SE, UI – SLC3 CE, SE, UI – SLC3 WNs - RH7.3 (local D0 group not yet ready for WNs - RH7.3 (local D0 group not yet ready for
SLC3)SLC3)
– PBSPro server not on CEPBSPro server not on CE CE submits jobs to the node golias (PBSPro server)CE submits jobs to the node golias (PBSPro server) local users can submit to local queues on goliaslocal users can submit to local queues on golias
SKURUT:SKURUT:– LCG2 (2_6_0), OpenPBS server on CELCG2 (2_6_0), OpenPBS server on CE– all nodes - SLC3all nodes - SLC3
19.10.2005, FZK19.10.2005, FZK [email protected]@fzu.cz99
QueuesQueues
Separate queues for different Separate queues for different experiments and privileged users:experiments and privileged users:atlas, atlasprod, lcgatlas, lcgatlasprodatlas, atlasprod, lcgatlas, lcgatlasprodalice, aliceprod, d0, d0prod, auger, star, ...alice, aliceprod, d0, d0prod, auger, star, ...short, longshort, long
priorities are set by some PBS priorities are set by some PBS parameters:parameters:– max_running, max_user_running, priority, max_running, max_user_running, priority,
node properties node properties still not optimal in heterogeneous still not optimal in heterogeneous
environmentenvironment
19.10.2005, FZK19.10.2005, FZK [email protected]@fzu.cz1010
Jobs statistics 2005/1-Jobs statistics 2005/1-66
Used CPU time (in days) per activity, for January – June 2005Used CPU time (in days) per activity, for January – June 2005
10553
4974
219455 1 886 23
d0 atlas starprod auger
alice long short other
19.10.2005, FZK19.10.2005, FZK [email protected]@fzu.cz1111
Simulations for ATLASSimulations for ATLAS
All PILEUP jobs on LCG
TRIUMF; 153; 2%
TAIWAN; 3207; 33%
PRAGUE; 3843; 38%
ROMA; 1718; 18%
IFAE; 871; 9%
19.10.2005, FZK19.10.2005, FZK [email protected]@fzu.cz1212
ALICE PDC – Phase2 ALICE PDC – Phase2 (2004)(2004)
2004: ALICE jobs submitted to Golias via AliEn2005: new version of AliEn is being installed
19.10.2005, FZK19.10.2005, FZK [email protected]@fzu.cz1313
Tier1 – Tier2 relationsTier1 – Tier2 relations
Requirements defined by experiments Requirements defined by experiments ATLAS and ALICEATLAS and ALICE
Direct network connection between Direct network connection between FZU and GridKa will be provided by FZU and GridKa will be provided by GEANT2 next yearGEANT2 next year
““Know-how” exchange welcomedKnow-how” exchange welcomed
19.10.2005, FZK19.10.2005, FZK [email protected]@fzu.cz1414
THE END