DOSAR Workshop VI

April 17, 2008

Louisiana Tech Site Report

Michael BryantLouisiana Tech University

4/17/2008DOSAR Workshop VI

2

Louisiana Tech University and LONI

COMPUTING IN LOUISIANA

Computing locally at LTU

• At the Center for Applied Physics Studies (CAPS),▫Small eight node cluster with 28 processors

Used by our local researchers and the Open Science Grid Dedicated Condor Pool of both 32-bit and 64-bit (w/ compat)

machines running RHEL5

• Additional resources at LTU through the Louisiana Optical Network Initiative (LONI)▫Dell 5TF Intel Linux cluster (not yet ready)

128 nodes (512 CPUs total) Two 2.33 GHz dual-core Xeon 64-bit processors 4 GB RAM per node

▫ IBM Power5 AIX cluster 13 nodes (104 CPUs)

4/17/2008DOSAR Workshop VI

3

•Focused on High Energy Physics, High Availability (HA), and Grid computing▫Fermilab (DØ), CERN (ATLAS), and ILC:

Dick Greenwood (DOSAR Institutional Rep.), Lee Sawyer, Markus Wobisch

▫High Availability (HA) and Grid computing: Box Leangsuksun Thanadech “Noon” Thanakornworakij Michael Bryant

Researchers at Louisiana Tech4/17/2008DOSAR Workshop VI

4

Louisiana Optical Network Initiative

• A 40Gb/sec fiber optics network linking Louisiana and Mississippi research universities

• Connected to the National LambdaRail (10Gb/sec) and Internet2 in Baton Rouge

• Provides 12 high-performance computing clusters around the state (9 of which are online)

LONI provides Louisiana researchers with one of the most advanced optical networks in the country and the most powerful distributed supercomputer resources available to any academic community with over 85 teraflops of computational capacity.

4/17/2008DOSAR Workshop VI

5

- http://loni.org

LONI Computing Resources• 1 x Dell 50 TF Intel Linux cluster

▫ “Queen Bee” named after Governor Kathleen Blanco who pledged $40 million over ten years for the development and support of LONI.

▫ 668 compute nodes (5,344 CPUs), RHEL4 Two quad-core 2.33 GHz Intel Xeon 64-bit processors 8 GB RAM per node

▫ Measured 50.7 TF peak performance▫ According to the June, 2007 Top500 listing*, Queen Bee ranked the

23rd fastest supercomputer in the world.▫ Half of Queen Bee's computational cycles is contributed to TeraGrid

• 6 x Dell 5 TF Intel Linux clusters housed at 6 LONI member institutions▫ 128 compute nodes (512 CPUs), RHEL4

Two dual-core 2.33 GHz Xeon 64-bit processors 4 GB RAM per node

▫ Measured 4.772 TF peak performance

• 5 x IBM Power5 575 AIX clusters housed at 5 LONI member institutions▫ 13 nodes (104 CPUs), AIX 5.3

Eight 1.9 GHz IBM Power5 processors 16 GB RAM per node

▫ Measured 0.851 TF peak performance

4/17/2008DOSAR Workshop VI

6

* http://top500.org/list/2007/06/100

National Lambda Rail

Louisiana Optical Network

IBM P5 Supercomputers

LONI Members

Dell 80 TF Cluster

NEXT ???

4/17/2008DOSAR Workshop VI

7

LONI: The big picture…by Chris Womack

4/17/2008DOSAR Workshop VI

8

LONI and the Open Science Grid

ACCESSING RESOURCES ON THE GRID

OSG Compute Elements

LONI_OSG1LONI_OSG1 LTU_OSGLTU_OSG

• Official LONI CE (osg1.loni.org)▫ Located at LSU

• OSG 0.8.0 production site

• Connected to Dell 5TF cluster▫ Opportunistic PBS queue▫ 64 CPUs out of 512 CPUs

The 16 nodes are shared with other PBS queues.

• Replaced LTU_CCT▫ Reason: aging hardware

• caps10.phys.LaTech.edu▫ Located at Louisiana Tech

• OSG 0.8.0 production site

• Connected to our small cluster▫ Dedicated Condor Pool▫ 8 nodes, 20 of the 28 CPUs

Other 8 CPUs used by local researchers.

• Waiting to be replaced…▫ LONI_OSG2 → Dell 5TF cluster

4/17/2008

9

DOSAR Workshop VI

• Installed OSG 0.8.0 on node setup by Sam White at LSU at the end of February

• Registered with the OSG and recently validated▫ LTU_CCT was removed from OSG and VORS (3/26/08)

• As of last night, the LONI cluster at LSU (Eric) and LONI_OSG1 are now operational. Thanks to Sam White!

• Simple globus-job-run tests are successful!

• We will begin running DZero and ATLAS production jobs again very soon.▫Need to first modify jobmanager-pbs to submit jobs under our

LONI allocation (using PBS -A project option)

Current Status of LONI_OSG14/17/2008DOSAR Workshop VI

10

Current Status of LTU_OSG

• Upgraded to OSG 0.8.0

• Upgraded compute nodes to RHEL5

• Added two new Dell Precision Workstations (16 CPUs, two quad-core 2.0GHz Xeon 64-bit processors, 16GB and 8GB)

• Connected to LONI 40Gbps network in June 2007

• Running DZero MC production jobs (sent using Joel’s AutoMC daemon)

• Installed standalone Athena 12.0.6 on caps10 for testing ATLAS analysis

4/17/2008DOSAR Workshop VI

11

4/17/2008DOSAR Workshop VI

12

Highly available CEs and PetaShare

LOOKING AHEAD

•Establish three LONI OSG CEs around the state:

OSG Roadmap4/17/2008DOSAR Workshop VI

13

LONI_OSG24/17/2008DOSAR Workshop VI

14

•Provides 200TB disk and 400TB tape storage▫dCache/SRM and DQ2 services

• Initial 10TB at each OSG site for ATLAS production

•More details tomorrow in Tevfik Kosar’s talk…

PetaShare storage4/17/2008DOSAR Workshop VI

15

4/17/2008DOSAR Workshop VI

16

QUESTIONS / COMMENTS?