Linux Cluster and Distributed Resource

Manager

Center for Genome Science, NIH, KCDCHong Chang Bum

hongiiv@gmail.com

Linux Cluster

• 3 Linux Cluster Machine

• KHAN

• KGENE

• BIO

KHAN

• Total 94 Nodes (1 Master + 94 work Node)

• 64Bit PowerPC 720 4-way, 16GB RAM

• 64Bit PowerPC 770 2-way, 2GB RAM

• User Space: /home1 1.6TB

• Scratch Space: /home2/scratch 935GB

• Software: /home1/biosoftware

• EIGNSTRAT, merlin, phase, plink, R ...

DRM

• Distribute Resource Manager

• Job Scheduler

• IBM LoadLeveler, PBS, OpenPBS, Torqueue, Grid Engine, Sun Grid Engine(SGE), LSF

• Job Distribute, Job Status, Queue status, Node Status, User Accounting

1,000 Serial Job

• 1,000 Data Sets

• input.1, input.2... input.1000

• program -i input.(1~1000) -o output.(1~1000)

• 1,000 Job Submit Scripts

• 1,000 times Queue Submit Commands

Job Array - LoadLeveler

• Large Data sets: 1,000(input.1 ~ input.1000)

1,000 times??

Using Script

1,000 Command Script

1,000 Job Submit Command

1,000 Command Script

1,000 Job Submit Command

Job Status

Using R Scripts• Interactive Program

• Need to use R to analyze your data

Convert Bash Shell Script??

--quiet, -q Don’t print startup message--no-save Don’t save it

Using I/O redirection Form

R and LoadLeveler