Reedbush Quick Start Guide - 東京大学€¦ · –Intel C,C++,Fortran Compiler –Intel MPI...

Reedbush Quick Start Guide

Information Technology Center

The University of Tokyo

Updated at 21st Sep. 2018

Index

1. Index

2. Cygwin installation (1)

3. Cygwin installation (2)

4. Building SSH key

5. Access to Web Portal

6. Change Password

7. Change password successfully

8. Re-login to Web portal

9. Upload public key (1)

10.Upload public key(2)

11.Upload public key(3)

See from “4. Building SSH key” if you are UNIX/Mac user. 1

12.Upload public key(4)

13.Login

14.Your Directory

15.Building program

16.Compile and Run

17.Batch script

18.Batch script sample(1)

19.Batch script sample(2)

20.Running Batch job

21.Deleting batch job

22.Application software

Cygwin installation (1)

• Install Cygwin

– Download setup-x86_64.exe(64bit) or setup-x86.exe(32bit) from http://cygwin.com/index.html, run it.

– Click X11’s Default at Package selection window, it change to “Install”

Click X11’s Default It change to ”Install”, they are ready to be installed.

2

Cygwin installation (2)

– Search OpenSSH and click it. It will change install target from “Skip”.

Search OpenSSH

Box is changed to checked if you click.

3

Building SSH key

• Open Cygwin64 Terminal or UNIX/Mac Terminal.

• Run ssh-keygen.

– Public key “id_rsa.pub” and private key “id_rsa” is build on your home directory/.ssh.

$ ssh-keygen -t rsa Generating public/private rsa key pair. Enter file in which to save the key (/home/username/.ssh/id_rsa): Created directory '/home/username/.ssh'. Enter passphrase (empty for no passphrase): Enter same passphrase again: Your identification has been saved in /home/username/.ssh/id_rsa. Your public key has been saved in /home/username/.ssh/id_rsa.pub. The key fingerprint is: SHA256:NedyH2JayWknqq87pQIvaLVs3omJK2ZijB8iR9Qnwf4 username@host The key's randomart image is: +---[RSA 2048]----+ | .. | | ... | | ..o . o . | | . .o . = o | | . . S . % o | | . o E .O = . | |= o+ + oo . | |=Boo=+o.o. | |=++++.o.+=. | +----[SHA256]-----+ $

Run ssh-keygen

Enter

Enter your passphrase

Re-Enter your passphrase

4

Access to Web Portal

• Access to following site with your browser.

– https://reedbush-www.cc.u-tokyo.ac.jp

• Login to Web Portal.

1．Enter your account.

2．Enter password.

3．Click “Login”

5

Change Password • Change your password on “Change Password” for the first

time.

• Password policy – more than 8 characters.

– the use of both upper-case and lower-case letters

– inclusion of one or more numerical digits

– inclusion of special characters, such as @, #, $

– not be a dictionary word

6

1．Enter current password

2．Enter new password

3．Re-Enter new password

Change password successfully

7

• Click “Return to Login”

• Return to Login and retry if you failed to change password.

1. Click “Return to Login”

Re-login to Web portal

• Login with new password.

8

1．Enter your account.

2．Enter password.

3．Click “Login”

Upload public key (1) • Click “SSH Configuration”

• At first time, ”No authorized keys have been registered” appears.

• Click “Add a new SSH 2 authorized key.”

9

1．Click “SSH Configuration”

1．Click “Add a new SSH 2 authorized key.”

Upload public key(2) • “Add Authorized Key” window

• Copy your public key “HOMEDIR/.ssh/id_rsa.pub” on your terminal.

10

Upload public key(3) • Paste this public key on “Key digits”

– Confirm key doesn’t include carriage return before you paste it.

• Confirm your public key format is “ssh-rsa key-digits key’s name (user@host)”.

11

Paste your public key.

Click “Create”

Confirm that ssh-rsa or ssh-dss is in header, new line character is not included. RSA key should be more than 2048bits, DSA key should be more than 1024bits.

Upload public key(4) • Confirm your key is registered at “Authorized Keys”.

• “No authorized keys have been registered” appears if your key is invalid.

12

Login • Login to Reedbush-u frontend server with ssh.

• Hostname:reedbush.cc.u-tokyo.ac.jp

% ssh -l username reedbush.cc.u-tokyo.ac.jp Authenticating with public key "rsa-key-20160612" The authenticity of host 'localhost (::1)' can't be established. RSA key fingerprint is XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX Are you sure you want to continue connecting (yes/no)? yes Warning: Permanently added 'localhost' (RSA) to the list of known hosts. Passphrase for key ""rsa-key-20160612": [username@reedbush-u1 ~]#

It appears for the first time only. Answer “yes”.

Enter passphrase.

13

Your Directory • /home/group_name/account_name

– Default directory when you login.

– Setting environment variables

– Maximum 2GB

– Home directory is unavailable to run batch job because compute node doesn’t mount it.

• /lustre/group_name/account_name

– Working directory

– [number of subscription nodes]TB: Group Course 4TB: when your group subscribes 4 nodes.

8TB: when your group subscribes 8 nodes.

– 1TB: Personal Course

– Build and run your program at this directory.

14

Building program • Default development environment

– Intel C,C++,Fortran Compiler

– Intel MPI

– You are able to change your development environment with module command.

• module – You can switch compiler and mpi environment with module

command.

– When you run batch job, you have to load same modules as when you compile.

15

List your module environment

$ module list Currently Loaded Modulefiles: 1) intel/16.0.3.210 2) intel-mpi/5.1.3.210 Load module

$ module load intel/16.0.3.210 intel-mpi/5.1.3.210

Compile and Run • Compile and Run

– Serial job (Intel Compiler) • icc、ifort

– Parallel job（Intel MPI） • mpiicc、mpiifort

16

Example $ mpiicc -o a.out -O2 mpiprogram.c $ mpirun -np 2 ./a.out Hello World from Process 0 of 2 running on reedbush-u1 Hello World from Process 1 of 2 running on reedbush-u1

Batch script • Move to your /lustre directory, make batch script.

• The following lines is necessary. – #!/bin/sh

– #PBS -q [Queue name]

– #PBS -l select=[number of nodes]:mpiprocs=[number of MPI processes/per node]:ompthreads=[number of OpenMP threads/per proccess]

– #PBS -W group_list=[group name]

– #PBS -l walltime=[wall time]

– cd $PBS_O_WORKDIR

– [Initializing module]

– [Load programing environment]

– [Running program]

• Option – #PBS -N [Job name] 17

Batch script sample(1) • Be sure to make and run on your /lustre directory.

• See run.sh(red character)

• Select appropriate queue for walltime and number of nodes which you’d like to use.

18

$ cd /lustre/group/username $ cat ./run.sh #!/bin/sh #PBS -q u-short #PBS -l select=1:mpiprocs=1:ompthreads=1 #PBS -W group_list=mygroup #PBS -l walltime=02:00:00 cd $PBS_O_WORKDIR . /etc/profile.d/modules.sh module load intel/16.0.3.210 ./a.out > ./output.log 2>&1

Queue name Number of nodes

Walltime

u-debug 1-16 30min

(u-interactive) - -

u-interactive_1 1 30min

u-interactive_4 2-4 10min

u-short 8 4H

(u-regular) - -

u-small 4-16 48H

u-medium 17-32 48H

u-large 33-64 48H

u-x-large 65-128 24H

Ex) Serial job

Your job will fail if it uses over 244GB per node.

Batch script sample(2) • See run.sh(red character)

• Select appropriate queue for walltime and number of nodes which you’d like to use.

19

$ cd /lustre/group/username $ cat ./run.sh #!/bin/sh #PBS -q u-short #PBS -l select=8:mpiprocs=32:ompthreads=1 #PBS -W group_list=mygroup #PBS -l walltime=02:00:00 cd $PBS_O_WORKDIR . /etc/profile.d/modules.sh module load intel/16.0.3.210 intel-mpi/5.1.3.210 mpirun ./a.out > ./output.log 2>&1

256MPI job （8nodes x 32 MPI processes ）

$ cd /lustre/group/username $ cat ./run.sh #!/bin/sh #PBS -q u-short #PBS -l select=8:mpiprocs=2:ompthreads=18 #PBS -W group_list=mygroup #PBS -l walltime=02:00:00 cd $PBS_O_WORKDIR . /etc/profile.d/modules.sh module load intel/16.0.3.210 intel-mpi/5.1.3.210 mpirun ./a.out > ./output.log 2>&1

Hybrid parallel job （8nodes x 2 MPI processes x 18 OpenMP threads）

Your job will fail if it uses over 244GB per node.

Running Batch job • qsub to submitting batch script

– Example

• rbstat to show job status

– Example

20

$ qsub run.sh 14.reedbush-pbsadmin0

Job ID is returned.

qsub [batch script file]

$ rbstat JOB_ID JOB_NAME STATUS PROJECT QUEUE START_DATE ELAPSE TOKEN NODE 123 STDIN RUNNING xxxx u-short 06/27 19:56:41 00:00:00 0.0 1 124 STDIN QUEUED xxxx u-short 06/27 19:56:48 00:00:00 0.0 1

Deleting batch job • qdel to delete batch job.

• Example

21

$ qdel [Job ID]

$ rbstat

JOB_ID JOB_NAME STATUS PROJECT QUEUE START_DATE ELAPSE TOKEN NODE 123 STDIN RUNNING xxxx u-short 06/27 19:56:41 00:00:00 0.0 1 124 STDIN QUEUED xxxx u-short 06/27 19:56:48 00:00:00 0.0 1 $ qdel 124 $ rbstat JOB_ID JOB_NAME STATUS PROJECT QUEUE START_DATE ELAPSE TOKEN NODE 123 STDIN RUNNING xxxx u-short 06/27 19:56:41 00:00:00 0.0 1

Application software • A list of software installed on Reedbush-u system is

shown by using “module avail”

• Load application module which you use.

22

$ module avail -------------------------------- /lustre/app/modulefiles/compiler --------------------------------------------------- intel/16.0.3.210(default) pgi/16.5 ------------------------------------ /lustre/app/modulefiles/mpi ----------------------------------------------------- hpcx/1.6.392/gnu hpcx/3.3-1.0.0.0/intel_ucx mvapich2/2.2rc1/intel openmpi/1.4.5/intel hpcx/1.6.392/intel intel-mpi/5.1.3.210(default) mvapich2/2.2rc1/pgi openmpi/1.8.3/gnu hpcx/3.3-1.0.0.0/gnu mpt/2.14 openmpi/1.10.2/gnu hpcx/3.3-1.0.0.0/gnu_ucx mvapich2/2.1.ddn1.3/gnu openmpi/1.10.2/intel hpcx/3.3-1.0.0.0/intel mvapich2/2.2rc1/gnu openmpi/1.10.2/pgi ------------------------------------ /lustre/app/modulefiles/lib ----------------------------------------------------- boost/1.61 mt-metis/0.4.4 openjdk/1.8.0.91-0.b14 revocap_coupler/2.1 superlu_mt/3.1 gsl/2.1 netcdf-c/4.4.0 parallel-netcdf/1.7.0 revocap_refiner/1.1.01 xabclib/1.03 massivethreads/0.95 netcdf-cxx/4.3.0 parmetis/4.0.3 scotch/6.0.4 metis/4.0.3 netcdf-fortran/4.4.4 petsc/3.7.1 superlu/5.2.0 metis/5.1.0 opencv/3.1.0 ppohAT/1.0.0 superlu_dist/5.1.0 -------------------------------- /lustre/app/modulefiles/application ------------------------------------------------- abinit-mp/7.0 bwa/0.7.13 gatk/3.5 openfoam/3.0.1 samtools/1.3.1 akaikkr/cpa2002v009c feram/0.24.02 gromacs/5.1.2 openfoam/3.0.1-mpt spark/1.6.1 alps/2.1.1-r6176 frontflow_blue/8.1 kmr/1.8.1 openmx/3.8 xtapp/150401 bioperl/1.6.924 frontflow_blue/8.1_gnu modylas/1.0.4 phase0/2015.01 bioruby/1.5.0 frontistr/4.4 ncbi-blast/2.3.0 r/3.2.5 $ module load openfoam/3.0.1