Upload
andy-sweepstaker
View
60
Download
1
Tags:
Embed Size (px)
DESCRIPTION
Unix tutorial Basics
Citation preview
www.msi.umn.edu
Unix Basics
Presented by: Patton FastUser Support Grouphelp-line: [email protected] 612-626-0802personal: [email protected] 612-625-6573
Research.We make it happen.
www.msi.umn.edu
I. Warnings!
II. Basic Concepts
III. Basic Commands
IV. The Fun Stuff
V. Resources
Please Ask Questions!
Outline
www.msi.umn.edu
• If you delete a file, it is GONE! The file can only beretrieved from a backup.
• If you overwrite a file it has been changed forever.The original file can only be restored from a backup.
• Home directories are backed up nightly.
• The scratch spaces are NOT backed up!
Bottom line? Be Careful With Your Files!
Warnings!
www.msi.umn.edu
Scratch Space Time Limits!
• Regatta (/regscratch1 and /regscratch2) – 7 days, 50 Gb
• SP (/scratch1 and /scratch2) – 7 days, 50 Gb
• Altix (/scratch1 and /scratch2) – 14 days
• Netfinity (/scratch1 and /scratch2) – 14 days
• SDVL (/scratch) – 14 days
• BSCL (/wrk) – 14 days
Bottom line? Be Careful With Your Files!
Warnings!– scratch space
www.msi.umn.edu
Core Resource Service Units
• IBM Power4 – 3.5 hours / su
• SP Winterhawk+ – 5 hours / su
• SP Nighthawk – 10 hours / su
• Altix – 3.5 hours / su
• Netfinity – 12 hours / su
Warnings!– service units
www.msi.umn.edu
IBM Power4364 Power4 processors603 Gb of memory
6 Tb of storage
SGI Altix432 Itanium2 processors876 Gb of memory
4 Tb of storage
IBM Linux Cluster160 Intel processors226 Gb of memory2.5 Tb of storage
IBM SP400 Power3 processors675 Gb of memory
4 Tb of storage
Basic Concepts– core resources
www.msi.umn.edu
Computational Genetics Lab16 CPU SunFire 68004 Dell PCs1 Mac G51 SGI Fuel1 SGI Tezro1 IBM IntelliStation
Basic Sciences Computing Lab48 CPU SGI Altix16 CPU SGI Onyx49 SGI Octanes8 CPU Sun UltraSparcIII2 Linux Workstations1 SunBlade 2000
10 Tb of storage64” x 80” stereo system
Scientific Dev. & Vis. Lab9 SunBlade 2000s7 SGI Octanes1 SGI Onyx
Basic Concepts– laboratory resources
Med. Chem./Supercomputing Inst.Vis. Workstation Lab
3 SGI Octanes3 SGI O2’s2 SunBlade 2000s
http://www.msi.umn.edu/bsclhttp://www.msi.umn.edu/vwl
http://www.msi.umn.edu/sdvlhttp://www.msi.umn.edu/cgl
www.msi.umn.edu
Basic Concepts– laboratory resources
Basic Sciences Computing Lab• Proteomics• Structural Biology
Computational Genetics Lab• Bioinformatics• Genomics
Lab for Large Scale Data Analysis• Data Mining• Database Modeling
Med. Chem./MSI Visualization Workstation Lab• Drug Design• Molecular Modeling
Scientific Development & Visualization Lab• Computational Fluid Dynamics• Scientific Computation
www.msi.umn.edu
After your first login you should change your initial passwordto something that is easy for you to remember, but not easyfor someone to guess.
%> yppasswd (Institute machines)
%> passwd (general)
You should NEVER give your password to anyone!
If you forget your password, send a message [email protected] and your password will be reset.
Note: Never share accounts or passwords!
Basic Concepts– changing passwords
www.msi.umn.edu
Initial PasswordsYour initial password should be the same for all machines within a givenlab, regardless of operating system.
Unix, Linux, Mac OS XAll of the Unix based machines share the same password file. When youchange your password on one of the machines within a given lab, it ischanged for all of the machines.
WindowsAll of the Windows bases machines share the same password file acrossall labs. When you change your password on one machine it is changedfor all machines regardless of the lab.
Basic Concepts– changing passwords
www.msi.umn.edu
Basic Concepts– console
Console Login:
• Enter username
• Enter password
Using the keyboard and mouse physically attached to themachine.
Console Logout:
Logout procedures vary between consoles. Make sure youlogout! Otherwise your files and account are open to thenext person. You could be logged out by an administrator.
www.msi.umn.edu
ssh [options] machine secure shell; a program for logging into a remotemachine providing encrypted communicationsbetween machines
options:-l login name-X sets environment variables for porting X-display
example:
type: ssh -l pfast sp open a secure connection for the user pfast onthe machine sp.msi.umn.edu
type: ssh bi2 open a secure connection to bi2.msi.umn.eduwith current username
SSH clients:
http://www.msi.umn.edu/user_support/ssh
Basic Concepts– remote login
www.msi.umn.edu
Why?
• So that you get all of your email sent to one account.
• Makes it easier for sys admins to communicate with you.
How?
1) Create the file .forward in your home directory
2) On the first line:
3) Create this same file in all of your accounts that you wantto forward.
Basic Concepts– forwarding email
www.msi.umn.edu
man [options] command displays manual pages about command-k keyword search for a command
info command displays Info document of command
When in doubt, check the man/info pages!
Need extra details? Check the man/info pages!
Basic Concepts– command line help
www.msi.umn.edu
Basic Concepts– file structure overview
/
bin@ home/ lib@ scratch/ usr/
user1/ user2/
file1
mail/ source/ work/
proj1/ proj2/
.cshrc
input output
bin/ lib/ local/
user1/ user2/
dir1/ dir2/
file1/ file2/directory ( / )executable ( * )link ( @ )
more*
www.msi.umn.edu
Basic Concepts– metacharacters
? single character wild card
* wild card, arbitrary number of characters
~ home directory of current user
~name home directory of user name
. current directory
.. parent directory (back one)
www.msi.umn.edu
command [options] argument
example:
ls -al /scratch
command name
[options] (usually preceded by a dash)
argument (quite often a filename or directory)
Note: UNIX commands, options, and arguments are allCasE seNsiTive!
Basic Concepts– command anatomy
www.msi.umn.eduCommand Description
ls [options] dirname list the contents of dirname
pwd display full pathname
mkdir dirname create the directory dirname
cd dirname change to the directory dirname
mv [options] source destination move a file or directory
cp [options] source destination copy a file or directory
rm [options] source remove a file or directory
ssh [options] hostname remote login
scp [options] user@host1:file user@host2:file remote copy; file transfer
Basic Commands
skip basic commands
www.msi.umn.eduls [options] dirname list the contents of dirname
options:
-a list all files including hidden files[hidden files are preceded by a ‘.’; eg .cshrc]
-l long listing showing ownership, permissions,and links
examples:
type: ls /home/user/temp view the contents of a directory with absolutepathname /home/user/temp
type: ls ../../temp list the contents of a directory using a relativepath.
Listing Contents
www.msi.umn.edu
mkdir dirname create the directory dirname
examples:
type: mkdir work create the directory work/ in the current workingdirectory
type: mkdir work/proj1 create the directory proj1/ in the work/ directory
type: mkdir /wrk/user2 create the directory user2/ in the /wrk/ directory
Make Directory
www.msi.umn.edu
cd change to your home directory
cd dirname change to the directory dirname
examples:
type: cd ~tom change to tom’s home directory
type: cd /wrk/user2 change to the directory /wrk/user2
Change Directory
www.msi.umn.edu
mv [options] presname newname rename a file
mv [options] srcfile destdir move a file to anotherdirectory
options:
-i confirm overwrites
example:
type: mv ~user1/file ./outputfile moves file from user1’s homedirectory to the current workingdirectory and renames it outputfile
Note: Be careful when overwriting files!
Moving or Renaming
www.msi.umn.edu
cp [options] srcfile destfile copy a file to another filename
cp [options] srcfile destdir copy a file to another directory
options:
-i confirm overwrites-R recursively copy a directory and its
contents, copies symbolic links
example:
type: cp -R proj1 proj2 copy the directory proj1/ and name itproj2/
Note: Be careful when overwriting files!
Copying
www.msi.umn.edu
cp [options] srcfile destfile copy a file to another filename
cp [options] srcfile destdir copy a file to another directory
options:
-i confirm overwrites-R recursively copy a directory and its
contents, copies symbolic links
example:
type: cp -R proj1 proj2 copy the directory proj1/ and name itproj2/
Note: Be careful when overwriting files!
Copying
Tip:If you have large files tomove from the scratchdirectories,copy (cp) themrather than moving (mv)them.
www.msi.umn.edu
rm [options] filename remove a file
options:
-f remove without prompting-i confirm removal-r recursively remove a directory and its contents
example:
type: rm -rf /scratch/user2/dir1 remove the directory /scratch2/user2/dir1and its contents
Note: Be careful when removing files!Especially with wild cards (*,?)!
Removing
www.msi.umn.edu
scp [options] host:file1 host:file2 secure copy; copy files betweenhosts on a network using the sshprotocol
options:-r recursive (used for directories)
example:type: scp pfast@wind:”output/*.out” . copy the output files from host wind
with username pfast to current directory
type: scp -r mail origin:mail_copy copy the mail directory to the originand rename the directory mail_copy
Note: You can use sftp if scp is not available.
Remote Copy
www.msi.umn.edu
scp [options] host:file1 host:file2 secure copy; copy files betweenhosts on a network using the sshprotocol
options:-r recursive (used for directories)
example:type: scp pfast@wind:”output/*.out” . copy the output files from host wind
with username pfast to current directory
type: scp -r mail origin:mail_copy copy the mail directory to the originand rename the directory mail_copy
Note: You can use sftp if scp is not available.
Remote Copy
Tip:If you have large files totransfer, use sftp insteadof scp.
www.msi.umn.educhmod [options] who ops permission filename change file permissions
use ls -l to view permissions
options
–R recursively change permissions
who can be any combination of:u (user) g (group)o (other/world) a (all or ugo)
ops adds or takes away permission, and can be:+ (add permission) – (remove permission)
permission can be any combination of:r (read) w (write) x (execute) X (check user first)
Changing Permissions
www.msi.umn.eduhistory display list of most recent commands
!string repeat command beginning with string
up/down arrows scroll up and down through mostrecent commands; csh and tcsh
left/right arrows use for editing current line
Control-E end of lineControl-A beginning of line
examples:
type: !ls repeat last ls command
Command Repetition
www.msi.umn.edu
TAB finish the current command, filename,or or directory if possible, or show theControl-D possible completers
(works with c and tc shells)
Command Completion
www.msi.umn.edu
tar [options] files collect or extract files or directoriesinto or from an archive
options:c create a new archivex extract files from archivef store files in archivev print function letter and name of files– store to standard output
example:type: tar xvf file.tar extract the files and or directories from the
archive, file.tar
Tarring
www.msi.umn.edu
compress file compress file and rename it file.Z(the program automaticallyrenames the file)
uncompress file.Z uncompress file.Z and rename it file
gzip file compress file and rename it file.gz(the program automaticallyrenames the file)
gunzip file.gz uncompress file.gz and rename itfile
Compression
www.msi.umn.edu
Process Control
Control-C cancel a foreground job
Control-Z stop (interrupt) a foreground job
jobs list of background jobs
bg run stopped job in the background
fg run stopped job in the foreground
& appended to the end of a command will placethat job in the background
example:type: blastp swissprot test.seq > test.out & run WU-BLAST job in the
background
www.msi.umn.edu
ps [options] display the status of the current processes andthe process id-number
options:-a all processes-u display processes owned by a particular user
jobs shows any jobs that are currently running inthe background or suspended
Process Control
www.msi.umn.edu
kill –9 id-number terminate a process owned by you; id-number (process id) can be found withthe ps command
or
kill %job-number terminate a process owned by you; job-number can be found with the jobscommand
Process Control
www.msi.umn.edu
du -sk file display total disk usage of file ordirectory in kilobytes
df display the current disk usage forall connected file systems
stat file view attributes of a file or directory
quota -v list your disk quota
Other Commands
www.msi.umn.edu
cat [options] files dumps the contents of files to thescreen
more [options] files a pager, that lets you view one pageat a time
less file similar to more, but more
Viewers
www.msi.umn.edu
Graphical:
nedit filename if not available, ask to have itinstalled
xemacs filename
Command-line:
pico filename full-screen text editor available onmost UNIX systems
vi filename full-screen text editor available onvirtually every UNIX system
Editors
www.msi.umn.edu
The Fun Stuff– redirect
> command output redirection (create new,overwrites existing file)
>> command output redirection (append)
< command input redirection (from file)
examples:
how to: combine individual sequence files into one file
type: cat seq1 seq2 > seq
how to: email a file to a collaborator
type: mail -s “results” [email protected] < new_blast_results
www.msi.umn.edu
The Fun Stuff– piping
command1 | command2 | … directs standard output of one commandinto standard input for the next command
examples:
type: ls -al | more look at the ls output one page at a time
type: qstat -a | grep user1 check the queue on the origin for anyprocesses being run by user1
www.msi.umn.edu
grep [options] string file find and print out lines containing the string ina file or multiple files
options:
-i ignore case
examples:
how to: extract sequence information from protein structure files
type: grep SEQRES pdbfile | cut -c20-70 > seqs
The Fun Stuff– pattern searching
www.msi.umn.edu.cshrc shell startup file
Some things to add to your .cshrc file:
alias alias-string com-string abbreviate a command string with analias string
examples:
type: alias la ls -a create a shortcut for ls -a
type: alias q qstat -u pfast check if you have any jobs running inthe queue (origin)
module load(unload) software load or unload the environment settingsfor a particular software
source /usr/local/gcg/gcgstartup runs the gcgstartup script to set thenecessary env. variables
The Fun Stuff– .cshrc file
www.msi.umn.edu
UNIX in a Nutshell by O’Reilly
Developing Bioinformatics Computer Skills by O’Reilly
http://www.iats.missouri.edu/servlets/knowledgebase/article/422
http://www.geek-girl.com/unix.html
http://riceinfo.rice.edu/Computer/Documents/Unix/unix1.01.pdf
Resources