Compiling and Building Programs - National Institute for ... · Compiling and Building Programs Where to use it? • Install Cygwin: a collection of tools which provide a Linux look

Compiling and Building Programs

Daniel Lucio


Overview• Where to use it?

• Command syntax

• How it works?

• Compiler options

• Which compiler to use?

• What options to know?

• Why file extension matter?

• How to build software?

• More information


Where to use it?

• Login to a Unix system like ’kraken’ or any other NICS/UT/XSEDE resource.

• Download and boot from a Linux LiveCD either from a CD/DVD or USB drive.

• http://www.puppylinux.com/

• http://www.knopper.net/knoppix/index-en.html

• http://www.ubuntu.com/

http://www.knopper.net/knoppix/index-en.html

http://www.ubuntu.com/


Where to use it?

• Install Cygwin: a collection of tools which provide a Linux look and feel environment for Windows.

• http://cygwin.com/index.html

• https://newton.utk.edu/bin/view/Main/Workshop0InstallingCygwin

• Online terminal emulator

• http://bellard.org/jslinux/

• http://simpleshell.com/

http://cygwin.com/index.html

http://bellard.org/jslinux/

http://simpleshell.com/


Command syntaxSYNOPSIS gcc [-c|-S|-E] [-std=standard] [-g] [-pg] [-Olevel] [-Wwarn...] [-pedantic] [-Idir...] [-Ldir...] [-Dmacro[=defn]...] [-Umacro] [-foption...] [-mmachine-option...] [-o outfile] [@file] infile...

Example:gcc -o hello hello.c


How it works?

Pre-

Proc

esso

r

Com

pile

r

Link

er

Source code

Header files

Libraries

Exec

utab

le

or

Prog

ram

mod files

Library

ldgcc

cpp


Compiler options-A- -A<name>[<value(s)>] -[no-]alias-args -[no]align -ansi -[no-]ansi-alias -auto-ilp32 -ax{K|W|N|B|P} -B<dir> -c -C -[no-]c99 -[no-]complex-limited-range -create-pch <file> -cxxlib-<mode> -D<name>[=<value>] -[no-]debug [keyword] -dM -dryrun -dynamic-linker<filename> -E -EP -export -export-dir <dir> -F -falias -fargument-alias -fargument-noalias -fargument-noalias-global -fast -fcode-asm -f[no-]common -fdata-sections -f[no-]exceptions -ffnalias -ffunction-sections -finline-functions -f[no-]math-errno -fminshared -fno-alias -fno-builtin -fno-builtin-<func> -fno-fnalias -fno-gnu-keywords

-fno-implicit-inline-templates -fno-implicit-templates -f[no-]omit-frame-pointer -fno-operator-names -fnsplit[-] -fp -fpack-struct -fpermissive -f[no-]pic -f[no-]PIC -fp-model <name> -[no-]fp-port -fpstkchk -fr32 -f[no-]rtti -freg-struct-return -fshort-enums -fsource-asm -fstack-security-check -fsyntax-only -ftemplate-depth-<n> -ftls-model=<model> -ftrapuv -[no-]ftz -funroll-loops -funsigned-bitfields -funsigned-char -f[no]verbose-asm -fvisibility=[extern|default|protected|hidden|internal] -fvisibility-default=<file> -fvisibility-extern=<file> -fvisibility-hidden=<file> -fvisibility-internal=<file> -fvisibility-protected=<file> -g -g0 -gcc-name=<name> -gcc-version=<nnn> -[no-]global-hoist -H -help -I<dir> -idirafter<dir> -i-dynamic

-imacros <file> -inline-debug-info -ip -[no-]IPF-fltacc -IPF-flt-eval-method0 -[no-]IPF-fma -[no-]IPF-fp-relaxed -IPF-fp-speculation{fast|safe|strict|off} -ip-no-inlining -ip-no-pinlining -ipo[value] -ipo-c -[no-]ipo-obj -ipo-S -ipo-separate -iprefix <prefix> -i-static -isystem<dir> -ivdep-parallel -iwithprefix <dir> -iwithprefixbefore <dir> -Kc++ -kernel -L<dir> -long_double -M -malign-double -map-opts -march={pentiumpro|pentiumii|pentiumiii|pentium4} -mcpu={pentium|pentiumpro|pentium4|itanium|itanium2} -MD -MF<file> -mfixed-range=f12-f15,f32-f127 -MG -m[no-]ieee-fp -MM -MMD -mno-relax -mno-serialize-volatile -mp

-MP -mp1 -MQ<target> -mrelax -mserialize-volatile -msse -msse2 -msse3 -MT<target> -[no-]multibyte-chars -nobss-init -no-cpprt -nodefaultlibs -no-gcc -nostartfiles -nostdinc -nostdlib -mtune=<cpu> -o<file> -O0 -O1 -O2 -O3 -Ob<n> -openmp -openmp-profile -openmp-report{0|1|2} -openmp-stubs -opt-report -opt-report-file<file> -opt-report-help -opt-report-level[{min|med|max}] -opt-report-phase{ipo|hlo|ilo|ecg|omp|all} -opt-report-routine<name> -Os -p -P -parallel -par-report{0|1|2|3} -par-threshold[<n>] -pc{32|64|80} -pch -pch-dir <dir>

-[no-]prec-div -[no-]prec-sqrt -[no-]prefetch -print-multi-lib -prof-dir <dir> -prof-file <file> -prof-gen[x] -prof-gen-sampling -prof-use -Qinstall <dir> -Qlocation,<tool>,<path> -Qoption,<tool>,<optlist> -qp -rcd -reserve-kernel-regs -[no]restrict -S -shared -shared-libcxa -[no-]sox -ssp -static -static-libcxa -std=c99 -strict-ansi -T <file> -tcheck -[no]traceback -u <symbol> -U<name> -unroll<n> -unroll0 -use-asm -use-msasm -use-pch <file> -V -v -vec-report[{0|1|2|3|4|5}] --version -w -w<n> -Wa<o1>[,<o2>,...] -W[no-]abi -Wall -Wbrief -Wcheck

-W[no-]comment[s] -Wcontext-limit=<n> -wd<L1>[,<L2>,...] -W[no-]deprecated -we<L1>[,<L2>,...] -Werror -Winline -Wl,<o1>[,<o2>,...] -W[no-]main -W[no-]missing-prototypes -wn<n> -Wp64 -Wp,<o1>[,<o2>,...] -W[no-]pointer-arith -wr<L1>[,<L2>,...] -W[no-]return-type -W[no-]shadow -W[no-]uninitialized -W[no-]unknown-pragmas -W[no-]unused-function -ww<L1>[,<L2>,...] -X -x <type> -x{K|W|N|B|P} -Xlinker <val> -Zp[n]

Intel C compiler: icc


Compiler options

Performance Options -ax{K|W|N|B|P} -falias -fast -fdata-sections -f[no-]exceptions -ffnalias -ffunction-sections -finline-functions -fno-alias -fno-builtin-<func> -fno-fnalias -f[no-]omit-frame-pointer -fp -[no-]fp-port -fpstkchk -march={pentiumpro|pentiumii|pentiumiii|pentium4} -mcpu={pentium|pentiumpro|pen- tium4|itanium|itanium2} -m[no-]ieee-fp -mp -mp1 -msse -msse2 -msse3 -mtune=<cpu> -O0 -O1 -O2 -O3 -Ob<n> -pc{32|64|80} -[no-]prec-div -[no-]prec-sqrt -rcd -x{K|W|N|B|P}



Compiler options Advanced Performance Options -[no-]alias-args -[no-]ansi-alias -auto-ilp32 -[no-]complex-limited-range -fargument-alias -fargument-noalias -fargument-noalias-global -fnsplit[-] -fp-model <name> -ftls-model=<model> -[no-]ftz -funroll-loops -ip -[no-]IPF-fltacc -IPF-flt-eval-method0 -[no-]IPF-fma -[no-]IPF-fp-relaxed -IPF-fp-speculation{fast|safe|strict|off} -ip-no-inlining -ip-no-pinlining -ipo[value] -ipo-c -[no-]ipo-obj -ipo-S -ipo-separate -ivdep-parallel -mno-serialize-volatile -mserialize-volatile -openmp -openmp-profile -openmp-report{0|1|2} -openmp-stubs -opt-report -opt-report-file<file> -opt-report-help -opt-report-level[{min|med|max}] -opt-report-phase{ipo|hlo|ilo|ecg|omp|all} -opt-report-routine<name> -p -parallel -par-report{0|1|2|3} -par-threshold[<n>] -[no-]prefetch -prof-dir <dir> -prof-file<file> -prof-gen[x] -prof-gen-sampling -prof-use -qp -tcheck -unroll<n> -unroll0 -vec-report[{0|1|2|3|4|5}]



Compiler options Output, Debug, and PCH Options -c -create-pch <file> -[no-]debug[keyword] -fcode-asm -fsource-asm -ftrapuv -f[no]verbose-asm -g -g0 -inline-debug-info -map-opts -o<file> -pch -pch-dir<dir> -print-multi-lib -S -use-asm -use-msasm -use-pch <file> ! Preprocessor Options -A- -A<name>[<value(s)>] -C -D<name>[=<value>] -dM -E -EP -H -I<dir> -idirafter<dir> -imacros<file> -iprefix<prefix> -isystem<dir> -iwithprefix<dir> -iwithprefixbefore<dir> -M -MD -MF<file> -MG -MM -MMD -MP -MQ<target> -MT<target> -no-gcc -nostdinc -P -ssp -U<name> -X



Compiler options

Component Control Options -Qinstall<dir> -Qlocation,<tool>,<path> -Qop- tion,<tool>,<optlist> ! Language Options -[no]align -ansi -[no-]c99 -export -export-dir <dir> -fno-gnu-keywords -fno-implicit-inline-templates -fno-implicit-templates -fno-operator-names -fpermissive -f[no-]rtti -fshort-enums -fsyntax-only -ftemplate-depth-<n> -funsigned-bitfields -funsigned-char -Kc++ -malign-double -[no]restrict -std=c99 -strict-ansi -Zp[n]



Compiler options

! Compiler Diagnostics Options -[no]traceback -w -w<n> -W[no-]abi -Wall -Wbrief -Wcheck -W[no-]comment[s] -Wcontext-limit=<n> -wd<L1>[,<L2>,...] -W[no-]deprecated -we<L1>[,<L2>,...] -Werror -Winline -W[no-]main -W[no-]missing-prototypes -wn<n> -Wp64 -W[no-]pointer-arith -wr<L1>[,<L2>,...] -W[no-]return-type -W[no-]shadow -W[no-]uninitialized -W[no-]unknown-pragmas -W[no-]unused-function -ww<L1>[,<L2>,...]



Compiler options Miscellaneous Options -B<dir> -dryrun -f[no-]common -f[no-]math-errno -fminshared -fno-builtin -fno-builtin-<func> -fpack-struct -f[no-]pic -f[no-]PIC -fr32 -freg-struct-return -fstack-security-check -fvisibility=[extern|default|protected|hidden|internal] -fvisibility-default=<file> -fvisibility-extern=<file> -fvisibility-hidden=<file> -fvisibility-internal=<file> -fvisibility-protected=<file> -gcc-name=<name> -gcc-version=<nnn> -[no-]global-hoist -help -kernel -long_double -lmpi -mfixed-range=f12-f15,f32-f127 -[no-]multibyte-chars -nobss-init -reserve-kernel-regs -[no-]sox -V -v --version -x <type>



Compiler options

! Linking or Linker Options -cxxlib-<mode> -dynamic-linker<filename> -i-dynamic -i-static -L<dir> -mno-relax -mrelax -no-cpprt -nodefaultlibs -nostartfiles -nostdlib -shared -shared-libcxa -static -static-libcxa -T <file> -u <symbol> -Wa<o1>[,<o2>,...] -Wl,<o1>[,<o2>,...] -Wp,<o1>[,<o2>,...] -Xlinker<val>



Which compiler to use?• What do you want to compile?

• Serial

• MPI

• OpenMP

• GPU

• MIC


Which compiler to use?• What is available on your system?

• GNU (gcc, gfortran,), PGI, Intel, CCE, Pathscale,…

lucio@darter1:~> module avail PrgEnv !----------------- /opt/cray/modulefiles ---------------- PrgEnv-cray/5.0.15 PrgEnv-gnu/5.1.29(default) PrgEnv-cray/5.1.18 PrgEnv-intel/5.0.15 PrgEnv-cray/5.1.29(default) PrgEnv-intel/5.1.18 PrgEnv-gnu/5.0.15 PrgEnv-intel/5.1.29(default) PrgEnv-gnu/5.1.18


Which compiler to use?• PGI - Very good Fortran and C, pretty good C++

• Good vectorization

• Good functional correctness with optimization enabled

• Good manual and automatic prefetch capabilities

• Very interested in the Linux HPC market although that is not their focus

• Excellent working relationship with makers like Cray, good bug fix responsiveness


Which compiler to use?• Pathscale - Good Fortran, C, possibly good C++

• Outstanding scalar optimization for loops that do not vectorize

• Fortran front end uses and older version of the CCE Fortran front end

• OpenMP uses a non-pthreads approach

• Scalar benefits will not get as much mileage with longer vectors


Which compiler to use?• Intel - Good Fortran, excellent C and C++,

• Automatic Vectorization capabilities are modest compared to PGI and CCE

• Use of inline assembly is encouraged

• Focus is more on best speed for scalar, non-scaling apps

• Tuned for Intel architectures, but, actually works well for some applications on AMD.


Which compiler to use?• GNU - so-so Fortran, outstanding C and C++,

• The best for compatibility

• Scalar optimizer was recently rewritten and is very good

• Vectorization capabilities focus mostly on inline assembly

• Known for incompatibilities between each other versions.


Which compiler to use?• CCE - Outstanding Fortran, very good C and okay C++

• Very good vectorization

• Very good Fortran language support; only real choice for Coarrays Fortran

• C support is quite good with builtin support for UPC

• Very good scalar optimizations and automatic parallelization

• Clean implementation of OpenMP 3.0, with tasks

• Available only for Linux-based Cray hardware


What options to know?

• PGI

• -‐fast -‐Mipa=fast(,safe)

• if precision can be flexible try -‐Mfprelaxed

• compiler feedback: -‐Minfo=all -‐Mneginfo

• man pgf90; man pgcc; man pgCC;

• or pgf90 -‐help [-‐fast]



• CCE

• <none, optimizations on by default>

• Compiler feedback: -‐rm(Fortran) -‐hlist=m (C)

• To disable OpenMP: -‐xomp or -‐Othread0

• man crayftn; man craycc; man crayCC



• Pathscale

• -‐Ofast Can yield looser precision

• Compiler feedback: -‐LNO:simd_verbose=ON

• man eko (Every Known Optimization)


What options to know?• GNU

• -‐O2 or -‐O3

• Compiler feedback: Non existant?

• man gfortran; man gcc; man g++

• Intel

• -‐fast

• Compiler feedback: -‐LNO:simd_verbose=ON

• man eko (Every Known Optimization)


Why file extensions matter? .f fixed-format Fortran source; compile .F fixed-format Fortran source; preprocess, compile .f90 free-format Fortran source; compile .F90 free-format Fortran source; preprocess, compile .f95 free-format Fortran source; compile .F95 free-format Fortran source; preprocess, compile .f03 free-format Fortran source; compile .F03 free-format Fortran source; preprocess, compile .for fixed-format Fortran source; compile .FOR fixed-format Fortran source; preprocess, compile .ftn fixed-format Fortran source; compile .FTN fixed-format Fortran source; preprocess, compile .fpp fixed-format Fortran source; preprocess, compile .FPP fixed-format Fortran source; preprocess, compile .cuf free-format CUDA Fortran source; compile .CUF free-format CUDA Fortran source; preprocess, compile .s assembler source; assemble .S assembler source; preprocess, assemble .o object file; passed to linker .a library archive file; passed to linker

PGI Fortran compiler

If coinstalled with pgcc, C file suffixes are also recognized and compiled with the pgcc compiler

Suffixes of source file names indicate the type of processing to be done:


How to build software?

wget "http://p3dfft.googlecode.com/files/p3dfft.2.5.1.tar.gz" tar -‐zxf p3dfft.2.5.1.tar.gz cd cd p3dfft.2.5.1/ !module swap pgi pgi/12.9.0 module load fftw !./configure -‐-‐prefix=$HOME/myp3dfft -‐-‐enable-‐fftw -‐-‐with-‐fftw="$FFTW_DIR/.." -‐-‐enable-‐useeven -‐-‐enable-‐stride1 -‐-‐enable-‐pgi FC=ftn FCFLAGS="-‐fastsse -‐tp istanbul-‐64 -‐Mpreprocess" CC=cc CFLAGS="-‐fastsse -‐tp istanbul-‐64" make make install

Installing p3dfft on Kraken


More information

http://www.nics.tennessee.edu/computing-resources/kraken/compiling

https://www.tacc.utexas.edu/user-services/user-guides/stampede-user-guide#appdev

http://www.nics.tennessee.edu/computing-resources/nautilus/compiling

http://www.sdsc.edu/us/resources/trestles/trestles_compile.html

http://www.nics.tennessee.edu/computing-resources/kraken/compiling

https://www.tacc.utexas.edu/user-services/user-guides/stampede-user-guide#appdev

http://www.sdsc.edu/us/resources/trestles/trestles_compile.html

Documents

Compiling and Building Programs - National Institute for ... · Compiling and Building Programs Where to use it? • Install Cygwin: a collection of tools which provide a Linux look