Continuous integration a walk-through...Continuous integration a walk-through Author Dr Alin Marin Elena Subject dlpoly, continuous integration, gitlab Created Date 20171109102900Z

Continuous integration a walk-through

Dr Alin Marin Elena

Daresbury, 2017


1 Theory

2 Practice

Elena et al. Daresbury, November 2017


Theory

1 Theory

2 Practice



Theory

Continuous Integration

The theory

Maintain a single sourcerepository

Automate the build

Make your build self-testing

Every commit should build onan integration machine

Keep the build fast

Test in a clone of the productionenvironment

Everyone can see what’shappening

Automate deployment

The practice

Developers check out code intotheir private workspaces.

When done, commit thechanges to the repository.

The CI bot monitors therepository and runs unit andintegration tests

If the build or tests fail, the CIserver alerts the team.

The team fix the issue at theearliest opportunity.

Continue to continuallyintegrate and test throughoutthe project.



Practice

1 Theory

2 Practice



Practice

Workflow

master user/masterforksync user/branchbranch

work

commit

tests pass?

cibo

t

No

fix

review&tests pass?Yes

merge request

Yes

No

fix



Practice

Workflow


work

commit

tests pass?

cibo

t

No

fix


merge request

Yes

No

fix



Practice

Workflow


work

commit

tests pass?

cibo

t

No

fix


merge request

Yes

No

fix



Practice

Workflow


work

commit

tests pass?

cibo

t

No

fix


merge request

Yes

No

fix



Practice

Workflow


work

commit

tests pass?

cibo

t

No

fix


merge request

Yes

No

fix



Practice

Workflow


work

commit

tests pass?

cibo

t

No

fix


merge request

Yes

No

fix



Practice

Workflow


work

commit

tests pass?

cibo

t

No

fix


merge request

Yes

No

fix



Practice

Workflow


work

commit

tests pass?

cibo

t

No

fix


merge request

Yes

No

fix



Practice

Workflow


work

commit

tests pass?

cibo

t

No

fix


merge request

Yes

No

fix



Practice

Workflow


work

commit

tests pass?

cibo

t

No

fix


merge request

Yes

No

fix



Practice

Workflow


work

commit

tests pass?

cibo

t

No

fix


merge request

Yes

No

fix



Practice

Workflow cont.

Tools

git, fortran, MPI

GitLab

cmake, python

doxygen, LATEX

The Rules

3 Check in frequently

7 Don’t check in broken code

7 Don’t check in untested code

7 Don’t check in when the build is broken

7 Don’t go home after checking in until the system builds



Practice

Testing

custom testing framework

in house python3 and xml

automatized and integrated inbuild/Gitlab

test at different optimisationlevels

[08:42:32 drFaustroll@circassia:../dl-poly/build-gcc]: cmake ../ -DBUILD_TESTING=On[08:43:12 drFaustroll@circassia:../dl-poly/build-gcc]: make testRunning tests.../usr/bin/ctest --force-new-ctest-processTest project /home/drFaustroll/playground/dl-poly/build-gccStart 1: TEST011/25 Test #1: TEST01 ........................... Passed 2.77 sec...

Start 25: TEST2525/25 Test #25: TEST25 ........................... Passed 3.08 sec

100% tests passed, 0 tests failed out of 25

Total Test time (real) = 111.23 sec



Practice

Code Compliance

enforce Fortran standard and a basic set of runtime checks.via compilers flags:

gcc: -g -O0 -std=f2008 -pedantic -fbacktrace -fcheck=all -finit-integer=2147483648

-finit-real=nan -finit-logical=true -finit-character=42

-ffpe-trap=invalid,zero,overflow,denormal

intel: -g -O0 -stand f08 -traceback -C -fp-stack-check -ftrapuv -qopt-report=5

external tools, extra code corectness checks, FORCHECK:cmake ../ -DWITH_FORCHECK=ON && make forcheck

2x[711 I] declared RECURSIVE but not recursively referenced2x[786 I] type kind inconsistent with specification1x[811 E] invalid argument value1x[812 I] derived-type component not used3x[813 I] derived-type component not referenced

52x[841 I] module object not used outside the module48x[859 I] variable, used as actual argument, unreferenced122x[877 I] INTENT(INOUT) dummy argument is defined before referenced13x[887 I] array unreferenced1x[888 I] array not used

number of error messages: 54number of informative messages: 2122



Practice

Code Coverage

How much tested is your code?

gcov - gcc only

text mode only but html output possible

rich statistics on which paths the code takes

compiler flags: -g -O0 –coverage -fprofile-arcs -ftest-coverage

cmake ../ -DWITH_COVERAGE=On -DBUILD_TESTING=On && make coverageSummary coverage rate:lines......: 25.4% (17753 of 69773 lines)functions..: 49.1% (280 of 570 functions)branches...: no data found

Reading tracefile coverage.info.cleaned|Lines |Functions|

Filename |Rate Num|Rate Num|======================================================================......source/z_density_module.f90 | 100% 6| 100% 1|source/zero_k_optimise.f90 | 0.0% 153| 0.0% 1|======================================================================

Total:|25.4% 69773|49.1% 570|[100%] Built target coverage



Practice

Policies

coding standard ftp://ftp.dl.ac.uk/ccp5/DL_POLY/DL_POLY_4.0/DOCUMENTS/Coding_Style.pdfcode of contribution

API documentation & manual ( use -DDOCS=On )

examples and tests


ftp://ftp.dl.ac.uk/ccp5/DL_POLY/DL_POLY_4.0/DOCUMENTS/Coding_Style.pdf

ftp://ftp.dl.ac.uk/ccp5/DL_POLY/DL_POLY_4.0/DOCUMENTS/Coding_Style.pdf

Documents

Continuous integration a walk-through...Continuous integration a walk-through Author Dr Alin Marin Elena Subject dlpoly, continuous integration, gitlab Created Date 20171109102900Z