| This work was funded by the RCUK Energy Programme[Grant number EP/P012450/1]

WPEC Subgroup 43 - GNDSMoving towards API driven codes – A FISPACT-II perspectiveThomas Stainer, Mark Gilbert, Lee Packer, Tim Eade, Greg BaileyJune 2019

|

• FISPACT-II API developments

• FISPACT-II as a nuclear data consumer

• Lessons learned

• GNDS design ideas + thoughts

• Summary

Overview

Thomas Stainer - WPEC Subgroup 43 - June 20192

|

FISPACT-II

• Generic inventory, transmutation code

• Consumes vast number of nuclear data libraries: TENDL 17, 15, 14, JEFF 3.3, ENDF/B-VII,…

• Supports ENDF-6 and EAF formats• We are heavy consumer of

nuclear data• Estimate at least 50% of users

issues/errors relate to nuclear data processing

• Large amount of development time wasted due to poor, inconsistent, missing nuclear data• Only caught later upstream

Thomas Stainer - WPEC Subgroup 43 - June 20193

|

• February 2019 – API version of FISPACT-II developed (internal only)• Significant refactor:

• Removed input/output file reading/writing• Purely API driven• Decoupled nuclear data from calculations• Nuclear data abstracted through use of APIs• New language bindings in Fortran, C, C++, and Python

• Large validation effort conducted • Automated testing in place – 49 test suites consisting of over 470 system tests + 403

unit tests• Regression analysis • FNS decay heat validation

• Ready to consume nuclear data in a better way:• Database connection, REST APIs, • Still maintains ENDF file support.

• Still not abstract enough, ENDF hangovers:• Requires knowledge of MT numbers• Internal data structures not optimized for common queries (linked lists overused)

FISPACT-II API

Thomas Stainer - WPEC Subgroup 43 - June 20194

|

Architecture

FispactProcess

FispactInputData

FispactOutputData FispactNuclearData

FispactNuclearDataReader

Monitor

Inputs

Computation Outputs

Error handling

File

processing

ENDF files

Thomas Stainer - WPEC Subgroup 43 - June 20195

Decoupled concrete ENDF files from code – Nuclear data accessed via getters/setters

Need consistent error handling for all language bindings

|

Architecture

Thomas Stainer - WPEC Subgroup 43 - June 20196

Pros:• Decoupled from ENDF files. • Nuclear data can be stored in any fashion.

Cons:• Need to write new “reader” for each concrete data format

FispactProcess

FispactInputData

FispactOutputDataFispactNuclearData

FispactNuclearDataReader

Monitor

Inputs

Computation Outputs

Error handling

File

processing

ENDF files

XMLReader

HDF5Reader

Concrete files

Database DatabaseConnection

|

FISPACT-II uses group wise data, and requires NJOY and PREPRO to produce libraries for end use.

• Natural elemental abundances and atomic weights (mainly NIST)• Decay data (MF8)

• Half lives • Decay modes + probabilities + secondaries• Decay spectra (discrete)• Spontaneous fission yields

• Cross section data (MF3, MF10)• Group wise data• Variances + covariances

• Resonance parameters (MF2)• Gamma attenuation data (legacy)• Fission yields• Probability tables (Self Shielding) – use CALENDF

FISPACT-II nuclear data

Thomas Stainer - WPEC Subgroup 43 - June 20197

ENDF-6

NIST

Legacy

|

• Large nuclear data libraries e.g. TENDL 2017 take a long time to read/process in comparison to inventory calculation • Typical ~80 seconds to read all

necessary data for 3850 nuclides• ~1-2 seconds to solve Bateman

equation• For large scale calculations, very

slow• API approach using in-memory

data• Much quicker – no processing• No serialization

Processing and Performance

Thomas Stainer - WPEC Subgroup 43 - June 20198

No large output files to sift through!

|

Nuclear data interrogation using F-II

Thomas Stainer - WPEC Subgroup 43 - June 20199

• Internal nuclear data structures are almost flat• Thin hierarchy – 3 nested structures at most• Problem is refactored Fortran still using Linked Lists in most places• Some effort to replace with more optimal data structures:

• hash tables for quick lookup of ZAIs to index• Contiguous data structures for iteration

GetReactionAvailableChannels(zai, error)

GetReactionXS(zai, mt, error)

|

Layers of abstraction

Thomas Stainer - WPEC Subgroup 43 - June 201910

getxs(922380, MT=102)

No accuracy issues due to file precision

Need to interrogate and visualize nuclear data easily

Database(PostGres)

REST-API

Data access layer

FISPACT-II API

Plotting library

Abst

ract

ion

|

• FISPACT-II is not a big code but it was a large effort to refactor and validate• Hard to motivate community to move to alternative methods and formats• Talk with users – let them define what APIs should look like• Large validation effort required, automation helps• Backwards compatibility is a must• Fixing bugs can look like introducing bugs in new/refactored code• Whilst providing bindings for many languages keeps clients happy, a lot of

maintenance for developers• Good documentation is essential – 700+ page reports and manuals are not ideal

Lessons learned

Thomas Stainer - WPEC Subgroup 43 - June 201911

Doxygen

FNS Validation

|

Our current software stack• NJOY, PREPRO, Talys, MCNP, FISPACT-II, …• We cannot have codes with file based inputs/outputs anymore

• Must avoid things like:“broadr -21 -22 -23”

• Our typical software stack for processing nuclear data is:• Single monolithic Fortran programs with input files containing magic numbers• Schema less input and output files• No automated testing in placeÞ Difficult to interface together, error prone, precision issues,…

• Version control – for codes and data => git is not the answer to everything• We need to get full stack improved

• NJOY21 and ENDFtk - but what about the rest of the pack?

Back to GNDS

Thomas Stainer - WPEC Subgroup 43 - June 201912

|

• ENDF-6 format is not the only problem• Lack of automated testing is a worry for our nuclear data

• All codes (processing, transport, inventory, …) need ability to mock and test easily

• Regression tests with every code change• Serialization tests – Read ENDF -> Write HDF5 -> Read HDF5 ->

Write ENDF• Minimal human verification of functionality – automate as much as

possible• Focus on interface

• Usability and maintenance should trump performance• Do not neglect performance, but API must work for users (both

evaluators + developers)• Must be abstract enough – decoupled from concrete file formats• Every API have an equal and opposite API – get/set, read/write• Who should drive this development – evaluators, transport codes?

• C API should be base line• Well supported• Longevity• C++ is nice but C is better for interoperability

• bind to Fortran, C++, Python, Java, …

Design principles

Thomas Stainer - WPEC Subgroup 43 - June 201913

Simple, high level interfaces

Testable, modular code

Interoperability

|

• Data provenience issues• Versioning – how can we reliably version nuclear data, and perform regression on

data only• Git cannot be used for everything!

• Full provenience tree - Who made changes that broke the tests? Was it a Friday evening when the last changes where committed? Etc..

• Validation requires well tested and validated ENDF reader – is ENDFtk the way?• How do we incorporate EAF and ACE formats?• ENDSF, EXFOR data – how do we incorporate this as well?• How can we automate as much as possible?

• CI is a must• Monolithic unmaintainable scripts should be abandoned

• Whilst performance is key, we should think from a user point of view.• It should be trivial to do simple queries for users without any notion of a “file”

• Get all nuclides with half life > 1e8 and have one stable child….• Get the (n,g) cross section for Ag108 for JEFF3.3..

• Should we engage with HEP?• ROOT - work ongoing to develop new database TNudy (Korea)

GNDS final thoughts

Thomas Stainer - WPEC Subgroup 43 - June 201914

|

• FISPACT-II is ready for GNDS• We can help with validation effort• API approach allows quicker analysis, in-memory nuclear data and no file

serialization• Adoption is starting to increase UKAEA, many new users requesting API version

• Next steps:• Development of REST APIs for FISPACT-II in development.

• Users will not require local version of nuclear data• Nuclear data as a service

fetch('https://fispact.com/api/v0.1/getzais&token=adsj67ads8d8').then(response => response.json()).then(json => console.log(json))

• Looking into alternative database options for FISPACT-II nuclear data• SQL, NON-SQL, in-memory(reddis), graph• Not clear what is the best representation for nuclear data

Summary

Thomas Stainer - WPEC Subgroup 43 - June 201915

|

Backup

Thomas Stainer - WPEC Subgroup 43 - June 201916

|

Code coverage

Thomas Stainer - WPEC Subgroup 43 - June 201917

• Unit + Integration + System tests• Covering Fortran, C, C++ and Python layers• 45 system tests, 470 FISPACT-II full simulations• 368 unit + integration tests• Over 100,000 assert statements• Collect code coverage from tests ~80% line coverage

Test type Line code coverage (%)

Function code coverage (%)

Implementation unit + integration tests only 8.6 16.1

API unit + integration tests only 16.4 31.1

Full unit + integration test suite 20.2 38.5

System tests only 68.3 66.3

Full test suite (unit + integration + system) 74.6 80.6

Full test suite (unit + integration + system) excluding unused code i.e. LSODES single precision solver

79.7 81.5

25594 lines2590 functions

|

• Inventories (nuclide quantities) can differ• Some nuclides at timesteps have different number of atoms < 0.01%• Very low impact on total heats and activities• Most differences around 1e-15

FISPACT-II API validation

Thomas Stainer - WPEC Subgroup 43 - June 201918

|

• Need rigorous validation system that can be fully automated and scheduled• People need to be able to trust the tools we are using in order to help adoption• Floating point issues can cause headaches, and difficult to trace• Write, Read, Write chains + compare to highlight serialization issues

Validation process

Thomas Stainer - WPEC Subgroup 43 - June 201919

ExistingENDF-6

fileIn-memory Xml file In-memory

New ENDF-6

file

ReadENDF

WriteXML

ReadXML

Write ENDF

Compare

|

Nuclear data interrogation using F-II

Thomas Stainer - WPEC Subgroup 43 - June 201920

Non trivial exercise in traditional file approach using F-II

|

• Command Line Tool only• Single executable program• Takes input file and produces output files• Use of keywords in input file to setup simulation• Human readable text output file

FISPACT-II traditional

Thomas Stainer - WPEC Subgroup 43 - June 201921

| Thomas Stainer - WPEC Subgroup 43 - June 201922

Problems with this approach…

Lots of files…Slow…

Reading nuclear data can take minutes

Reading and writing files is not only slow but introduces bugs*.i, *.out, *.json, *.tab1,

*.tab2, fluxes, files…..

Bugs…

| Thomas Stainer - WPEC Subgroup 43 - June 201923

More problems with this approach…

Cannot interface easily (at all?) with other codes

Must write input files from <insert transport code here>, float to string issues

Maybe we could make a GUI to read and write input and output files, but doesn’t scale, especially for web applications

|

What is an API?

In computer programming, an API can be considered as the suite of functions, commands, subroutines, protocols, objects and/or tools for building software and interacting with an application. Generally, it is the set of clearly defined methods, or interfaces, of communication among various components.

FISPACT-II API

Thomas Stainer - WPEC Subgroup 43 - June 201924

No more files! (well almost, some are still needed)

|

• Science must be reproducible• Easy way to define configuration and parameters of codes• Nuclear data is still text files

We still need files…

Thomas Stainer - WPEC Subgroup 43 - June 201925

but we can be smarter..• Less files• Support multiple formats• Use of databases instead of text files

|

Program 1

input files output files

Program 2

input files output files

Program 3

input files output files

Parser/Writer Parser/Writer

We still need files…

Thomas Stainer - WPEC Subgroup 43 - June 201926

Program 1

input files

Program 2 Program 3

output files

API API

Before

After

|

Numerical stability for nuclear instability

Thomas Stainer - WPEC Subgroup 43 - June 201927

212,640 FISPACT-II calculations

Took ~7 hours on local machine

Using exe instead would have taken ~9-10 days

(Actual – Expected)/Expected

100%

|

Numerical stability

Thomas Stainer - WPEC Subgroup 43 - June 201928

10 steps per half-life

|

Numerical stability

Thomas Stainer - WPEC Subgroup 43 - June 201929

100 steps per half-life

|

Error handling improvements

Thomas Stainer - WPEC Subgroup 43 - June 201930

Faster than writing to log file