News and IssuesTom JunkDUNE Software Management MeetingDecember 20, 2017

Upcoming Dates• Holidays 2017- Would like to use the time to run MCC10 – 6M FD neutrino events

requested by deep learning teams

• January 29 to Feb. 1, 2018: Collaboration meeting at CERN- Expect two S&C parallel sessions, possibly shared with related

groups

- Lunchtime tutorials

• Feb. 2,3 ProtoDUNE Data Readiness Review at CERN- Addresses computing resources, reco algorithms, OM, DQM,

calibration, offline operations

12/20/17 Tom Junk | News and Issues2

Pandora and 35-ton• Redmine issue:

https://cdcvs.fnal.gov/redmine/issues/18584• Easy problem – fcl files needed updating- particlestitcher functionality now in standard Pandora module, so no

need to include it as a separate step.

- Small amount of rewiring module labels so data still flows.- John Marshall provided an example and it was easy to follow (once

we knew what to do).

• More difficult problem: larpandoracontent/LArPlugins/LArRotationalTransformationPlugin.ccemits this message when running with 35t and its caller throws an unknown exception:LArRotationalTransformationPlugin::Initialize - Plugin does not support provided LArTPC configurations

12/20/17 Tom Junk | News and Issues3

Pandora and 35-ton cont'd• Pandora team discussed this. The current plugins assume the

same wire angles in different drift volumes, and 35t breaks this assumption. Two options presented in the ticket:

1. We have tested and pushed a branch "feature/LArRotationalTransformationPlugin" to Redmine that will allow the DUNE35t reconstruction to proceed, with the understanding that the transformations will only be approximations in some drift volumes. The diff should read cleanly, and will show that the maximum allowed difference between like wire (u and v) angles between volumes is now configurable and carries a larger default value.

2. We decide that Pandora should no longer be included in the DUNE35t reconstruction, as the Pandora team does not have sufficient person power to maintain it. It is not an experiment that Pandora has targeted directly, but has been able to use developments put together for e.g. MicroBooNE. Our routine testing considers tens of thousands of MicroBooNE events nightly, about ten thousand ProtoDUNE events upon demand and a few hundred DUNEFD events upon demand. It does not include DUNE35t (more detail about our tests, with Travis CI, Coverity, codecov and Valgrind are available upon request).

Please let us know which you prefer.

12/20/17 Tom Junk | News and Issues4

Discussion of Options• If we were still considering 45±εo for the FD wire angles, we

would dig our heels in on this and insist that a plugin be created.• But as it is, 35-ton data analysis is almost complete, and• The publications aren't using Pandora reco• We could keep Pandora for the long drift volume, and live with

approximate geometry for the short drift volume.• It would make analyses confusing if the short and long drift

volumes were treated differently in this way however.

12/20/17 Tom Junk | News and Issues5

Muon tracking efficiency in Pandora• Tingjun noticed in the CI muon tracking efficiency test (1x2x6

FD workspace, GENIE events), that the muon tracking efficiency for Pandora drops from 96% to 72% with the Pandora release that comes with LArSoft v06_60_00

• Ongoing investigation – trying to reproduce in different environments or track down the issue.

• Update from yesterday evening – John Marshall was able to reproduce the issue and has committed a fix to a feature branch of larpandoracontent, asking Andy Blake to check it.

12/20/17 Tom Junk | News and Issues6

12/20/17 Tom Junk | News and Issues7

TRACEv3_08_01

artv2_08_04

canvasv3_00_03

clhepv2_3_4_4

cppunitv1_13_2

messagefacilityv2_01_01

pythonv2_7_13d

rootv6_10_04d

artdaq_corev1_07_12

awscliv1_7_15

boostv1_64_0

caffev1_0e

glogv0_3_5

gslv2_3

hdf5v1_10_1b

lapackv3_7_1

leveldbv1_20

lmdbv0_9_21

opencvv3_3_0a

protobufv3_3_1

gflagsv2_2_1

snappyv1_1_7

fhiclcppv4_06_01

tbbv2017_7

sqlitev3_16_02_00a

fftwv3_3_6_pl2

libxml2v2_9_4b

pythiav6_4_28h

mysql_clientv5_5_54

postgresqlv9_5_6

xrootdv4_5_0c

cetlibv3_01_01

cetlib_exceptv1_01_03

cigetcertv1_16_1

cigetcertlibsv1_1

setpathv1_11

cpnv1.7

cryv1_7h

dune_pardatav01_26_00

dunetpcv06_60_00_02

duneutilv06_60_00

genie_phyoptv2_12_8

genie_xsecv2_12_8

larsoftv06_60_00

lbne_raw_datav1_04_17

larbatchv01_34_03

ifdh_artv2_04_02

laranav06_13_02

larrecov06_49_00

lareventdisplayv06_15_05

larexamplesv06_07_13

larpandorav06_18_00

larsoft_datav1_02_01

larwirecellv06_08_07

jobsub_clientv1_2_5

sam_web_clientv2_0

eigenv3_3_3

geant4v4_10_3_p01a

geniev2_12_8a

lhapdfv5_9_1h

log4cppv1_1_2

pdfsetsv5_9_1b

ifbeamv2_1_3

ifdhcv2_2_3

libwdav2_24_0

ifdhc_configv2_2_3

nucondbv2_1_3

jsoncppv1_7_7

jsonnetv0_9_3

nutoolsv2_16_09

larsimv06_38_00

tensorflowv1_3_0a

larcorev06_15_08

larcorealgv1_12_00

larcoreobjv1_18_00

lardatav06_33_02

lardataobjv1_24_00

rangev3_0_2_4

nusimdatav1_08_04

larevtv06_16_12

marleyv1_0_0a

larpandoracontentv03_09_00

pandorav03_07_00

wirecellv0_6_2

dunetpc dep. tree with version numbers

gcc,cetbuildtools,andGEANT4'sdependenciessuppressed.

madefromoutputofupsdepend

Adding Unpacking modules

12/20/17 Tom Junk | News and Issues8

• Jingbo has added a feature branch to dunetpcfeature/online_monitoring_artdaq

• Has raw decoder modules for RCE, FELIX, SSP, and Timing data.

• We would like to include these in dunetpc's released version- introduces a dependence on dune-raw-data and dunepdsprce

- dune-raw-data is not yet included in prebuilt releases (built from source for the online).

• Feature branch was based on an older dunetpc with dependence on an older version of artdaq_core. This must be kepto stable so that the online TransferInput source works between artdaq and the OM processes.

Adding Unpacking modules• Development head of dunetpc can be setup along with dune-raw-data using

the "offline" qualifier: nu:e14:s56 (John and Tom verified this).

• dune-raw-data has two other qualifiers to maintain stable dependencies- online:e14:s50

- online_monitoring:e14:s50

• John Freeman (or Kurt and others) would have to maintain dune-raw-data versions moving forwards (long after ProtoDUNE-SP runs). Similar to lbne_raw_data, which is still on the maintenance list.

• John had supplied tarballs of lbne_raw_data to be unwound in /grid/fermiappand CVMFS. lbne_raw_data is also built on Jenkins as dunetpc is build on Jenkins and needs it.

• But then Jingbo's modules would have to be merged in to dunetpc on the develop branch. They may evolve independently on the feature branch.

12/20/17 Tom Junk | News and Issues9