Ctpdev:digi2raw and ctp-raw-recoder fixes by lietava · Pull Request #7472 · AliceO2Group/AliceO2

lietava · 2021-10-30T19:41:45Z

Fixes described here:
#7121 (comment)
:
CTP decoding was producing more logs than all other detectors together, so I've changed a lot of INFOs to DEBUG in your code.
Also, had to touch your digits class.

The CTP digit2row still has this custom option --file-per-link instead of --file-for [all,link,cru], could you please fix this? The output naming is ok.

Then, your raw->digits decoder is for some reason named o2-ctp-reco-workflow, Could you please rename it to something more appropriate, e.g. o2-ctp-raw-decoder. More serious is that it declares a lot of options which it is not supposed to support, I guess it comes from some copy/paste.

--input-type arg (=raw) digits, raw
--output-type arg (=digits) digits, raw
--disable-mc disable sending of MC information
--disable-root-input disable root-files input reader
--disable-root-output disable root-files output writer
Can you have input other than raw and output other than digits?
Raw decoders are not supposed to support MC, so you don't need this option at all.
And for the raw decoders there is no root input anyway.

lietava · 2021-10-30T19:42:37Z

@shahor02 please, have a look and let me know if I understood correctly.

davidrohr · 2021-10-30T21:49:45Z

@lietava : this fails in the full system test with

## sw/BUILD/O2-full-system-test-latest/log
--
command o2-ctp-digi2raw -o raw/CTP --file-per-link had nonzero exit code 1
----- Last log: -----
/sw/BUILD/f1e60ca2e640e01e5f09722b7f71455a79be460f/O2-full-system-test/full-system-test-sim
Running: TIME="#walltime %e" /sw/slc8_x86-64/O2/7472-slc8_x86-64-local1/share/scripts/monitor-mem.sh /usr/bin/time --output=ctpraw.log_time './ctpraw.log_tmp.sh'
/usr/bin/time --output=ctpraw.log_time ./ctpraw.log_tmp.sh
ERROR: unrecognised option '--file-per-link'

could you have a look?

shahor02 · 2021-10-30T21:58:11Z

@lietava could you change in the line https://github.com/lietava/AliceO2/blob/2517903f47b611e0babb12da3cefe41499f1c722/prodtests/full_system_test.sh#L133
-file-per-link to --file-for cru
and commit it to this PR?

lietava · 2021-10-30T23:36:48Z

full_system_test --file-for cru done

shahor02 · 2021-10-31T20:34:50Z

Hi @lietava
could you rebase your branch to fresh dev and force-push? The FST in the FullCI failed with strange error from the dpl-workflow, but for me the dev works fine.

lietava · 2021-10-31T21:20:40Z

@shahor02 dev rebased to ctpdev and pushed

davidrohr

The FullCI is failing, since you have removed the configKeyValues options (see below). Could you add that again please?

davidrohr · 2021-10-31T23:05:19Z

-    {"disable-mc", o2::framework::VariantType::Bool, false, {"disable sending of MC information"}},
-    {"disable-root-input", o2::framework::VariantType::Bool, false, {"disable root-files input reader"}},
-    {"disable-root-output", o2::framework::VariantType::Bool, false, {"disable root-files output writer"}},
-    {"configKeyValues", o2::framework::VariantType::String, "", {"Semicolon separated key=value strings ..."}},


This option configKeyValues must remain

lietava · 2021-11-01T01:14:32Z

@davidrohr configKeyValues is back

alibuild · 2021-11-02T20:08:58Z

Error while checking build/O2/fullCI for 913a145 at 2021-11-02 20:08:

sw/BUILD/O2-latest/log
/bin/sh: warning: setlocale: LC_ALL: cannot change locale (en_US.UTF-8)
-- Installing: /sw/slc8_x86-64/O2/7472-slc8_x86-64-local2/share/macro/run_trac_its.C
-- Installing: /sw/slc8_x86-64/O2/7472-slc8_x86-64-local2/share/macro/CreateBCPattern.C
-- Installing: /sw/slc8_x86-64/O2/7472-slc8_x86-64-local2/share/macro/UploadDummyAlignment.C
-- Installing: /sw/slc8_x86-64/O2/7472-slc8_x86-64-local2/share/macro/UploadMatBudLUT.C
-- Installing: /sw/slc8_x86-64/O2/7472-slc8_x86-64-local2/share/macro/CreateCTPOrbitResetObject.C
-- Installing: /sw/slc8_x86-64/O2/7472-slc8_x86-64-local2/lib/cmake/O2/O2Targets.cmake
-- Installing: /sw/slc8_x86-64/O2/7472-slc8_x86-64-local2/lib/cmake/O2/O2Targets-relwithdebinfo.cmake
-- Installing: /sw/slc8_x86-64/O2/7472-slc8_x86-64-local2/lib/cmake/O2/O2Config.cmake
-- Installing: /sw/slc8_x86-64/O2/7472-slc8_x86-64-local2/lib/cmake/O2/AddRootDictionary.cmake
-- Installing: /sw/slc8_x86-64/O2/7472-slc8_x86-64-local2/lib/cmake/O2/rootcling_wrapper.sh.in
-- Up-to-date: /sw/slc8_x86-64/O2/7472-slc8_x86-64-local2/lib/cmake/O2
-- Installing: /sw/slc8_x86-64/O2/7472-slc8_x86-64-local2/lib/cmake/O2/FindFlukaVMC.cmake
-- Installing: /sw/slc8_x86-64/O2/7472-slc8_x86-64-local2/lib/cmake/O2/FindJAliEnROOT.cmake
-- Installing: /sw/slc8_x86-64/O2/7472-slc8_x86-64-local2/lib/cmake/O2/FindOpenMPMacOS.cmake
-- Installing: /sw/slc8_x86-64/O2/7472-slc8_x86-64-local2/lib/cmake/O2/FindGeant4VMC.cmake
-- Installing: /sw/slc8_x86-64/O2/7472-slc8_x86-64-local2/lib/cmake/O2/FindXRootD.cmake
-- Installing: /sw/slc8_x86-64/O2/7472-slc8_x86-64-local2/lib/cmake/O2/FindGeant4.cmake
-- Installing: /sw/slc8_x86-64/O2/7472-slc8_x86-64-local2/lib/cmake/O2/FindAliRoot.cmake
-- Installing: /sw/slc8_x86-64/O2/7472-slc8_x86-64-local2/lib/cmake/O2/FindO2GPU.cmake
-- Installing: /sw/slc8_x86-64/O2/7472-slc8_x86-64-local2/lib/cmake/O2/FindLibUV.cmake
-- Installing: /sw/slc8_x86-64/O2/7472-slc8_x86-64-local2/lib/cmake/O2/FindlibjalienO2.cmake
-- Installing: /sw/slc8_x86-64/O2/7472-slc8_x86-64-local2/lib/cmake/O2/FindFastJet.cmake
-- Installing: /sw/slc8_x86-64/O2/7472-slc8_x86-64-local2/lib/cmake/O2/O2TestsAdapter.cmake
-- Installing: /sw/slc8_x86-64/O2/7472-slc8_x86-64-local2/lib/cmake/O2/FindGLFW.cmake
-- Installing: /sw/slc8_x86-64/O2/7472-slc8_x86-64-local2/lib/cmake/O2/FindFairRoot.cmake
-- Installing: /sw/slc8_x86-64/O2/7472-slc8_x86-64-local2/lib/cmake/O2/FindO2arrow.cmake
-- Installing: /sw/slc8_x86-64/O2/7472-slc8_x86-64-local2/lib/cmake/O2/O2SimulationDependencies.cmake
-- Installing: /sw/slc8_x86-64/O2/7472-slc8_x86-64-local2/lib/cmake/O2/CMakeLists.txt
-- Installing: /sw/slc8_x86-64/O2/7472-slc8_x86-64-local2/lib/cmake/O2/FindHepMC3.cmake
-- Installing: /sw/slc8_x86-64/O2/7472-slc8_x86-64-local2/lib/cmake/O2/FindRapidJSON.cmake
-- Installing: /sw/slc8_x86-64/O2/7472-slc8_x86-64-local2/lib/cmake/O2/Findpythia6.cmake
-- Installing: /sw/slc8_x86-64/O2/7472-slc8_x86-64-local2/lib/cmake/O2/O2CompileFlags.cmake
-- Installing: /sw/slc8_x86-64/O2/7472-slc8_x86-64-local2/lib/cmake/O2/FindVGM.cmake
-- Installing: /sw/slc8_x86-64/O2/7472-slc8_x86-64-local2/lib/cmake/O2/FindGeant3.cmake
-- Installing: /sw/slc8_x86-64/O2/7472-slc8_x86-64-local2/lib/cmake/O2/FindHepMC.cmake
-- Installing: /sw/slc8_x86-64/O2/7472-slc8_x86-64-local2/lib/cmake/O2/Findpythia.cmake
-- Installing: /sw/slc8_x86-64/O2/7472-slc8_x86-64-local2/lib/cmake/O2/O2Dependencies.cmake
++ cp compile_commands.json /sw/slc8_x86-64/O2/7472-slc8_x86-64-local2
+++ readlink /sw/SOURCES/O2/7472-slc8_x86-64/0
++ DEVEL_SOURCES=/O2
++ '[' /O2 '!=' /sw/SOURCES/O2/7472-slc8_x86-64/0 ']'
++ perl -p -i -e 's|/sw/SOURCES/O2/7472-slc8_x86-64/0|/O2|' compile_commands.json
++ ln -sf /sw/BUILD/1fd11b93f9c2444530bf99529d866d430b6da65f/O2/compile_commands.json /O2/compile_commands.json
++ mkdir -p etc/modulefiles
++ cat
+++ [[ slc == osx ]]
++ mkdir -p /sw/slc8_x86-64/O2/7472-slc8_x86-64-local2/etc/modulefiles
++ rsync -a --delete etc/modulefiles/ /sw/slc8_x86-64/O2/7472-slc8_x86-64-local2/etc/modulefiles

Full log here.

alibuild · 2021-11-03T01:16:59Z

Error while checking build/O2/o2-cs8 for 7997051 at 2021-11-03 02:16:

sw/BUILD/O2-latest/log
      Start  98: test_Framework_test_SimpleStatefulProcessing01
84/98 Test  #98: test_Framework_test_SimpleStatefulProcessing01 ........   Passed    0.55 sec
      Start  99: test_Framework_test_SimpleStringProcessing
85/98 Test  #99: test_Framework_test_SimpleStringProcessing ............   Passed    0.55 sec
      Start 100: test_Framework_test_SimpleTimer
86/98 Test #100: test_Framework_test_SimpleTimer .......................   Passed    1.50 sec
      Start 101: test_Framework_test_SimpleWildcard
87/98 Test #101: test_Framework_test_SimpleWildcard ....................   Passed    1.46 sec
      Start 102: test_Framework_test_SimpleWildcard02
88/98 Test #102: test_Framework_test_SimpleWildcard02 ..................   Passed    1.47 sec
      Start 103: test_Framework_test_SingleDataSource
89/98 Test #103: test_Framework_test_SingleDataSource ..................   Passed    1.52 sec
      Start 104: test_Framework_test_Task
90/98 Test #104: test_Framework_test_Task ..............................   Passed    0.53 sec
      Start 105: test_Framework_test_ExternalFairMQDeviceWorkflow
91/98 Test #105: test_Framework_test_ExternalFairMQDeviceWorkflow ......   Passed    2.10 sec
      Start 106: test_Framework_test_VariablePayloadSequenceWorkflow
92/98 Test #106: test_Framework_test_VariablePayloadSequenceWorkflow ...   Passed    1.25 sec
      Start 107: test_Framework_test_ProcessorOptions
93/98 Test #107: test_Framework_test_ProcessorOptions ..................   Passed    0.54 sec
      Start 116: test_Framework_benchmark_RawParser
94/98 Test #116: test_Framework_benchmark_RawParser ....................   Passed   12.97 sec
      Start 117: test_Framework_benchmark_DPLRawPageSequencer
95/98 Test #117: test_Framework_benchmark_DPLRawPageSequencer ..........   Passed    6.97 sec
      Start 118: test_Framework_test_CustomGUIGL
96/98 Test #118: test_Framework_test_CustomGUIGL .......................***Not Run (Disabled)   0.00 sec
      Start 119: test_Framework_test_CustomGUISokol
97/98 Test #119: test_Framework_test_CustomGUISokol ....................***Not Run (Disabled)   0.00 sec
      Start 120: test_Framework_test_SimpleTracksED
98/98 Test #120: test_Framework_test_SimpleTracksED ....................***Not Run (Disabled)   0.00 sec

100% tests passed, 0 tests failed out of 93

Label Time Summary:
benchmark    =  19.93 sec*proc (2 tests)
dplutils     =  19.93 sec*proc (2 tests)
framework    =  50.88 sec*proc (89 tests)
workflow     =  36.48 sec*proc (26 tests)

Total Test time (real) =  70.97 sec

The following tests did not run:
	 85 - test_Framework_test_BoostSerializedProcessing (Disabled)
	 88 - test_Framework_test_DanglingInputs (Disabled)
	118 - test_Framework_test_CustomGUIGL (Disabled)
	119 - test_Framework_test_CustomGUISokol (Disabled)
	120 - test_Framework_test_SimpleTracksED (Disabled)
++ set +x
++ [[ ! -n '' ]]

Full log here.

Ctpdev:digi2raw and ctp-raw-recoder fixes

shahor02 previously approved these changes Oct 30, 2021

View reviewed changes

shahor02 marked this pull request as ready for review October 30, 2021 20:15

shahor02 requested a review from a team as a code owner October 30, 2021 20:15

lietava dismissed shahor02’s stale review via f23ea1c October 30, 2021 23:34

lietava added 3 commits October 31, 2021 22:18

digi2raw file-for; ctp-reco-workflow rebamed to ctp-raw-decoder

3dbcc46

clang format

4cde80c

full_system_test --file-for cru done

54fe40e

lietava force-pushed the ctpdev branch from f23ea1c to 54fe40e Compare October 31, 2021 21:19

davidrohr requested changes Oct 31, 2021

View reviewed changes

lietava added 2 commits November 1, 2021 02:11

configKeyValues is back

aa46a50

clang format

ce2ecaa

shahor02 added 7 commits November 1, 2021 20:11

Suppress unneeded options

a8d9a34

Restore ignore-dist-stf option

cce1405

Restore getRawToDigitConverterSpec

cb2c507

remove unnecessary includes

4015b86

fixes for CTP bool noLostTF

bdaa547

do not pass suppressed CTP option in the dpl-workflow

ebd3f66

fix1

913a145

fix typo

7997051

shahor02 merged commit 83ba2d0 into AliceO2Group:dev Nov 3, 2021

ezradlesser pushed a commit to ezradlesser/AliceO2 that referenced this pull request Dec 2, 2021

Merge pull request AliceO2Group#7472 from lietava/ctpdev

cb6c92c

Ctpdev:digi2raw and ctp-raw-recoder fixes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Ctpdev:digi2raw and ctp-raw-recoder fixes#7472

Ctpdev:digi2raw and ctp-raw-recoder fixes#7472
shahor02 merged 13 commits into
AliceO2Group:devfrom
lietava:ctpdev

lietava commented Oct 30, 2021

Uh oh!

lietava commented Oct 30, 2021

Uh oh!

davidrohr commented Oct 30, 2021

Uh oh!

shahor02 commented Oct 30, 2021

Uh oh!

lietava commented Oct 30, 2021

Uh oh!

shahor02 commented Oct 31, 2021

Uh oh!

lietava commented Oct 31, 2021

Uh oh!

davidrohr left a comment

Uh oh!

davidrohr Oct 31, 2021

Uh oh!

lietava commented Nov 1, 2021

Uh oh!

alibuild commented Nov 2, 2021

Uh oh!

alibuild commented Nov 3, 2021

Uh oh!

Reviewers

Assignees

Labels

Milestone

Development

Uh oh!

4 participants

Uh oh!

Conversation

lietava commented Oct 30, 2021

Uh oh!

lietava commented Oct 30, 2021

Uh oh!

davidrohr commented Oct 30, 2021

Uh oh!

shahor02 commented Oct 30, 2021

Uh oh!

lietava commented Oct 30, 2021

Uh oh!

shahor02 commented Oct 31, 2021

Uh oh!

lietava commented Oct 31, 2021

Uh oh!

davidrohr left a comment

Choose a reason for hiding this comment

Uh oh!

davidrohr Oct 31, 2021

Choose a reason for hiding this comment

Uh oh!

lietava commented Nov 1, 2021

Uh oh!

alibuild commented Nov 2, 2021

Uh oh!

alibuild commented Nov 3, 2021

Uh oh!

Reviewers

Assignees

Labels

Milestone

Development

Uh oh!

4 participants