Use JAliEn credentials in CCDB API by Atlantic777 · Pull Request #5826 · AliceO2Group/AliceO2

Atlantic777 · 2021-03-30T11:51:57Z

This PR adds JAliEn credentials manager to CCDB API and decorates curl handle with JAliEn credentials. Depends on: alisw/alidist#2932

cc @costing @iraklic @yuw726

Signed-off-by: Nikola Hardi <nhardi@cern.ch>

sawenzel · 2021-04-06T08:15:30Z

 #include <TMessage.h>
 #include "CCDB/CcdbObjectInfo.h"

+#if !defined(__CINT__) && !defined(__MAKECINT__) && !defined(__ROOTCLING__) && !defined(__CLING__)


why do we need this?

The libjalien isn't a ROOT library and doesn't have a ROOT dictionary. I noticed problems with ROOT interpreter when CcdbApi.h was included because it couldn't understand this type.

There is similar code in JAliEn-ROOT / AliEn-ROOT-Legacy, but maybe there is another way around it. In any case, if this header is not explicitly included in a ROOT interpreter session then maybe it works without it.

@sawenzel What do you advise?

No hard opinion here. But usually this should not be necessary. One can already use o2::ccdb::CcdbApi in a ROOT session without including the header. And the protection could be put outside as well.
We can also merge as is and remove later.

jgrosseo · 2021-04-12T06:30:32Z

@TimoWilken The CI errors seems unrelated, or am I mistaken?

TimoWilken · 2021-04-12T08:03:39Z

@jgrosseo the libjalienO2 dependency (added in this PR) isn't picked up by QC. ~~I'll open a PR in alidist and restart these checks when that's merged.~~ While the O2 build runs fine, could you hold off on merging for now so we don't break the QC build?

Barthelemy · 2021-04-12T08:15:23Z

QC should not need to be touched. Dependencies should be transitive in both aliBuild and CMake.

Barthelemy · 2021-04-12T08:33:44Z

I don't have a solution but this is not going to work:
HINTS "${LIBJALIENO2}"

This variable is unknown in downstream packages. You could use the environment variable LIBJALIENO2_ROOT

jgrosseo · 2021-04-29T14:06:41Z

@costing Do you know who could follow this up after @Atlantic777 departure. My understanding is that just the cmake has to be fixed.

iraklic · 2021-04-29T14:37:58Z

@jgrosseo that would probably be me. I read through this thread but I'm not entirely sure what is going on. If you know what needs to be fixed in cmake please let me know and I will have a look.

jgrosseo · 2021-04-29T14:40:02Z

Apart from resolving the merge conflict which is probably simple, follow the comments by @Barthelemy about the different variable name

jgrosseo · 2021-04-29T14:40:11Z

Thanks for looking into it!

yuw726 · 2021-04-29T14:54:29Z

Hi @Barthelemy @jgrosseo @iraklic,
Isn't ${LIBJALIENO2} passed to cmake in o2 alidist script? Please correct me if I am wrong.

Barthelemy · 2021-05-03T07:01:32Z

@yuw726 the cmake variable LIBJALIENO2 is set to the value of the environment variable LIBJALIENO2_ROOT when configuring O2.

Later in the build, QualityControl is built and when it loads the dependencies it tries to use the FindlibjalienO2.cmake and fails because, at this stage, LIBJALIENO2 is not knows. You should try to use the environment variable LIBJALIENO2_ROOT in the Find script. Not guaranteed to work but it is worth trying.

To test do aliBuild build QualityControl.

jgrosseo · 2021-05-06T14:02:57Z

@iraklic Any news?

iraklic · 2021-05-07T06:44:47Z

@iraklic Any news?

Sorry, I am having very busy week. It is on my todo. BTW: do I understand it right that you expect a new PR from me with the conflict in CCDB/src/CcdbApi.cxx resolved or there is some other way of modifying this PR (I have never done that)?

jgrosseo · 2021-05-07T06:56:31Z

@iraklic Any news?

Sorry, I am having very busy week. It is on my todo. BTW: do I understand it right that you expect a new PR from me with the conflict in CCDB/src/CcdbApi.cxx resolved or there is some other way of modifying this PR (I have never done that)?

You cannot push to the branch which made this PR, but you can just start from that branch, push that into your own fork, and then open a PR based on that one. This will include all these changes. We will at that point close this PR.

iraklic · 2021-05-07T06:58:15Z

@iraklic Any news?

Sorry, I am having very busy week. It is on my todo. BTW: do I understand it right that you expect a new PR from me with the conflict in CCDB/src/CcdbApi.cxx resolved or there is some other way of modifying this PR (I have never done that)?

You cannot push to the branch which made this PR, but you can just start from that branch, push that into your own fork, and then open a PR based on that one. This will include all these changes. We will at that point close this PR.

Thank you. That's what I was going to do, but wanted to confirm.

jgrosseo · 2021-05-18T19:14:39Z

@iraklic Any news?

Sorry, I am having very busy week. It is on my todo. BTW: do I understand it right that you expect a new PR from me with the conflict in CCDB/src/CcdbApi.cxx resolved or there is some other way of modifying this PR (I have never done that)?

You cannot push to the branch which made this PR, but you can just start from that branch, push that into your own fork, and then open a PR based on that one. This will include all these changes. We will at that point close this PR.

Thank you. That's what I was going to do, but wanted to confirm.

Any news?

jgrosseo · 2021-05-21T09:37:26Z

Fixed conflict

costing · 2021-05-27T06:42:55Z

Done, thank you @TimoWilken ! I think it's also requiring a review from @Barthelemy

Barthelemy · 2021-05-27T07:05:38Z

@costing could you give me a bit of context ?

jgrosseo · 2021-05-27T08:02:54Z

@costing could you give me a bit of context ?

It is fine. I approved it.

jgrosseo · 2021-05-27T08:03:57Z

fullCI is unrelated. I'll ignore the space checker on this one (apologies @vkucera , we'll fix this later), and merge when the other two CIs have passed.

Barthelemy · 2021-05-27T09:35:01Z

I tried to compile and got the following error:

2021-05-27@11:25:43:DEBUG:QualityControl:QualityControl:0: -- The following REQUIRED packages have not been found:
2021-05-27@11:25:43:DEBUG:QualityControl:QualityControl:0: 
2021-05-27@11:25:43:DEBUG:QualityControl:QualityControl:0:  * libjalienO2
2021-05-27@11:25:43:DEBUG:QualityControl:QualityControl:0:    For CCDB API
2021-05-27@11:25:43:DEBUG:QualityControl:QualityControl:0:

The command I used on CC7 is

aliBuild build QualityControl --defaults o2 --debug

alidist is up to date with master

jgrosseo · 2021-05-27T09:57:45Z

@TimoWilken do you know why QC misses this?

iraklic · 2021-06-14T18:40:38Z

I am trying to understand nuts and bolts of what is going on here, but I am having trouble to compile dev at this point. It crushes with this:
/home/iraklic/jAliEn_Clean/sw/SOURCES/Monitoring/v3.8.6/v3.8.6/src/Transports/Unix.cxx:18:10: fatal error: filesystem: No such file or directory #include <filesystem> ^~~~~~~~~~~~ compilation terminated.

I checked out dev and building with aliBuild build O2 --defaults o2

Any idea?

Barthelemy · 2021-06-15T12:52:22Z

some of your stuff is not up to date. Make sure you update O2

iraklic · 2021-06-15T13:19:36Z

What kind of stuff? I pulled O2 using aliBuild init O2@dev --defaults o2 and even did git pull today just in case and tried to rebuild. But have the same issue.

iraklic · 2021-06-21T17:13:06Z

Is there a mattermost/slack channel or something similar when I can ask about my building error? Or should I use alice-project-analysis-task-force mailing list? Or is there a fail-safe tag for O2 I can check out that will surely build? Any help is appreciated.

iraklic · 2021-06-21T21:44:33Z

OK. looks like I am not alone in this: https://alice.its.cern.ch/jira/browse/O2-2327. But the problem is that the workaround yields an entirely new issue:

In file included from /home/iraklic/jAliEn_Clean/sw/SOURCES/O2/master/0/Common/Utils/src/ShmManager.cxx:19:
/home/iraklic/jAliEn_Clean/sw/SOURCES/O2/master/0/Common/Utils/include/CommonUtils/ShmManager.h:40:20: error: field 'counter' has incomplete type 'std::atomic<int>'
   40 |   std::atomic<int> counter = 0; // atomic counter .. counter number of attached processes
      |                    ^~~~~~~

Any suggestions are appreciated

jgrosseo · 2021-07-07T11:53:03Z

The JIRA ticket you quote is closed and it is said " This should not be the case anymore, provided you move to alibuild 1.8.4 " .
Which alibuild do you have?

jgrosseo · 2021-07-07T11:53:36Z

You can ask in O2 release integration or O2 analysis challenge on mattermost any additional questions.

github-actions · 2021-08-07T00:37:38Z

This PR did not have any update in the last 30 days. Is it still needed? Unless further action in will be closed in 5 days.

iraklic · 2021-08-07T01:03:34Z

@jgrosseo has this been merged through other PR? I am asking because there are mentions of conflicts mentioned here being fixed and so on. The compilation problem that I had I fixed using the workaround suggested in the ticket that I mentioned. Anyways please let me know if this has been merged though some other PR or if you want me to check it on my own and open new PR with this change. Thanks.

jgrosseo · 2021-08-16T09:26:54Z

@iraklic I don't think this has been merged through another PR. It is just that other changes had been made to the same code and the conflicts need to be resolved.

jgrosseo · 2021-08-16T09:29:06Z

I reopened it and do not see a conflicts actually.
However, there are failures in the CI? Could you update this PR with what needs to be done, so that it goes green and we can merge it?

pzhristov · 2021-08-30T14:22:23Z

I will try to fix the CMake part and provide modifications in the alidist recipes since the CI issues are related to the discovery of the package.

pzhristov · 2021-09-01T18:19:08Z

The following PR alisw/alidist#3275 should solve the issues we observe in the CI.

Volodymyr Yurchenko and others added 4 commits March 30, 2021 10:02

Use libjalien to load the certificate and CAs

3352c2e

Signed-off-by: Nikola Hardi <nhardi@cern.ch>

Add the credentials manager

e13f187

JAliEn credentials manager improvements

9261565

Enable redirection on upload in CcdbApi

7f11461

Atlantic777 requested review from a team, Barthelemy, costing and sawenzel as code owners March 30, 2021 11:51

Include clang-format formattig updates

55f7e2d

costing previously approved these changes Mar 30, 2021

View reviewed changes

jgrosseo previously approved these changes Apr 1, 2021

View reviewed changes

sawenzel reviewed Apr 6, 2021

View reviewed changes

Merge branch 'dev' into feature-jalien-credentials

ba20200

costing approved these changes May 27, 2021

View reviewed changes

jgrosseo approved these changes May 27, 2021

View reviewed changes

github-actions Bot added the stale label Aug 7, 2021

github-actions Bot closed this Aug 13, 2021

jgrosseo reopened this Aug 16, 2021

github-actions Bot closed this Aug 22, 2021

costing reopened this Aug 30, 2021

pzhristov mentioned this pull request Sep 1, 2021

Preparation to use JAliEn credentials in CCDB API (PR 5826) alisw/alidist#3275

Merged

pzhristov merged commit fe381f8 into AliceO2Group:dev Sep 3, 2021

aphecetche mentioned this pull request Sep 7, 2021

CCDBApi: protection against missing X509_CERT_DIR env var #6984

Merged

Uh oh!

Conversation

Atlantic777 commented Mar 30, 2021

Uh oh!

sawenzel Apr 6, 2021

Choose a reason for hiding this comment

Uh oh!

Atlantic777 Apr 9, 2021

Choose a reason for hiding this comment

Uh oh!

jgrosseo Apr 12, 2021

Choose a reason for hiding this comment

Uh oh!

sawenzel Apr 12, 2021

Choose a reason for hiding this comment

Uh oh!

jgrosseo commented Apr 12, 2021

Uh oh!

TimoWilken commented Apr 12, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Barthelemy commented Apr 12, 2021

Uh oh!

Barthelemy commented Apr 12, 2021

Uh oh!

jgrosseo commented Apr 29, 2021

Uh oh!

iraklic commented Apr 29, 2021

Uh oh!

jgrosseo commented Apr 29, 2021

Uh oh!

jgrosseo commented Apr 29, 2021

Uh oh!

yuw726 commented Apr 29, 2021

Uh oh!

Barthelemy commented May 3, 2021

Uh oh!

jgrosseo commented May 6, 2021

Uh oh!

iraklic commented May 7, 2021

Uh oh!

jgrosseo commented May 7, 2021

Uh oh!

iraklic commented May 7, 2021

Uh oh!

jgrosseo commented May 18, 2021

Uh oh!

jgrosseo commented May 21, 2021

Uh oh!

costing commented May 27, 2021

Uh oh!

Barthelemy commented May 27, 2021

Uh oh!

jgrosseo commented May 27, 2021

Uh oh!

jgrosseo commented May 27, 2021

Uh oh!

Barthelemy commented May 27, 2021

Uh oh!

jgrosseo commented May 27, 2021

Uh oh!

iraklic commented Jun 14, 2021

Uh oh!

Barthelemy commented Jun 15, 2021

Uh oh!

iraklic commented Jun 15, 2021

Uh oh!

iraklic commented Jun 21, 2021

Uh oh!

iraklic commented Jun 21, 2021

Uh oh!

jgrosseo commented Jul 7, 2021

Uh oh!

jgrosseo commented Jul 7, 2021

Uh oh!

github-actions Bot commented Aug 7, 2021

Uh oh!

iraklic commented Aug 7, 2021

Uh oh!

jgrosseo commented Aug 16, 2021

TimoWilken commented Apr 12, 2021 •

edited

Loading