nightly linux coverage build failing

[bcoe]

I've noticed our nightly coverage CI build is failing with the following error:

20:01:54 In file included from ../deps/v8/src/libplatform/task-queue.h:12:0,
20:01:54                  from ../deps/v8/src/libplatform/task-queue.cc:5:
20:01:54 ../deps/v8/src/base/macros.h:237:33: error: 'is_trivially_copyable' is not a member of 'std'
20:01:54    static constexpr bool value = std::is_trivially_copyable<T>::value;
20:01:54                                  ^
20:01:54 ../deps/v8/src/base/macros.h:237:33: note: suggested alternative:
20:01:54 In file included from ../deps/v8/src/libplatform/task-queue.h:12:0,
20:01:54                  from ../deps/v8/src/libplatform/task-queue.cc:5:
20:01:54 ../deps/v8/src/base/macros.h:210:8: note:   'v8::base::is_trivially_copyable'
20:01:54  struct is_trivially_copyable {
20:01:54         ^

https://ci.nodejs.org/job/node-test-commit-linux-coverage-daily/

This started three days ago, and seems similar to the issue discussed here:

#30020

I was wondering if there were any modifications we made to our other Linux jobs to make tests work on V8@7.9.

CC: @targos, @nodejs/build

• Version: head
• Platform: linux
• Subsystem: build

[richardlau]

I was wondering if there were any modifications we made to our other Linux jobs to make tests work on V8@7.9.

Yes, there were. nodejs/build#1975 (followed by nodejs/build#1985, nodejs/build#1994). (aside, noticed when investigating that nodejs/build#1994 regressed nodejs/build#1985 so I've submitted a fixup nodejs/build#2045.)

For https://ci.nodejs.org/job/node-test-commit-linux-coverage-daily/ we should make the job run the select-compiler.sh script to select the correct level of gcc to compile with... with the complication that in that job $nodes ends up being benchmark which doesn't match anything in the existing script. Does anyone know why this job runs (or if it needs to run) on benchmark? For comparison, the https://ci.nodejs.org/job/node-test-commit-v8-linux/ job runs on benchmark-ubuntu1604-intel-64.

[rvagg]

probably because it's available and won't block anything else?

Maybe just upgrade the machine it's using and you can skip these compiler problems. Is there anything special that needs to be installed on the host? (that could be a reason it's on a specific machine). ubuntu1804-docker is available and doesn't get much of a workout. It'll select among 4 docker hosts that share capacity with the sharedlibs and alpine tests but capacity is nice and flexible with these.

[bcoe]

@rvagg @richardlau I don't believe, at this time, coverage has any special machine requirements; any Linux environment with inspector enabled should do the trick.

[richardlau]

Does anyone know why this job runs (or if it needs to run) on benchmark?

So found this: https://github.com/nodejs/build/tree/master/jenkins/scripts/coverage#steps

• At the end of the scheduled job it rsync's the generated data to the benchmarking data machine. We do this so that once the job is complete the data is in a place where we know we can pull it from, and that pulling that data will not affect any other jobs (for example jobs measuring performance on the benchmark machine).
• At hourly intervals the the data is rsync'd from the benchmarking data machine to the website. This is triggered from the nodejs.org website machine and data is pulled from the benchmarking data machine. This allows us to minimize who can modify the nodejs.org website as no additional access is required.

which I guess means coverage doesn't need to run on the benchmarking machine but does need to be able to rsync the data to it.

[richardlau]

and also this:

The current setup depends on past runs being in /home/iojs/coverage-out/out on the machine that it is run on so that the generated index includes the current and past data. For this and other reasons described in the other sections, the job is pegged to run on: iojs-softlayer-benchmark

So we might lose historic data if we move hosts and /home/iojs/coverage-out/out isn't preserved.

[bcoe]

@richardlau we're in the process of testing coverage collection on codecov.io, which has the benefit of allowing us to merge coverage between Windows and Linux.

but, I don't think this experiment is quite ready yet, and we definitely want to keep our history of coverage (I didn't realize it lived on that specific machine).

[bcoe]

@mhdawson can you speak towards whether rebuilding our coverage machine will cause issues for the historical data we host on coverage.nodejs.org?

My thinking, if we can get coverage merging like I hope, is that we might eventually link out to an external site for the coverage reports ... but we certainly don't want to lose the history, it's nice to be able to see growth over time.

[Trott]

I think it's OK if we end up losing coverage data from past years. It's not ideal, but I don't think it's worth a special effort to keep coverage data around from more than a year ago. If someone disagrees, perhaps they can archive it themselves (or see if it can be archived by archive.org or similar).

[Trott]

Is the solution for this just that someone needs to re-run ansible for a few machines?

[Trott]

Is the solution for this just that someone needs to re-run ansible for a few machines?

(That's what @bcoe suggested, I think?)

[Trott]

Is the solution for this just that someone needs to re-run ansible for a few machines?

(That's what @bcoe suggested, I think?)

(Ah yes, definitely what he suggested, but in a DM to me.)

[rvagg]

whether rebuilding our coverage machine

for what purpose are we needing to rebuild? I thought we were just getting a newer compiler hooked in to the coverage job to solve the problem? As @richardlau said:

For https://ci.nodejs.org/job/node-test-commit-linux-coverage-daily/ we should make the job run the select-compiler.sh script to select the correct level of gcc to compile with... with the complication that in that job $nodes ends up being benchmark which doesn't match anything in the existing script

@richardlau were you going to do this?

[richardlau]

whether rebuilding our coverage machine

for what purpose are we needing to rebuild? I thought we were just getting a newer compiler hooked in to the coverage job to solve the problem? As @richardlau said:

For https://ci.nodejs.org/job/node-test-commit-linux-coverage-daily/ we should make the job run the select-compiler.sh script to select the correct level of gcc to compile with... with the complication that in that job $nodes ends up being benchmark which doesn't match anything in the existing script

@richardlau were you going to do this?

@rvagg I can add select-compiler.sh to the job but by itself it's not going to fix this as it will be called with $nodes set to benchmark which the script doesn't handle yet. I'll PR a change to the script but I'm going to be guessing as the job runs on infra-softlayer-ubuntu1404-x64-2 which I don't have access to (since it's infra- and not test-). I think based on https://github.com/nodejs/build/blob/172cef05296037fb2a7459196d8849359a53cf25/ansible/roles/baselayout/vars/main.yml#L116-L122 that gcc-6 should be available on that machine.