Add CI testing for nodereport

[mhdawson]

If we are ready to test across platforms I think we should, at least nightly and when changes are proposed.

@rnchamberlain can you confirm if we are ready for nightly testing or not and if so what the test should run.

I see the job

1. build node
2. do an npm install using the nodereport repo
3. run the tests.

[richardlau]

We'll need #17 to land for AIX support. Probably #15 too so that the CI can parse the test results.

[rnchamberlain]

As @richardlau - we need #15 #16 #17 to land, also another npm publish. Once those are in I think we should be in good shape to get a CI test going.

Re 1. build node - can we just unzip a .tar for each platform?

[richardlau]

We should also see if we can get nodereport added to CITGM runs.

[richardlau]

nodejs/citgm#226 for adding to CITGM.

[gdams]

I think that now we are looking at adding a linter to nodereport we should also look at adding a travis CI. I know that travis doesn't cover all operating systems but it would be good to have an automatic run on all new PR's for ubuntu and darwin

[gibfahn]

Presumably we could use Appveyor for windows as well?

[gdams]

yeah I've got all the configuration files to use Appveyor

[mhdawson]

I think we can close this now right as we have the CI job in place ?

[gibfahn]

I think we are looking to get automatic runs in place for new PRs (i.e. triggering the Jenkins CI on new Pull Requests). Is there another Issue open to track that? If not we should probably leave this open.

cc/ @gdams

[rnchamberlain]

Great to have the CI, thanks! It would be good if we could improve the CI test harness output a bit. There is a long key printed on the left hand side which makes the useful information on the right hand side wrap, and harder to read, eg see this job: https://ci.nodejs.org/view/post-mortem/job/nodereport-continuous-integration/52/MACHINE=ubuntu1604-64/console

verbose: e29356f2d00afe1a9b4f96e9a120208c6ce6a2ff npm-test:| ok 3 - Validating node-report.20170210.190340.31017.001.txt # time=31.601ms

Is that key needed, else can we remove it?

[gdams]

that's the citgm stuff, this is why I suggested not using citgm and just using npm test

[gibfahn]

Yeah, after looking at it in more detail I agree that it doesn't make sense to use citgm to test single modules, the benefit doesn't outweigh the hassle it adds.

It should be pretty easy to fix the jobs though. @mhdawson It's just a question of git clone node-report + npm test | tee test.tap instead of npm install citgm and citgm node-report.

[gdams]

@gibfahn I wouldn't do that

npm install $REPO/node-report#$BRANCH
cd node_modules/node-report
npm install
npm test | tee test.tap

[gibfahn]

@gdams do you know what the difference is between npm installing from a git repo and git cloning (other than the fact it doesn't put it in node_modules)?

[rnchamberlain]

@gibfahn npm installing from a git repo will honour .npmignore, so it won't install the tests and demos etc, whereas git cloning will