Skip to content

Periodic test against master and canary #187

Description

@mmarchini

I think we should have nightly/weekly test runs against master and canary. This would help us know earlier when something breaks. nodejs/node#17685 was an awesome start, but as @bnoordhuis said in the PR it won't catch semantic changes.

I'm willing to setup the infrastructure for it. I just need to know if we can have this on Travis (by using https://docs.travis-ci.com/user/cron-jobs/) or if we should introduce this on Node.js CI.

/cc @joyeecheung

Activity

  1. mmarchini commented on Apr 24, 2018

    @mmarchini
    ContributorAuthor

    We could even have a status table in the readme 😄

    Version Status
    6.x build passing
    8.x build passing
    9.x build passing
    10.x build passing
    master build passing
    canary build passing

    or

    Version v6.x v8.x v9.x v10.x master canary
    Status build passing build passing build passing build passing build passing build passing
  2. joyeecheung commented on Apr 24, 2018

    @joyeecheung
    Member

    There is nodejs/build#1025 by the way.

    I have seen people wgetting nightlies in the scripts to run tests on nightlies with Travis, but that looks rather hacky...

  3. mmarchini commented on Apr 24, 2018

    @mmarchini
    ContributorAuthor

    I saw that PR a while ago, but it seems stalled for some time. I'll take a look and see if we can get some traction on it.

    BTW, should we move our current CI infrastructure from Travis to our Jenkins?

  4. joyeecheung commented on Apr 24, 2018

    @joyeecheung
    Member

    BTW, should we move our current CI infrastructure from Travis to our Jenkins?

    @mmarchini That's the plan, but until it happens Travis is better than nothing.

  5. mmarchini commented on May 1, 2018

    @mmarchini
    ContributorAuthor

    I have seen people wgetting nightlies in the scripts to run tests on nightlies with Travis, but that looks rather hacky...

    I looked into Travis docs and it seems like they use nvm to manage Node.js versions. Maybe we can use it to get nightlies in a not-so-hacky way. I think I'll try to add something to Travis while we don't have support for llnode on Jenkins.

  6. mmarchini commented on May 1, 2018

    @mmarchini
    ContributorAuthor

    Proof of Concept: https://github.com/mmarchini/llnode

    I've also removed Node.js v4.x, added v10.x (which is broken 😢) and added Mac OS X builds for v6.x, v8.x, v9.x, and v10.x. .travis.yaml grew a lot, but I think it's worth. Nightlies are ignores in the final CI result.

    Setting up Cron jobs is straightforward, but need admin access to Travis I think. It can be done in the menu Settings at the end of the page.

    I'll open a PR tomorrow. First I wanted to test the Cron thingy.

    P.S.: I had to use https://github.com/bjfish/travis-matrix-badges to get badges for each Job since Travis don't have this feature.

  7. joyeecheung commented on May 2, 2018

    @joyeecheung
    Member

    Setting up Cron jobs is straightforward, but need admin access to Travis I think. It can be done in the menu Settings at the end of the page.

    Last time I checked we cannot access the admin page of Travis (mainly because we do not authorize Travis or any other third-party services to read the nodejs organization - that's also the reason why we have to use the GitHub bot to update CI status). Now this restriction makes less sense since we've moved the security repos out of this organization though.

  8. mmarchini commented on May 2, 2018

    @mmarchini
    ContributorAuthor

    Last time I checked we cannot access the admin page of Travis

    Uh, ok, nightly tests will have to wait for Jenkins then. But we could still update our travis.yaml to cover more versions while Jenkins is not ready for llnode. I'll open a PR later today.

  9. mmarchini commented on May 2, 2018

    @mmarchini
    ContributorAuthor

    Crazyish idea: maaaaybe we could contact Travis support, explain the situation and see if they could enable daily Cron jobs against master for us 😶

  10. mmarchini commented on May 31, 2018

    @mmarchini
    ContributorAuthor

    @joyeecheung now that we can properly enable Travis in org repositories, should we still migrate to Jenkins or should we stick with Travis?

    I think it's a good idea to stay with Travis since it will not inflict more burden on the Build WG and also make our setup a lot easier.

  11. joyeecheung commented on Jun 1, 2018

    @joyeecheung
    Member

    @mmarchini We can keep Travis because it's automatic, but enabling Jenkins would allow us to test it on more platforms.

    (Also I think we are trying to add Travis to the core as well, just for a faster check)

  12. mmarchini commented on Jun 18, 2018

    @mmarchini
    ContributorAuthor

    Ok, since we don't need the github-bot for Travis anymore (nodejs/github-bot#183), we can properly change settings on Travis now 🎉. I enabled daily runs against llnode master, let's see if it works properly.

    We still need to update the labels on README.md to reflect the status from https://travis-ci.com/ instead of https://travis-ci.org/.

    P.S.: @joyeecheung I'm not sure if I should have access to Travis settings, but I have 🤔

  13. joyeecheung commented on Jun 18, 2018

    @joyeecheung
    Member

    @mmarchini I guess if you have write access to this repo, then you can have access to Travis?

  14. mmarchini commented on Jul 1, 2018

    @mmarchini
    ContributorAuthor

    Waiting for bjfish/travis-matrix-badges#9 to land before updating our README badges.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions