Skip to content

test: cctest MultipleEnvironmentsPerIsolate & AtExitWithArgument are flaky (no-snapshot) #14206

Closed
Closed
test: cctest MultipleEnvironmentsPerIsolate & AtExitWithArgument are flaky (no-snapshot)#14206
Labels
c++Issues and PRs that require attention from people who are familiar with C++.flaky-testIssues and PRs involving tests that fail intermittently in CI.help wantedIssues that need assistance from volunteers or PRs that need help to proceed.lib / srcIssues and PRs involving general changes in the lib/ or src/ directories.testIssues and PRs related to Node.js core tests and test infrastructure.v8 engineIssues and PRs related to the V8 dependency.

Description

@refack
refack
opened on Jul 12, 2017
  • Version: master
  • Platform: multiple
  • Subsystem: test,V8

Moving nodejs/build#789 to core
The test has intermitently failed on:

  • ppcbe-ubuntu1404
  • armv7-wheezy
  • centos5-64
  • docker_alpine34-64

https://ci.nodejs.org/job/node-test-commit-plinux/10137/nodes=ppcbe-ubuntu1404/console
https://ci.nodejs.org/job/node-test-commit-plinux/10138/nodes=ppcbe-ubuntu1404/console
https://ci.nodejs.org/job/node-test-commit-plinux/10139/nodes=ppcbe-ubuntu1404/console
https://ci.nodejs.org/job/node-test-commit-plinux/10156/nodes=ppcbe-ubuntu1404/
https://ci.nodejs.org/job/node-test-commit-arm/10826/nodes=armv7-wheezy/console
https://ci.nodejs.org/job/node-test-commit-plinux/10165/nodes=ppcbe-ubuntu1404/console
https://ci.nodejs.org/job/node-test-commit-plinux/10166/nodes=ppcbe-ubuntu1404/console
https://ci.nodejs.org/job/node-test-commit-linux/11117/nodes=centos5-64
https://ci.nodejs.org/job/node-test-commit-plinux/10172/nodes=ppcbe-ubuntu1404/console
https://ci.nodejs.org/job/node-test-commit-linux/11130/nodes=ubuntu1604_docker_alpine34-64/console

One of the more interesting outputs (from docker_alpine34-64):

[----------] 3 tests from EnvironmentTest
[ RUN      ] EnvironmentTest.AtExitWithEnvironment
[       OK ] EnvironmentTest.AtExitWithEnvironment (384 ms)
[ RUN      ] EnvironmentTest.AtExitWithArgument
[       OK ] EnvironmentTest.AtExitWithArgument (388 ms)
[ RUN      ] EnvironmentTest.MultipleEnvironmentsPerIsolate


#
# Fatal error in , line 0
# API fatal error handler returned after process out of memory
#
Received signal 4 ILL_ILLOPN 55c3c9aafe69

The more common output:

[----------] 3 tests from EnvironmentTest
[ RUN      ] EnvironmentTest.AtExitWithEnvironment
[       OK ] EnvironmentTest.AtExitWithEnvironment (362 ms)
[ RUN      ] EnvironmentTest.AtExitWithArgument
[       OK ] EnvironmentTest.AtExitWithArgument (354 ms)
[ RUN      ] EnvironmentTest.MultipleEnvironmentsPerIsolate
Build was aborted

This is the test code

TEST_F(EnvironmentTest, MultipleEnvironmentsPerIsolate) {
const v8::HandleScope handle_scope(isolate_);
const Argv argv;
Env env1 {handle_scope, isolate_, argv};
Env env2 {handle_scope, isolate_, argv};

AtExit(*env1, at_exit_callback1);
AtExit(*env2, at_exit_callback2);
RunAtExit(*env1);
EXPECT_TRUE(called_cb_1);
EXPECT_FALSE(called_cb_2);

RunAtExit(*env2);
EXPECT_TRUE(called_cb_2);
}

From
test/cctest/test_environment.cc

Activity

  1. refack
    added
    c++Issues and PRs that require attention from people who are familiar with C++.
    flaky-testIssues and PRs involving tests that fail intermittently in CI.
    help wantedIssues that need assistance from volunteers or PRs that need help to proceed.
    lib / srcIssues and PRs involving general changes in the lib/ or src/ directories.
    testIssues and PRs related to Node.js core tests and test infrastructure.
    v8 engineIssues and PRs related to the V8 dependency.
    on Jul 12, 2017

  2. addaleax commented on Jul 12, 2017

    @addaleax
    addaleax
    on Jul 12, 2017
    Member

    Quoting #13861 (comment):

    Running the cctest locally makes valgrind complain in a way that points to us not cleaning up libuv handles properly in the multi-Environment tests, similar to the problem fixed by d5db4d2. If I’m right (and valgrind complaining should be a good sign for that), this is not a problem that’s going to effect regular users, and more likely to be a problem with the test itself.

    I’ve also previously asked whether it’s possible to get a core dump from that machine to make sure; I think you might be the best person to have an answer?

    I think all we need is somebody who has the time to dig into the valgrind logs to figure this out.

  3. Trott commented on Jul 13, 2017

    @Trott
    Trott
    on Jul 13, 2017
    Member

    Still happening, of course: https://ci.nodejs.org/job/node-test-commit-plinux/10186/nodes=ppcbe-ubuntu1404/console

    Is anyone on this? Anyone especially sensible to loop in on it?

  4. refack commented on Jul 13, 2017

    @refack
    refack
    on Jul 13, 2017
    ContributorAuthor

    I'll try on Windows, but !@#$@%#$%^ MMMV (it might not even be a problem here 🤷‍♂️ )

  5. refack commented on Jul 13, 2017

    @refack
    refack
    on Jul 13, 2017
    ContributorAuthor

    We can try to call @indutny to the flag (it's partly his fault, JK nothing but   ❤️ )

  6. mhdawson commented on Jul 13, 2017

    @mhdawson
    mhdawson
    on Jul 13, 2017
    Member

    Created this issue to request access to the BE machine were it recreates most easily for @jBarz : nodejs/build#795

    He's tried to reproduce locally with no luck.

  7. jBarz commented on Jul 13, 2017

    @jBarz
    jBarz
    on Jul 13, 2017
    Contributor

    I can reproduce on build machine. However, it is intermittent (about once in 5 tries)

  8. mhdawson commented on Jul 13, 2017

    @mhdawson
    mhdawson
    on Jul 13, 2017
    Member

    I wondering if we should exclude this test until we can figure out what is going on. It seems to happen quire regularly for PPC and alpine.

  9. jBarz commented on Jul 14, 2017

    @jBarz
    jBarz
    on Jul 14, 2017
    Contributor

    Continuing to investigate. So far I have found it to hang at the following location in api.cc from v8 when initializing the isolate.

    Isolate::Scope isolate_scope(v8_isolate);
 if (params.entry_hook || !i::Snapshot::Initialize(isolate)) {
   isolate->Init(NULL);  // <--- Hangs here
 }

  10. refack commented on Jul 14, 2017

    @refack
    refack
    on Jul 14, 2017
    ContributorAuthor

    I wondering if we should exclude this test until we can figure out what is going on. It seems to happen quire regularly for PPC and alpine.

    If we have a way to make sure it doesn't get neglected, I'm +1

  11. 17 remaining items

  12. jBarz commented on Aug 8, 2017

    @jBarz
    jBarz
    on Aug 8, 2017
    Contributor

    hey @refack Last week was extremely busy with other things.
    I am going to try to get this resolved this week.

  13. addaleax
    added 4 commits that reference this issue on Aug 10, 2017

    src: add overlooked handle to cleanup

    5a0ee6d

    test: properly order freeing resources in cctest

    682a951

    Revert "test: add DISABLED_ prefix to commented out test"

    3f95be9

    Revert "test: disable MultipleEnvironmentsPerIsolate"

    557eb7b

  14. addaleax commented on Aug 10, 2017

    @addaleax
    addaleax
    on Aug 10, 2017
    Member

    Fix is in #14749

  15. addaleax
    added a commit that references this issue on Aug 10, 2017

    src: add overlooked handle to cleanup

    b559e66
  16. addaleax
    closed this as completedin 93f25f4on Aug 10, 2017
  17. addaleax
    added 6 commits that reference this issue on Aug 10, 2017

    Revert "test: add DISABLED_ prefix to commented out test"

    ef7da42

    Revert "test: disable MultipleEnvironmentsPerIsolate"

    c80d400

    src: add overlooked handle to cleanup

    033773c

    test: properly order freeing resources in cctest

    0bc3124

    Revert "test: add DISABLED_ prefix to commented out test"

    8ff2a5c

    Revert "test: disable MultipleEnvironmentsPerIsolate"

    a781bb4
  18. sjarmak
    added a commit that references this issue on Jul 27, 2026

    test: properly order freeing resources in cctest

    93f25f4
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    c++Issues and PRs that require attention from people who are familiar with C++.flaky-testIssues and PRs involving tests that fail intermittently in CI.help wantedIssues that need assistance from volunteers or PRs that need help to proceed.lib / srcIssues and PRs involving general changes in the lib/ or src/ directories.testIssues and PRs related to Node.js core tests and test infrastructure.v8 engineIssues and PRs related to the V8 dependency.

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions