Skip to content

Investigate flaky test-dgram-udp6-send-default-host on OS X #6577

Closed
Closed
Investigate flaky test-dgram-udp6-send-default-host on OS X#6577
Labels
dgramIssues and PRs related to UDP and the dgram module.macosIssues and PRs related to the macOS platform.testIssues and PRs related to Node.js core tests and test infrastructure.

Description

@Trott
Trott
opened on May 4, 2016
  • Version: master branch
  • Platform: OS X
  • Subsystem: test? dgram?

Failure in CI on test of unrelated change: https://ci.nodejs.org/job/node-test-commit-osx/3210/nodes=osx1010/tapTestReport/test.tap-213/

not ok 213 test-dgram-udp6-send-default-host.js
# 
# assert.js:90
#   throw new assert.AssertionError({
#   ^
# AssertionError: message was received correctly
#     at Socket.<anonymous> (/Users/iojs/build/workspace/node-test-commit-osx/nodes/osx1010/test/parallel/test-dgram-udp6-send-default-host.js:29:10)
#     at emitTwo (events.js:106:13)
#     at Socket.emit (events.js:191:7)
#     at UDP.onMessage (dgram.js:532:8)
  ---
  duration_ms: 0.145

/cc @nodejs/testing @addaleax

Activity

  1. Trott
    added
    dgramIssues and PRs related to UDP and the dgram module.
    testIssues and PRs related to Node.js core tests and test infrastructure.
    macosIssues and PRs related to the macOS platform.
    on May 4, 2016

  2. Trott commented on May 4, 2016

    @Trott
    Trott
    on May 4, 2016
    MemberAuthor

    /cc @mcollina

  3. Trott commented on May 4, 2016

    @Trott
    Trott
    on May 4, 2016
    MemberAuthor

    Stress test: https://ci.nodejs.org/job/node-stress-single-test/687/nodes=osx1010/

  4. santigimeno commented on May 4, 2016

    @santigimeno
    santigimeno
    on May 4, 2016
    Member

    Have seen this failure locally a couple of times though I cannot reproduce it consistently :/.
    Maybe use assert.strictEqual here would help to find out what's going on.

  5. Trott commented on May 4, 2016

    @Trott
    Trott
    on May 4, 2016
    MemberAuthor

    @santigimeno Yup, I had the same thought and am putting in the PR for that right now...

  6. Trott
    added a commit that references this issue on May 4, 2016

    test: improve assert message for debugging

    2268660

  7. Trott commented on May 4, 2016

    @Trott
    Trott
    on May 4, 2016
    MemberAuthor

    #6581

  8. mcollina commented on May 5, 2016

    @mcollina
    mcollina
    on May 5, 2016
    SponsorMember

    This is fairly unexpected, as the ipv4 equivalent of this test is not flaky.

    Is this flaky only on Mac? Hopefully #6581 should tell what's happening. I have never seen it fail here.

  9. santigimeno
    added a commit that references this issue on May 5, 2016

    test: improve assert message for debugging

    9408fbb

  10. santigimeno commented on May 5, 2016

    @santigimeno
    santigimeno
    on May 5, 2016
    Member

    It looks like the problem is that sometimes the messages are not received in order, which makes sense being UDP. See output from https://ci.nodejs.org/job/node-stress-single-test/nodes=osx1010/690 :

    not ok 1 test-dgram-udp6-send-default-host.js
# 
# assert.js:90
#   throw new assert.AssertionError({
#   ^
# AssertionError: 'hello' === 'yyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyy
#     at Socket.<anonymous> (/Users/iojs/build/workspace/node-stress-single-test/nodes/osx1010/test/parallel/test-dgram-udp6-send-default-host.js:29:10)
#     at emitTwo (events.js:106:13)
#     at Socket.emit (events.js:191:7)
#     at UDP.onMessage (dgram.js:532:8)
  ---
  duration_ms: 0.139

  11. addaleax commented on May 5, 2016

    @addaleax
    addaleax
    on May 5, 2016
    Member

    Could it be that the kernel decides to reorder the packets here for some reason? That’s allowed, although I can’t see why it would. But it would explain why the test is flaky and why this is an OS-specific issue.

  12. santigimeno commented on May 5, 2016

    @santigimeno
    santigimeno
    on May 5, 2016
    Member

    @addaleax Some comments here like this suggest that could be the reason

  13. addaleax commented on May 5, 2016

    @addaleax
    addaleax
    on May 5, 2016
    Member

    👍 I guess the IPv4 test has a chance of being flaky too, then?

  14. santigimeno commented on May 5, 2016

    @santigimeno
    santigimeno
    on May 5, 2016
    Member

    Is there an equivalent IPv4 test? I can't find it.

  15. addaleax commented on May 5, 2016

    @addaleax
    addaleax
    on May 5, 2016
    Member

    @santigimeno test/parallel/test-dgram-send-default-host.js

  16. santigimeno commented on May 5, 2016

    @santigimeno
    santigimeno
    on May 5, 2016
    Member

    @addaleax Thanks! I guess it should also be rewritten

  17. addaleax
    closed this as completedin #6607on May 10, 2016
  18. evanlucas
    added a commit that references this issue on May 17, 2016

    test: allow out-of-order replies in dgram tests

    01f010f
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    dgramIssues and PRs related to UDP and the dgram module.macosIssues and PRs related to the macOS platform.testIssues and PRs related to Node.js core tests and test infrastructure.

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions