Buffer.from() output breaks after optimizing in nodejs 22.7

[unilynx]

Version

v22.7.0

Platform

Darwin moe.fritz.box 23.5.0 Darwin Kernel Version 23.5.0: Wed May  1 20:13:18 PDT 2024; root:xnu-10063.121.3~5/RELEASE_ARM64_T6030 arm64

Subsystem

Buffer

What steps will reproduce the bug?

let i = 0;
for(; i < 1_000_000; i++) {
  const ashex = Buffer.from("\x80").toString("hex");
  if (ashex === '80')
    break;
 else if(ashex !== 'c280')
    console.log("Unexpected return value", ashex); //this never happened for me
}

if(i<1_000_000) {
  console.log("FAILED after %d iterations",i);
  process.exit(1);
} else
  console.log("PASSED after %d iterations",i);

How often does it reproduce? Is there a required condition?

For me, it will consistently fail somewhere between 7000 to 1200 iterations.

What is the expected behavior? Why is that the expected behavior?

In node 20 this code does not fail , even after 1_000_000 iterations. Buffer.from("\x80").toString("hex") always returns c280 on node 20

What do you see instead?

Buffer.from("\x80").toString("hex") incorrectly returns 80 after sufficient iterations

Additional information

No response

[avivkeller]

I managed to reproduce using your code snippet, and the slightly modified one below (for readability)

let i = 0;

while (i < 1_000_000) {
  const asHex = Buffer.from("\x80").toString("hex");

  if (asHex === '80') {
    break;
  } else if (asHex !== 'c280') {
    console.log("Unexpected return value:", asHex);
  }

  i++;
}

if (i < 1_000_000) {
  console.log("FAILED after %d iterations", i);
  process.exit(1);
} else {
  console.log("PASSED after %d iterations", i);
}

$ node repro.js
FAILED after 8827 iterations
$ node repro.js
FAILED after 5509 iterations
$ node repro.js
FAILED after 48741 iterations
$ node repro.js
FAILED after 7487 iterations
$ node repro.js
FAILED after 9189 iterations

[unilynx]

bisecting gives me, as first bad commit:

commit c9dabe2
Author: Robert Nagy ronagy@icloud.com
Date: Thu Aug 15 03:01:05 2024 +0200

buffer: use fast API for writing one-byte strings

however, that one's even worse and shows a lot of incorrect 'asHex' values:

Unexpected return value afdead0b
Unexpected return value 00dead0b
Unexpected return value 00000000
Unexpected return value 00dead0b
Unexpected return value 01dead0b
Unexpected return value 00000000
Unexpected return value 905e0150
Unexpected return value 00738148

so it looks like the more extreme bug got fixed (probably in 7800893) but then issues still remained for (eg) \x80

[avivkeller]

CC @ronag

[unilynx]

just to confirm the above - if I take the v22.x branch (5ae0260) but revert 7800893 and a5a60e6, the issue is no longer present

[ronag]

I'll have a look.

[ronag]

let i = 0;

while (i < 1_000_000) {
  const buf = Buffer.from("\x80")

  if (buf[0] !== 194 || buf[1] !== 128) {
    console.log("Unexpected return value:", buf, buf[0], buf[1]);
    break
  }

  i++;
}

if (i < 1_000_000) {
  console.log("FAILED after %d iterations", i);
  process.exit(1);
} else {
  console.log("PASSED after %d iterations", i);
}

[ronag]

This seems to be something in V8.

In

uint32_t FastWriteString(Local<Value> receiver,
                         const v8::FastApiTypedArray<uint8_t>& dst,
                         const v8::FastOneByteString& src,
                         uint32_t offset,
                         uint32_t max_length) {
}

src.length === 1 i.e. FastOneByteString thinks that Buffer.from(['\x80']) has a length of 1.

@targos @joyeecheung @nodejs/buffer

[ronag]

Which is kind of correct... is it the slow path that is broken?

[ronag]

actually @anonrig @lemire might have ideas?