Repository navigation
Closed
Labels
confirmed-bugIssues and PRs for confirmed bugs.Issues and PRs for confirmed bugs.streamIssues and PRs related to Node.js streams.Issues and PRs related to Node.js streams.
Description
Version
v18.14.1
Platform
Linux 703748a51615 5.18.8-051808-generic #202206290850 SMP PREEMPT_DYNAMIC Wed Jun 29 08:59:08 UTC 2022 x86_64 Linux
Subsystem
stream
What steps will reproduce the bug?
const stream = require("node:stream");
const consumers = require("node:stream/consumers");
const createInnerTransfrom = () => new stream.Transform({
objectMode: true,
construct(callback) {
this.push('header from constructor\n');
callback();
},
transform: (row, encoding, callback) => {
callback(null, JSON.stringify(row) + '\n');
},
});
const createOuterTransfrom = () => {
let innerTranfrorm;
return new stream.Transform({
objectMode: true,
transform(row, encoding, callback) {
if (!innerTranfrorm) {
innerTranfrorm = createInnerTransfrom();
innerTranfrorm.on('data', (data) => this.push(data));
callback();
}
else if (innerTranfrorm.write(row)) {
process.nextTick(callback);
}
else {
innerTranfrorm.once('drain', callback);
}
},
});
};
consumers.text(stream.Readable.from([
'create InnerTransform',
'firstLine',
'secondLine',
]).pipe(createOuterTransfrom())).then((text) => console.log('output:\n', text));How often does it reproduce? Is there a required condition?
always
What is the expected behavior?
output:
header from constructor
"firstLine"
"secondLine"
What do you see instead?
output:
header from constructor
"secondLine"
"firstLine"
Additional information
I expect Transform to always process incoming items in order, even with reckless code like in the example.
Metadata
Metadata
Assignees
Labels
confirmed-bugIssues and PRs for confirmed bugs.Issues and PRs for confirmed bugs.streamIssues and PRs related to Node.js streams.Issues and PRs related to Node.js streams.
Activity
[-]Transform stream changing order of items[/-][+]stream.Transform changing order of items[/+]rluvaton commentedon Feb 24, 2023
You accidentally wrote the expected in the current behavior
zuozp8 commentedon Feb 24, 2023
I corrected it, thank you
ronag commentedon Feb 24, 2023
Do you think you can make an even simpler example?
rluvaton commentedon Feb 24, 2023
What I saw is when you remove the
constructit works wellronag commentedon Feb 24, 2023
That's a good hint.
zuozp8 commentedon Feb 24, 2023
sometimes it starts/stops occuring when you change some
process.nextTick(callback)intocallack()or the other way around, so it's some kind of race-conditionI wasn't able to replicate the problem without nested Transforms
rluvaton commentedon Feb 24, 2023
When you wrap the:
like this:
it solves the problem
rluvaton commentedon Feb 24, 2023
Found another solution and updated my PR
mcollina commentedon Feb 24, 2023
This is indeed a bug. Here is a simplified version:
This fixes the example:
So I think we are missing a
nextTickin the constructor logic.mcollina commentedon Feb 24, 2023
I think https://github.com/nodejs/node/blob/1f75a9513fc829fbd5326a5c9bfa7678b7431eb8/lib/internal/streams/readable.js#LL222C7-L222C7 should be delayed with a
nextTick.rluvaton commentedon Feb 24, 2023
delaying it with
nextTickdid not worked...mcollina commentedon Feb 24, 2023
Given the following script:
It will error:
This is correct, as you are pushing things after the stream ended.
rluvaton commentedon Feb 24, 2023
I tested with your simplified version
mcollina commentedon Feb 24, 2023
Here is the fix: #46818
stream: always delay construct callback by a nextTick
stream: always delay construct callback by a nextTick
stream: always delay construct callback by a nextTick
stream: always delay construct callback by a nextTick
stream: always delay construct callback by a nextTick