Repository navigation
iso8859-1 vs windows-1252 #25851
Description
Activity
The consequence of all of this is that when I build Node.js with Chromium's ICU, test/parallel/test-icu-transcode.js fails due to the character "€"
From a glance of the test I couldn't see how the encoding differences would come into play if the test itself was decoded properly. Was the test file stored in Windows-1252? And then got interpreted as utf8 by the CJS loader?
The test loads the euro sign as utf8 and converts to latin1. With iso8859-1 it doesn't encode so it gets replaced by "?".
Is this affected by
full-icu?Also #13722.
I can check again, but iirc full icu doesn't fix this issue. If it did, the full icu build would fail the above mentioned test.
Same actually also applies to "ascii". According to WHATWG it is to be interpreted as Windows-1252 as well.
We have a dedicated WHATWG text encoder/decoder
util.TextDecoderandutil.TextEncoder, which should strictly follow the semantics of the Encoding Standard.buffer.transcode()should indeed just fall back on whatever the linked ICU does. I believe #25866 is a good change because of this.Reacted by Steven R. Loomis- added a commit that references this issue
on Feb 5, 2019 Test fix landed.
- added a commit that references this issue
on Feb 6, 2019 sorry to miss this, but the fix looks good.
This is somewhat related to #13722, but not quite.
Wikipedia contains the gist:
Chromium's ICU interprets "iso8859-1" to mean Windows-1252. Node.js does not. The WHATWG spec suggests Chromium's behavior to be correct.
The consequence of all of this is that when I build Node.js with Chromium's ICU,
test/parallel/test-icu-transcode.jsfails due to the character "€", which Windows-1252 includes, but ISO-8859-1 does not.I propose: