Skip to content
This repository was archived by the owner on May 24, 2025. It is now read-only.
This repository was archived by the owner on May 24, 2025. It is now read-only.

Implementation feedback: x86/x64 lacks float64 <> float16 instructions #12

Open
Open
Implementation feedback: x86/x64 lacks float64 <> float16 instructions#12

Description

@syg
syg
opened on Mar 28, 2024

Currently, Math.f16round and assigning into Float16Array does a conversion from float64 to float16 using the roundTiesToEven mode.

On x86/x64, as far as V8 can tell there are no generally available instructions to directly narrow float64 to float16. There are two extensions that supports float16. F16C, an SSE extension, which has been available since 2007. And AVX512-FP16, which is much newer and is currently only available on the latest Sapphire Rapids Xeon chips. Both extensions only support converting float32 to float16. F16C only supports converting float32 to float16. AVX512-FP16 does support float64 to float16 but is only on newest Xeons.

(On ARM such instructions exist, this is an x86-only issue.)

Notably, using roundTiesToEven, narrowing float64 to float16 is not equivalent to narrowing float64 to float32 to float16. This means that on x86/x64, there is no efficient codegen possible for float16 rounding in the foreseeable future. There is only efficient codegen possible by first converting to float32, e.g. Math.f16round(Math.fround(n)) and f16a[idx] = Math.fround(n).

The main problem is that the obvious code you want to write is slow because it requires software emulation.

There are alternatives, but each one is kind of unsavory in its own way.

  • We could spec float64 to float16 conversion directly as float64 to float32 to float16, under roundTiesToEven. This is unsavory because Math.f16round sure doesn't sound like it should be doing an intermediate narrowing first.
  • We could spec float64 to float16 conversion as using roundTiesToZero, in which case it's unobservable if you first round to a float32 intermediate. This is unsavory because all other floating point operations in JS use roundTiesToEven.
  • We could add a new f16via32round and Float16ViaFloat32Array (names up to bikeshedding) that do an intermediate round.

For comparison, this is probably less of an issue for other languages that are adding float16 because float32s are already a primitive type in those languages. In C++, this would only come up if you're actually converting doubles to halfs. Similar for Wasm.

But for JS, since the only number type we have is double, it'll end up being a very common operation whether intentional or not. I think there's also a question whether real world code tend to go from float64 to float16 directly, or they're mostly going from float32 (that is, maybe x86 doesn't have instructions for it because there's no demand).

Thoughts from other implementers @dminor @msaboff?

h/t @SPY

Activity

  1. bakkot commented on Mar 28, 2024

    @bakkot
    bakkot
    on Mar 28, 2024
    Member

    Swift also added float16 recently. You maybe interested in their implementation of this logic, which does a cast via float->half and then handles the rounding edge cases explicitly (via some bit-twiddling I do not understand).

  2. anba commented on Mar 29, 2024

    @anba
    anba
    on Mar 29, 2024

    And AVX512-FP16, which is much newer and is currently only available on the latest Sapphire Rapids Xeon chips. Both extensions only support converting float32 to float16.

    vcvtsh2sd and vcvtsd2sh from AVX512-FP16 support direct float64<>float16, but as you've noted, this extension is currently only available on Sapphire Rapids processors. (float16 to float64 conversion doesn't require AVX512-FP16 when implemented through vcvtph2ps followed by vcvtss2sd, which means it only requires support for F16C.)

    (On ARM such instructions exist, this is an x86-only issue.)

    More detailed:

    • ARM64 supports float64<>float16 through fcvt.
    • ARM32 with Neon supports float16 to float64 through vcvtb.f32.f16 followed by vcvt.f64.f32. float64 to float16 requires software emulation.

  3. syg commented on Mar 29, 2024

    @syg
    syg
    on Mar 29, 2024
    Author

    vcvtsh2sd and vcvtsd2sh from AVX512-FP16 support direct float64<>float16,

    Aren't those for integers?

  4. anba commented on Mar 29, 2024

    @anba
    anba
    on Mar 29, 2024

    Aren't those for integers?

    No, I don't think so. Clang/GCC also emit both instructions when using -mavx512fp16: https://godbolt.org/z/4d9M4fMc9

  5. syg commented on Mar 29, 2024

    @syg
    syg
    on Mar 29, 2024
    Author

    Thanks for checking, corrected the OP.

  6. syg commented on Apr 1, 2024

    @syg
    syg
    on Apr 1, 2024
    Author

    After sitting on this for a while, I think my main concern is mainly that, unlike in other languages that have float as a primitive type, in JS it will be too easy to do the slow thing of assigning directly into a Float16Array.

    @bakkot What are your thoughts on having Float16Via32Array (hopefully with a less horrible name)? We have precedence for TAs with special conversion rules in Uint8Clamped.

  7. ljharb commented on Apr 1, 2024

    @ljharb
    ljharb
    on Apr 1, 2024
    Member

    The weirdness of Uint8Clamped has come up a number of times, and is a big part of the pushback to https://github.com/tc39/proposal-dataview-get-set-uint8clamped - that seems like an undesirable precedent to follow.

  8. syg commented on Apr 1, 2024

    @syg
    syg
    on Apr 1, 2024
    Author

    The weirdness of Uint8Clamped has come up a number of times, and is a big part of the pushback to https://github.com/tc39/proposal-dataview-get-set-uint8clamped - that seems like an undesirable precedent to follow.

    It'll be as unmotiviated to add to DataView as Uint8Clamped, but why is that undesirable? That seems like a fairly small thing to me, that DataView isn't 1:1 with TA constructors.

  9. ljharb commented on Apr 1, 2024

    @ljharb
    ljharb
    on Apr 1, 2024
    Member

    I meant that the general feeling has consistently been "Uint8Clamped was a mistake and it'd be better if it didn't exist".

  10. syg commented on Apr 1, 2024

    @syg
    syg
    on Apr 1, 2024
    Author

    I meant that the general feeling has consistently been "Uint8Clamped was a mistake and it'd be better if it didn't exist".

    Could you refresh my memory on this feeling? My recollection of the discussion around the DataView proposal is that it was primarily about that a getUint8Clamped getter is misleading in that clamping is a conversion behavior that happens on set, and there isn't such a thing as a Uint8Clamped as a integer type. I don't make the connection back to that "Uint8ClampedArray itself considered harmful".

  11. ljharb commented on Apr 1, 2024

    @ljharb
    ljharb
    on Apr 1, 2024
    Member

    Sorry if I was unclear. You're correct about the implications for my proposal that were discussed - but the sentiment I've repeatedly heard around any Typed Array discussion in committee, including that one, is that Uint8ClampedArray is "weird" and shouldn't be there.

  12. syg commented on Apr 1, 2024

    @syg
    syg
    on Apr 1, 2024
    Author

    I see. I don't know if it's "weird" per se, but Uint8ClampedArray has a narrower use case than the other TA constructors. But so does Float16Array, and I'd like the common use patterns to not be slow. Like, the main motivation of Float16Array is we're making a TA type for a better WebGPU and similar kinds of APIs. This is very much like Uint8ClampedArray, which exists to better processing APIs that want to get bytes that are known to not have underflowed/overflowed.

  13. bakkot commented on Apr 2, 2024

    @bakkot
    bakkot
    on Apr 2, 2024
    Member

    Hm. Since this is a pretty specialized use, it would not be the worst thing to have such an ugly operation, but I'd want to know how bad the penalty of doing it in software actually is. The approach Swift takes looks like it should be cheap almost all of the time (i.e. it's pretty rare that you would actually take the branch).

    I suspect a lot of applications would prefer to pay the performance penalty in exchange for having the behavior match (say) Pytorch or C++.

    Would we expose both? If so I'd also want to expose whether the hardware supports Float16 casts, so that you could choose that option on hardware which supported it.

  14. syg commented on Apr 2, 2024

    @syg
    syg
    on Apr 2, 2024
    Author

    I suspect a lot of applications would prefer to pay the performance penalty in exchange for having the behavior match (say) Pytorch or C++.

    This is where my above question on how people are using float16 comes from. It is true other languages let you go from float64 -> float16, but are most people actually going through float32 anyway, since their source is actually in float32? If so that means today to match that behavior they'd be calling a lot of Math.frounds.

    Would we expose both? If so I'd also want to expose whether the hardware supports Float16 casts, so that you could choose that option on hardware which supported it.

    Yes, definitely both.

  15. anba commented on Jul 29, 2024

    @anba
    anba
    on Jul 29, 2024

    Implementing the technique linked in #12 (comment) reduces the overhead for Float64 -> Float16 conversion to ~50% when compared to using Math.fround for Float64 -> Float32 -> Float16 in local µ-benchmarks.

    Relevant SpiderMonkey bugs:

    Generated x86 assembly code to convert Float64 -> Float16:

    cvtsd2ss   %xmm1, %xmm0         # Convert Float64 -> Float32.
vmovd      %xmm0, %r12d         # Get bit representation of Float32 value.
andl       $0x7fffffff, %r12d   # Mask off sign bit.
cmpl       $0x33000000, %r12d   # Underflow to zero.
jb         .L1
cmpl       $0x47800000, %r12d   # Overflow to infinity.
jae        .L1
cmpl       $0x38800000, %r12d   # Detection of subnormal numbers.
setae      %bl
movzbl     %bl, %ebx
shll       $12, %ebx
andl       $0x1fff, %r12d
cmpl       %ebx, %r12d          # Check round and sticky bits.
jne        .L1
movdqa     .L2(%rip), %xmm15    # Load [0, 0, 0, 1].
psignd     %xmm1, %xmm15
paddd      %xmm15, %xmm0        # Adjust mantissa by -1/0/+1.
.set .L1
vcvtps2ph  $0x4, %xmm0, %xmm0   # Convert Float32 -> Float16.

    This looks okay-ish to me.

  16. anba commented on Jul 29, 2024

    @anba
    anba
    on Jul 29, 2024

    From what I've read, it seems like Zen 6 will support AVX512-FP16. And for future Intel CPUs, it seems like AVX10 will provide FP16 support.

  17. Constellation commented on Aug 12, 2024

    @Constellation
    Constellation
    on Aug 12, 2024
    Member

    @anba 's shown code looks simple and looks like the future Intel CPUs will eventually get native support for double -> FP16. So, mildly, I prefer to the current proposal.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions