make crypto functions execute in the threadpool asynchronously

[jonathanong]

i would expect:

• the async implementation to be accessed through the streaming API
• the sync implementation to be accessed through the legacy .update() and .digest() methods
• semver major as people expect the stream to be return the result synchronously (have seen it in many modules where they don't listen to the readable event)

btw i have no idea how much performance benefit this would provide, if any. i just like the idea of having everything executing in the threadpool...

nodejs/node-v0.x-archive#4298

[kyriosli]

I dont think so. Not like zlib, apis in crypto module are commonly in O(n) or O(nlog n) time.

[Fishrock123]

cc @indutny?

[indutny]

I don't think that there will be any performance benefit for AES/Hashing and stuff like that. Maybe DiffieHellman, but it does not have stream API.

[Fishrock123]

Is there a good case for keeping this request open?

[indutny]

Yeah, for DH.

[jorangreef]

I would like to second this, but for symmetric ciphers (createCipher, createCipheriv, createDecipher, createDeciperiv).

It would really be great to have a way to do cipher and decipher ops off the main thread, not for any performance benefit, but so as not to block the event loop.

I know a single cipher op over 64KB takes around 0.2ms (AES-256-GCM), but for a server doing crypto at rest on an SSD this could mean that 5000-10000 cipher ops a second would quickly burn through the entire main thread budget.

Being able to do cipher and decipher ops in the threadpool would make it easier to get more throughput, without the event loop blocking, and with the event loop being the control plane and not the data plane.

[jorangreef]

I created a module, @ronomon/crypto-async, to test out the idea of doing cipher, hash and hmac operations in the threadpool. Latency per operation is only slightly affected (or not at all) through interaction with the thread pool, but the throughput gains are up to 3x. Best of all, the event loop is not blocked. The benchmark is here.

[bnoordhuis]

@jorangreef Looks interesting and the numbers are impressive but I'm curious how it holds up in real-world scenarios. Since it uses thread pool, it's going to suffer from head-of-line blocking if there are slow DNS or file operations queued up.

[jorangreef]

Thanks @bnoordhuis, would increasing UV_THREADPOOL_SIZE to 64 or 128 help? I think UV_THREADPOOL_SIZE should usually be more than the number of CPU cores because most of the threads will be waiting and will not be hot on the CPU?

[bnoordhuis]

That should help when most threads are blocked on I/O but latency will probably go through the roof when you have all 64 or 128 threads doing encryption.

With computationally bound workloads you normally don't want more than N or N-1 threads (where N = number of cores) because otherwise you pay too much in scheduling overhead.

[jorangreef]

Yes, with the benchmark I set concurrency to 1/2 number of available cores to keep latency within reasonable bounds: https://github.com/ronomon/crypto-async/blob/master/benchmark.js#L6

[jorangreef]

I was surprised that latency increased already when using N, and N-1 threads (where N = number of cores). I thought it would only increase from N-1. Is there some contention in the threadpool that can be optimized?