You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
In any case, having V8 use threads outside of the thread pool seems wrong. If 4 is the right default, you would in fact use more than 4 if V8 spins up its own threads. If you expect 4 + additional V8 threads to be a good number, then 4 is chosen too conservatively.
@hashseed I'm not sure what the performance difference is (waiting for a libuv thread vs. OS scheduler), but if V8 were to use the libuv thread pool, some node requests would/could get blocked (even more so than they may be currently), whereas they may not have before.
Iiuc V8 simply prefers to let the embedder take control. In case of Chrome, page start-up can be very busy wrt threading, and Chrome's scheduler likely has a better grasp than the OS scheduler.
The problem is that a single threadpool is used for IO and CPU threads.
If there were two threadpools, one could be used for IO (and have many threads) and the other could be used for CPU (and have threads === cores).
This would make tuning possible. Currently, there's no way to tune the threadpool for both use cases.
If libuv knows that all its threads are IO only, then the above can be rolled out as follows:
All AsyncWorker, Node and other userland threads continue to use the current threadpool. This becomes known as the CPU threadpool and keeps the existing 4 thread default.
Add a new "IO" threadpool with a saner, conservative yet tuneable default (perhaps 16).
Libuv moves only its own known IO ops into this new threadpool and exposes a way for userland to do the same if it wants to.
@jorangreef We've been discussing such schemes in libuv since the thread pool was first added. You can find at least two attempts in my fork, maybe more.
Both attempts stranded on not performing significantly better most of the time and significantly worse some of the time. :-/
Were you benchmarking latency or throughput? "Significantly worse" latency (say 1.1x) may not be that bad if it means significantly more throughput.
Was the workload mostly IO bound or mostly CPU bound?
The idea with two separate threadpools is to delegate these choices to the user. Everyone's benchmark requirements are different.
And if there is no advantage gained by separating CPU and IO intensive operations into their own theadpools, then the arguments against increasing the default 4 thread limit should no longer hold.
Are you 100% confident that the current hardcoded 4 thread default is the optimal solution?
A bit of both. Node and libuv's benchmarks test both ends of the spectrum.
Just to double-check, when you benchmarked using two separate threadpools (one for IO tasks, one for CPU tasks), did you let both threadpools contend for the same set of cores or did you pin them to separate sets of cores?
In a sufficiently warmed-up webapp (assuming the most common node use case), do the v8 threads incur any significant CPU work? I thread-profiled a client server app with ~20% CPU consumption by the server process, and could not find v8 threads as contributing. Any suggested (v8) tunables to get more conclusive info?
@gireeshpunathil the v8 threads mostly handle i/o, I've only seen an uptick in thread cpu with heavy garbage collection activity. Nodejs is single-threaded, so the main thread will normally account for almost all the cpu usage. One can introduce some concurrency with the cluster module.
@andrasq - thanks. But sorry, your explanation seems orthogonal to my understanding:
v8 threads handle mostly i/o
which I/O? Are you talking about the primordial (main) thread? I doubt that is classified under v8 thread.
Nodejs is single-threaded, ...
in this discussion and in #14001 we are definitely talking about a number of background threads from v8 module, that is different from the priomordial thread and libuv worker threads. And this discussion focusses on the tradeoff of tenanting the v8 threads with libuv worker threads.
So, my question stands as:
(i) What are those v8 threads which will potentially contend for time slice with more critical work from libuv? Execution tracers? CPU profilers? GC helpers? JIT helpers?
(ii) How do we bring those threads to the forefront to be seen as eating up cycles? (any tunables in the commandline and tunables in the code). This will help us make observations in terms of time-slice distribution variations between the threads and make inferences on throughput, latency as a function of some of these tunables.
(iii) How significant these threads would be in terms of CPU consumption in a fully warmed up production server (no tracing and no new scripts)?
thanks @bnoordhuis - that explains.
In terms of measurements, perf(1) did not help to get thread-wise split up (there is a -t flag, but behavior is weird) so I was using AIX tprofand the result, as I mentioned earlier, suggests that v8 threads contribute very less (not displayed in the top consumers)
I agree this depends on the workload characteristics of the application. Given:
(i) every new page brings in new script (chrome) vs. relatively boot-time-only script (node)
(ii) highly transactional based web workload means most objects collected in the scavenge phase
v8 threads may not consume much slice for either GC or JIT, and is in-line with my tprof observation.
I wish if I could modify the test to manifest the contrasting characteristics and was /am looking for pointers on that line - so that we know the extent to which these threads are insignificant, and what is the tipping point beyond which they show up.
This discussion seems to have run its course. I'm going to close, but if you think that's wrong and there's something concrete, feel free to re-open (if GitHub allows) or comment (requesting it be re-opened if you wish) or open another issue as appropriate.
FWIW the work for creating a Node.js-specific v8::Platform is being done at #14001. In that PR, it was decidednot to merge the libuv and V8 thread pools into one, but instead manage them separately for performance.
Activity
mscdex commentedon Mar 15, 2017
If that were possible, doing so would further reduce the performance of
fs,dns.lookup()(default method used internally by node), etc.jiakai1000 commentedon Mar 15, 2017
@mscdex We can increase libuv thread number.
Also, doing so we can make thread pool load more balance.
mscdex commentedon Mar 15, 2017
Sure, you can increase it, but I would guess most people use the default (currently 4).
jiakai1000 commentedon Mar 15, 2017
Well, and we also can increase default libuv thread number...
mscdex commentedon Mar 15, 2017
I'm not 100% sure but I think 4 may have been chosen as that is/was the typical number of cores/cpus on most machines?
jasnell commentedon Mar 15, 2017
While increasing the default will hopefully be something we can do at some point, 4 is still the most common average if I'm remembering correctly.
hashseed commentedon Mar 15, 2017
In any case, having V8 use threads outside of the thread pool seems wrong. If 4 is the right default, you would in fact use more than 4 if V8 spins up its own threads. If you expect 4 + additional V8 threads to be a good number, then 4 is chosen too conservatively.
mscdex commentedon Mar 15, 2017
@hashseed I'm not sure what the performance difference is (waiting for a libuv thread vs. OS scheduler), but if V8 were to use the libuv thread pool, some node requests would/could get blocked (even more so than they may be currently), whereas they may not have before.
hashseed commentedon Mar 15, 2017
FWIW V8 hands over thread management to Chrome when embedded in Chrome.
jiakai1000 commentedon Mar 15, 2017
@hashseed Could you please explain the reason what Chrome did ?
mscdex commentedon Mar 15, 2017
Node is quite a different thing than Chrome though ;-)
hashseed commentedon Mar 15, 2017
@jeisinger probably knows more.
Iiuc V8 simply prefers to let the embedder take control. In case of Chrome, page start-up can be very busy wrt threading, and Chrome's scheduler likely has a better grasp than the OS scheduler.
10 remaining items
jeisinger commentedon Jul 19, 2017
That enum is not used by v8. We only added it because chrome used to use Windows worker pool reflecting WT_EXECUTELONGFUNCTION
jorangreef commentedon Jul 19, 2017
The problem is that a single threadpool is used for IO and CPU threads.
If there were two threadpools, one could be used for IO (and have many threads) and the other could be used for CPU (and have threads === cores).
This would make tuning possible. Currently, there's no way to tune the threadpool for both use cases.
If libuv knows that all its threads are IO only, then the above can be rolled out as follows:
All AsyncWorker, Node and other userland threads continue to use the current threadpool. This becomes known as the CPU threadpool and keeps the existing 4 thread default.
Add a new "IO" threadpool with a saner, conservative yet tuneable default (perhaps 16).
Libuv moves only its own known IO ops into this new threadpool and exposes a way for userland to do the same if it wants to.
bnoordhuis commentedon Jul 19, 2017
@jorangreef We've been discussing such schemes in libuv since the thread pool was first added. You can find at least two attempts in my fork, maybe more.
Both attempts stranded on not performing significantly better most of the time and significantly worse some of the time. :-/
jorangreef commentedon Jul 20, 2017
Thanks @bnoordhuis
Were you benchmarking latency or throughput? "Significantly worse" latency (say 1.1x) may not be that bad if it means significantly more throughput.
Was the workload mostly IO bound or mostly CPU bound?
The idea with two separate threadpools is to delegate these choices to the user. Everyone's benchmark requirements are different.
And if there is no advantage gained by separating CPU and IO intensive operations into their own theadpools, then the arguments against increasing the default 4 thread limit should no longer hold.
Are you 100% confident that the current hardcoded 4 thread default is the optimal solution?
bnoordhuis commentedon Jul 20, 2017
Hah, I don't think I ever claimed it was. Good enough most of the time, but optimal? No, sir!
A bit of both. Node and libuv's benchmarks test both ends of the spectrum.
jorangreef commentedon Jul 20, 2017
Just to double-check, when you benchmarked using two separate threadpools (one for IO tasks, one for CPU tasks), did you let both threadpools contend for the same set of cores or did you pin them to separate sets of cores?
gireeshpunathil commentedon Jul 24, 2017
In a sufficiently warmed-up webapp (assuming the most common node use case), do the v8 threads incur any significant CPU work? I thread-profiled a client server app with ~20% CPU consumption by the server process, and could not find v8 threads as contributing. Any suggested (v8) tunables to get more conclusive info?
andrasq commentedon Jul 24, 2017
@gireeshpunathil the v8 threads mostly handle i/o, I've only seen an uptick in thread cpu with heavy garbage collection activity. Nodejs is single-threaded, so the main thread will normally account for almost all the cpu usage. One can introduce some concurrency with the
clustermodule.gireeshpunathil commentedon Jul 25, 2017
@andrasq - thanks. But sorry, your explanation seems orthogonal to my understanding:
which I/O? Are you talking about the primordial (main) thread? I doubt that is classified under v8 thread.
in this discussion and in #14001 we are definitely talking about a number of background threads from v8 module, that is different from the priomordial thread and libuv worker threads. And this discussion focusses on the tradeoff of tenanting the v8 threads with libuv worker threads.
So, my question stands as:
(i) What are those v8 threads which will potentially contend for time slice with more critical work from libuv? Execution tracers? CPU profilers? GC helpers? JIT helpers?
(ii) How do we bring those threads to the forefront to be seen as eating up cycles? (any tunables in the commandline and tunables in the code). This will help us make observations in terms of time-slice distribution variations between the threads and make inferences on throughput, latency as a function of some of these tunables.
(iii) How significant these threads would be in terms of CPU consumption in a fully warmed up production server (no tracing and no new scripts)?
bnoordhuis commentedon Jul 25, 2017
Compiler and GC threads.
perf(1)?
Depends on the application.
gireeshpunathil commentedon Jul 25, 2017
thanks @bnoordhuis - that explains.
In terms of measurements, perf(1) did not help to get thread-wise split up (there is a
-tflag, but behavior is weird) so I was using AIXtprofand the result, as I mentioned earlier, suggests that v8 threads contribute very less (not displayed in the top consumers)I agree this depends on the workload characteristics of the application. Given:
(i) every new page brings in new script (chrome) vs. relatively boot-time-only script (node)
(ii) highly transactional based web workload means most objects collected in the scavenge phase
v8 threads may not consume much slice for either GC or JIT, and is in-line with my tprof observation.
I wish if I could modify the test to manifest the contrasting characteristics and was /am looking for pointers on that line - so that we know the extent to which these threads are insignificant, and what is the tipping point beyond which they show up.
Trott commentedon Aug 10, 2017
This discussion seems to have run its course. I'm going to close, but if you think that's wrong and there's something concrete, feel free to re-open (if GitHub allows) or comment (requesting it be re-opened if you wish) or open another issue as appropriate.
TimothyGu commentedon Aug 10, 2017
FWIW the work for creating a Node.js-specific v8::Platform is being done at #14001. In that PR, it was decided not to merge the libuv and V8 thread pools into one, but instead manage them separately for performance.
davisjam commentedon Sep 3, 2018
And FWIW In #22631 I am working on uniting the V8 and libuv threadpools in Node-land using my pluggable threadpool PR in libuv.