«

I'm looking at this method in the Executors class:

--------------------------------
newFixedThreadPool
public static ExecutorService newFixedThreadPool(int*nThreads)

Creates a thread pool that reuses a fixed set of threads operating off a shared unbounded queue. If any thread terminates due to a failure during execution prior to shutdown, a new one will take its place if needed to executesubsequent tasks.

--------------------------------

How much is too much in terms of me telling it how many threads I want? Isit okay to ask for 100 threads? What about 256?

Thanks.

On Wed, 13 Feb 2013 15:25:33 -0800 (PST), bob smith wrote:

> How much is too much in terms of me telling it how many threads I want? Is it okay to ask for 100 threads? What about 256?

While modern operating systems support thousands of threads, there are
really not many advantages in using more threads than you have CPUs/cores,
and the additional context switching of having more threads than cores
comes with its own performance cost.

Liebe Gruesse,
Joerg

--
Ich lese meine Emails nicht, replies to Email bleiben also leider
ungelesen.

On 2/13/2013 3:25 PM, bob smith wrote:
> How much is too much in terms of me telling it how many threads I
> want? Is it okay to ask for 100 threads? What about 256?

It depends on your system and your application, and you should measure
your system performance (and other metrics) to determine what is best
for your particular case. I'm sorry I don't know much about practical
ways to go about those measurements.

However, 100 to 300 threads is trivial for most modern system. Since
most tasks will be IO-bound, it make sense to have lots of threads so
some can be running while others wait for data to be returned. (Sorry
to contradict Mr. Meier but I'm pretty sure he's off base.) Given a
"typical" application I'd probably use 300 threads to start off and
measure from there.

3000 is probably approaching some sort of upper limit on most "average"
servers. Don't neglect to measure actual running performance, but I
don't think commodity hardware can handle that many practically.

On Wed, 13 Feb 2013 16:06:37 -0800, markspace wrote:

> However, 100 to 300 threads is trivial for most modern system. Since
> most tasks will be IO-bound, it make sense to have lots of threads so
> some can be running while others wait for data to be returned. (Sorry
> to contradict Mr. Meier but I'm pretty sure he's off base.) Given a

Obviously, you can pretty much have Integer.MAX_VALUE threads if all of
them are waiting without negatively impacting your waiting performance -
that's not really a sensible way to look at that question

I said, and stand by, that context switching is not free and is the cost of
having too many threads. Threads that sleep or otherwise wait, such as for
IO, don't typically cause much context switching.

That being said, I would expect the case of 100s of threads waiting for IO
probably an edge case (for use cases such as web servers), and the majority
of cases would have IO restricted to a small number of the threads. YMMV.

> 3000 is probably approaching some sort of upper limit on most "average"
> servers. Don't neglect to measure actual running performance, but I
> don't think commodity hardware can handle that many practically.

We once ran (by accident) something that triggered 100,000 threads (+ GC
and whatever) on a normal Linux server with a quad core CPU and it still
worked (although much slower than with the intended 100). As far as I know,
neither Linux nor the hardware was anything unusual.

Liebe Gruesse,
Joerg

--
Ich lese meine Emails nicht, replies to Email bleiben also leider
ungelesen.

Joerg Meier wrote:
> markspace wrote:
>> However, 100 to 300 threads is trivial for most modern system. Since
>> most tasks will be IO-bound, it make sense to have lots of threads so
>> some can be running while others wait for data to be returned. (Sorry
>> to contradict Mr. Meier but I'm pretty sure he's off base.) Given a
>
> Obviously, you can pretty much have Integer.MAX_VALUE threads if all of
> them are waiting without negatively impacting your waiting performance -
> that's not really a sensible way to look at that question

Perhaps not, but see below.

> I said, and stand by, that context switching is not free and is the cost of
> having too many threads. Threads that sleep or otherwise wait, such as for
> IO, don't typically cause much context switching.

But how "not free" is it? Let's look at your experience in a minute.

> That being said, I would expect the case of 100s of threads waiting for IO
> probably an edge case (for use cases such as web servers), and the majority
> of cases would have IO restricted to a small number of the threads. YMMV.
>
>> 3000 is probably approaching some sort of upper limit on most "average"
>> servers. Don't neglect to measure actual running performance, but I
>> don't think commodity hardware can handle that many practically.
>
> We once ran (by accident) something that triggered 100,000 threads (+ GC
> and whatever) on a normal Linux server with a quad core CPU and it still
> worked (although much slower than with the intended 100). As far as I know,
> neither Linux nor the hardware was anything unusual.

The question is how much slower, and whether 1,000 threads would have experienced
a severe slowdown for that setup. Or 10,000.

Quite often resource limits show up as a discontinuity in the scaling graph. Service time
degrades gracefully up to some n, then catastrophically above that.

In your case there might have been such a knee in that 10^3- 10^4 range. The numbers
would tell.

And that's the point for the OP. You cannot know ahead of time what the right number is,
really. You need to measure, under realistic loads. (And be pretty good at simulating "realistic".)

Notice that answers here are in terms of "if you're I/O bound", "if you're waiting", "if this", "if that".

You need to know which ifs apply to your case.

--
Lew

On Wed, 13 Feb 2013 15:25:33 -0800 (PST), bob smith
<> wrote, quoted or indirectly quoted someone
who said :

>How much is too much in terms of me telling it how many threads I want? Is=
> it okay to ask for 100 threads? What about 256?

The last time I looked, each thread ate up 1MB.

I think the best thing to do is run some benchmarks. It will depend
on how much RAM you have, how many CPU's how much time each thread
spends sleeping. It also depends on how busy the machine is with
other work.

See http://mindprod.com/project/tweakable.html for a proposal to solve
this class of problem.
--
Roedy Green Canadian Mind Products http://mindprod.com
The first 90% of the code accounts for the first 90% of the development time.
The remaining 10% of the code accounts for the other 90% of the development
time.
~ Tom Cargill Ninety-ninety Law

On 02/13/2013 07:25 PM, bob smith wrote:
> I'm looking at this method in the Executors class:
>
> --------------------------------
> newFixedThreadPool
> public static ExecutorService newFixedThreadPool(int nThreads)
>
> Creates a thread pool that reuses a fixed set of threads operating off a shared unbounded queue. If any thread terminates due to a failure during execution prior to shutdown, a new one will take its place if needed to execute subsequent tasks.
>
> --------------------------------
>
> How much is too much in terms of me telling it how many threads I want? Is it okay to ask for 100 threads? What about 256?
>
> Thanks.
>
Just a cautionary, and that's not saying you're ignoring this aspect:
the question "how many" in terms of units of concurrent execution is a
question that should be answered by looking at logic requirements first,
API and platform performance capabilities second.

For example, given N units of work, described by tasks (and ultimately
executed by threads under the control of some Executor, say), the
business logic for one particular use case might require you to process
these tasks 1 by 1, in some order, in another use case the business
logic might dictate that you have to process in groups of M < N, and in
yet another use case the business logic places no _logical_ bound on how
many tasks can be executed at the same time.

Also, consider whether you have a concurrent problem or a parallel
problem. If your application is doing a bunch of things, among them
handling continual requests (user or external system inputs), that's
really concurrency. But if you've got one problem that benefits from
being split up, that's parallelism and you may want to look at ForkJoin
instead.

Task design also is driven by business logic and comes before choosing
things like thread pools and sizes thereof. Task granularity in a
concurrent problem should be whatever logic dictates. An important
consideration here too is synchronous versus asynchronous, and how you
plan to get task results (if any). Runnables vs. Callables, Futures, etc.

Addressing number of threads for pools, a bit, a lot of small tasks can
be worse than a moderate number of medium-sized ones. As others have
said, only performance testing will tell. But don't artificially chop up
your logic in a _concurrent_ problem thinking that you can process a
passel of these things with umpteen threads, because that may backfire.

Overall point being, about the last thing I'd be worrying about is how
many threads to assign to a fixed size thread pool. *Eventually* I'd
think about it, sure. After lots of testing.

AHS

On Thu, 14 Feb 2013 00:46:50 +0100, Joerg Meier <>
wrote, quoted or indirectly quoted someone who said :

>While modern operating systems support thousands of threads, there are
>really not many advantages in using more threads than you have CPUs/cores,

It pays to have more threads than cores if your threads are probing
websites. You want to have enough waiting to keep your port busy. You
can also do that in more efficient but complicated way with a single
thread that processes the next response in turn.

Your advice would apply for CPU intensive tasks.
--
Roedy Green Canadian Mind Products http://mindprod.com
The first 90% of the code accounts for the first 90% of the development time.
The remaining 10% of the code accounts for the other 90% of the development
time.
~ Tom Cargill Ninety-ninety Law

On 2/13/13 3:25 PM, bob smith wrote:
> I'm looking at this method in the Executors class:
>
> --------------------------------
> newFixedThreadPool
> public static ExecutorService newFixedThreadPool(int nThreads)
>
> Creates a thread pool that reuses a fixed set of threads operating off a shared unbounded queue. If any thread terminates due to a failure during execution prior to shutdown, a new one will take its place if needed to execute subsequent tasks.
>
> --------------------------------
>
> How much is too much in terms of me telling it how many threads I want? Is it okay to ask for 100 threads? What about 256?
>
> Thanks.
>
>
As you have read in the rest of this thread, the answer is "it depends".

You can ask for many many threads (thousands). The real question is "how
many threads *should* I for task X?"

Which, of course, depends on the task.

A rule of thumb I use for CPU bound tasks (ray-tracing for example) is
N+1 where N is the number of CPU cores. The +1 is to have an extra
thread available to pick up slack if any one of the other threads ends
up blocked by something, such as synchronization, paging, etc...

For IO bound tasks, I tend not to use a fixed pool, but use a pool that
automatically adjusts its size to meet demand. You also need to be aware
of the effect that you might have down-stream. If your IO is all local
disk access on a spinning media (eg, non SDD hard disk), too many
threads *might* cause undue head shifting, slowing down the overall
operation.

For Memory (or other allocated resource) Intensive tasks, you want to
have an upper bound of threads that prevents any thread from failing due
to running out of resources.

For mixed tasks (some tasks which block on IO for a portion of time,
then process the result in a CPU intensive manor), you'll have to make a
guess and then fine-tune to each machine. Provide sensible defaults but
let the end-user configure it.

As a matter of fact, many applications have configuration options for
thread-pool size. Let the end-user (or administrator at least)
decide/figure out what is best for their particular use-cases,
workflows, and systems.

HTH,
Daniel.

On 14.02.2013 01:46, Joerg Meier wrote:
> On Wed, 13 Feb 2013 16:06:37 -0800, markspace wrote:
>
>> However, 100 to 300 threads is trivial for most modern system. Since
>> most tasks will be IO-bound, it make sense to have lots of threads so
>> some can be running while others wait for data to be returned. (Sorry
>> to contradict Mr. Meier but I'm pretty sure he's off base.) Given a
>
> Obviously, you can pretty much have Integer.MAX_VALUE threads if all of
> them are waiting without negatively impacting your waiting performance -
> that's not really a sensible way to look at that question

Obviously.

> I said, and stand by, that context switching is not free and is the cost of
> having too many threads.

Nobody questioned that.

> Threads that sleep or otherwise wait, such as for
> IO, don't typically cause much context switching.

Yes, but in a pool with a fixed upper limit of threads they consume a
slot while sitting there doing nothing (and causing no CPU cost). In
this case a core will be idle if you limit pool size to number of cores.
If the application is doing IO (or anything else which might cause
intermediate blocking) you are unnecessarily restricting throughput of
the application if you take number of cores (or threads as they are
called in some architectures) as limit. What the best limit is
obviously depends on the nature of the application.

> We once ran (by accident) something that triggered 100,000 threads (+ GC
> and whatever) on a normal Linux server with a quad core CPU and it still
> worked (although much slower than with the intended 100). As far as I know,
> neither Linux nor the hardware was anything unusual.

There you go.

Cheers

robert

--
remember.guy do |as, often| as.you_can - without end
http://blog.rubybestpractices.com/