Velocity Reviews

Velocity Reviews (http://www.velocityreviews.com/forums/index.php)
-   Java (http://www.velocityreviews.com/forums/f30-java.html)
-   -   How to wait for multiple threads to finish (http://www.velocityreviews.com/forums/t136405-how-to-wait-for-multiple-threads-to-finish.html)

Gary J 08-26-2004 04:30 PM

How to wait for multiple threads to finish
 
I have a java program that spawns a lot of threads to process a very large
tree data model. How do I force a "parent" thread to wait until all of its
"child" threads are finished.

Here's a totally contrived example. TreeCounter simply counts the elements
in a tree of Elements:

public class TreeCounter
{
private int count;

void countChildren(Element element)
{
count++;

List children = element.getChildren();
for (Iterator iter = children.iterator(); iter.hasNext();)
{
Element child = iter.next();

new Thread()
{

public void run()
{
countChildren(child)
}

}.start();

}

// wait for all threads created by addChild() to finish before returning

// print out results
System.out.println(count + "elements processed");
}

}

This code was written just for this post so there may be some errors but you
get the idea.

In the above code, each incarnation of countChildren() can create multiple
threads.

How can I ensure that I don't get to the println() statement before all the
threads have finished? I know I can make threads run one at a time by
making a method "synchronized" but here I really want to make sure they all
run at the same time;

Thanks



Frank 08-26-2004 05:10 PM

Re: How to wait for multiple threads to finish
 
Gary J wrote:
> I have a java program that spawns a lot of threads to process a very large
> tree data model. How do I force a "parent" thread to wait until all of its
> "child" threads are finished.
>
> How can I ensure that I don't get to the println() statement before all the
> threads have finished? I know I can make threads run one at a time by
> making a method "synchronized" but here I really want to make sure they all
> run at the same time;
>
> Thanks


Didn't read your example, but here we go;

One way to solve this would be to keep a count of the running threads.
Increase the counter for each thread created and have the threads
decrease it again just before leaving the run() method. Once the counter
hits zero again, the main thread can continue.

A few implementation hints:
1. You need to synchronize access to the counter.
2. Make the counter volatile, it's going to be accessed by multiple threads.
3. To prevent unneccessary polling from the main thread, have it wait()
on the lock object and make the worker threads notify() before leaving
the synchronized method.

-Frank

Paul Lutus 08-26-2004 05:34 PM

Re: How to wait for multiple threads to finish
 
Gary J wrote:

> I have a java program that spawns a lot of threads to process a very large
> tree data model. How do I force a "parent" thread to wait until all of
> its "child" threads are finished.



1. Create and start the threads. Each thread increments a synchronized
global thread counter.

2. Have each thread signal that is is finished by decrementing the global
counter.

3. Put the main process on a timer that periodically tests the counter, and
idles the rest of the time. When the count gets to zero, take the next
action.

--
Paul Lutus
http://www.arachnoid.com


Michael Borgwardt 08-26-2004 05:54 PM

Re: How to wait for multiple threads to finish
 
Gary J wrote:

> I have a java program that spawns a lot of threads to process a very large
> tree data model.


Is it running on a computer with a lot of processors? otherwise, using many
threads is just a waste of CPU time.

Chris Uppal 08-26-2004 06:17 PM

Re: How to wait for multiple threads to finish
 
Gary J wrote:

> I have a java program that spawns a lot of threads to process a very large
> tree data model. How do I force a "parent" thread to wait until all of
> its "child" threads are finished.


Thread.join() is designed for exactly this. Simply put each thread you create
into some sort of Collection (an ArrayList for example) and then use a loop to
call join() on each in turn. If the "joinee" has finished then join() will
return immediately, if not then it will wait until it has finished. By the end
of the loop you can be sure that all of them have finished.

-- chris



Gary J 08-26-2004 06:55 PM

Re: How to wait for multiple threads to finish
 
I thought of using a counter but it seemed too simple a solution. I was a
little unsure what to make volatile. I made counter volatile but its inside
a synchronized method so does it matter?

For those interested Here's what I implemented:

class ThreadLock
{
volatile int counter;

synchronized void add()
{
System.out.println(counter + " threads.");
counter++;
}

synchronized void del()
{
counter--;
System.out.println(counter + " threads.");
notify();
}

synchronized int getCount()
{
return counter;
}
}

in the Main Method I declare:

private ThreadLock threads = new ThreadLock();

and the threads are created like this:

threads.add();
new Thread()
{
public void run()
{
doStuff()
threads.del();
}
}.start();


The main method checks for all the threads to be completed like this:

while (threads.getCount() > 0)
try
{
threads.wait();
} catch(Exception e)
{}
System.out.println("Worker threads complete.");


Is it possible to put this last code inside the ThreadLock class? In other
words:
class ThreadLock
{
....

void resume()
{
while (getCount() > 0)
try
{
wait();
} catch(Exception e)
{}
}
}

then the main nethod would just call:
counter.resume();

thanks




"Frank" <abc@def.ghi> wrote in message news:412e19b6$1@news.broadpark.no...
Gary J wrote:
> I have a java program that spawns a lot of threads to process a very large
> tree data model. How do I force a "parent" thread to wait until all of

its
> "child" threads are finished.
>
> How can I ensure that I don't get to the println() statement before all

the
> threads have finished? I know I can make threads run one at a time by
> making a method "synchronized" but here I really want to make sure they

all
> run at the same time;
>
> Thanks


Didn't read your example, but here we go;

One way to solve this would be to keep a count of the running threads.
Increase the counter for each thread created and have the threads
decrease it again just before leaving the run() method. Once the counter
hits zero again, the main thread can continue.

A few implementation hints:
1. You need to synchronize access to the counter.
2. Make the counter volatile, it's going to be accessed by multiple threads.
3. To prevent unneccessary polling from the main thread, have it wait()
on the lock object and make the worker threads notify() before leaving
the synchronized method.

-Frank



Gary J 08-26-2004 06:58 PM

Re: How to wait for multiple threads to finish
 
In this case it is indeed running on a multi-processor computer. Running
without multiple threads takes about 120% longer than running with them.
The more threads, the faster it runs. There's probably a point of
diminishing returns of course.


"Michael Borgwardt" <brazil@brazils-animeland.de> wrote in message
news:2p6me6FhcvmlU1@uni-berlin.de...
Gary J wrote:

> I have a java program that spawns a lot of threads to process a very large
> tree data model.


Is it running on a computer with a lot of processors? otherwise, using many
threads is just a waste of CPU time.



Gary J 08-26-2004 07:09 PM

Re: How to wait for multiple threads to finish
 
You're absolutely correct.

When I initially saw the join() method I thought my problems were over but I
discovered a possible problem with that approach.

The problem is that in traversing a tree threads are created and finished
the time which means the contents of the Collection would constantly change.
The main method would have to constantly recheck and join to all the threads
in the list on every pass until there were no more threads to join to. At
that point I'm really doing the same thing as checking a counter but with
more work :)

Of course I could be mistaken and haven't tried this one out yet. There's
probably a flaw in my logic.


"Chris Uppal" <chris.uppal@metagnostic.REMOVE-THIS.org> wrote in message
news:qP6dnT-ot_RbsLPcRVn-ig@nildram.net...
Gary J wrote:

> I have a java program that spawns a lot of threads to process a very large
> tree data model. How do I force a "parent" thread to wait until all of
> its "child" threads are finished.


Thread.join() is designed for exactly this. Simply put each thread you
create
into some sort of Collection (an ArrayList for example) and then use a loop
to
call join() on each in turn. If the "joinee" has finished then join() will
return immediately, if not then it will wait until it has finished. By the
end
of the loop you can be sure that all of them have finished.

-- chris




anon 08-26-2004 11:11 PM

Re: How to wait for multiple threads to finish
 
Gary J wrote:
> In this case it is indeed running on a multi-processor computer. Running
> without multiple threads takes about 120% longer than running with them.
> The more threads, the faster it runs. There's probably a point of
> diminishing returns of course.
>

Absolutely. The rule of thumb I've always heard is no more
active/running threads _per_process_ than 2x the number of available CPUs.

YMMV, but I've been more-or-less obeying that limit for years, and it
does seem to hold true for *most* situations.

Babu Kalakrishnan 08-27-2004 07:03 AM

Re: How to wait for multiple threads to finish
 
Michael Borgwardt wrote:
> Gary J wrote:
>
>> I have a java program that spawns a lot of threads to process a very
>> large
>> tree data model.

>
>
> Is it running on a computer with a lot of processors? otherwise, using many
> threads is just a waste of CPU time.


Not necessarily. Consider a case when each thread is waiting for some inputs
that would arrive at some indeterminate time. Processing in parallel instead of
serially would allow the inputs that arrived to be processed while other threads
(probably started first) are still waiting.

BK


All times are GMT. The time now is 09:24 AM.

Powered by vBulletin®. Copyright ©2000 - 2014, vBulletin Solutions, Inc.
SEO by vBSEO ©2010, Crawlability, Inc.