Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Java > Process.waitFor() returns, but threads reading output/error streams are blocked on read

Reply
Thread Tools

Process.waitFor() returns, but threads reading output/error streams are blocked on read

 
 
Michael Grove
Guest
Posts: n/a
 
      08-18-2005
java gurus,

i'm seeing some odd behavior running a process via Runtime.exec on redhat
linux (seen it on RHEL AS 2.1, 3, 4) with 1.4 and 1.5 JVMs (the only ones
i've tried). if i start a specific process via Runtime.exec and set up
threads to read in the out/err streams, i find that after Process.waitFor()
returns (~5 seconds) the threads are still stuck in InputStream.read(). i
would have expected them to return -1 (or fail with IOExceptions) almost
immediately after process termination. i've tried closing the streams from
the main thread and destroying the process, but that was not enough to break
the threads out of read. i found that if i changed my reader threads to
poll InputStream.available() they will find the streams closed after the
main thread closes them, although available() will return 0 indefinitely if
the main thread fails to close the streams manually.

i'll include the java source below, but the problem seems extremely specific
to the program i'm running in Runtime.exec(). i've been unable to duplicate
the problem with another program. the program is a non-interactive command
line tool (i'll call "cmd") that creates a socket connection to another
process on the same machine and requests an action to be performed (in my
case the action is "-start", passed in on the command line). the program
takes about 5 seconds to complete. upon successful completion no data is
written to standard out/err. the exit code is 0 on success, non-zero on
failure. i can confirm in the process list that the program does terminate
(which Process.waitFor() tells me anyway), and i can also confirm the
program performed the action i expected. i discovered that when my java
process is hung if i terminate the process the command line tool connects to
that will break the threads out of read() with a -1 response. that leads me
to believe that something was not cleaned up when the command line tool
exited. i've also found that if i invoke the command line tool
differently - e.g. i run "cmd -start" after a previous run of "cmd -start"
(close to a no-op) or run "cmd -status" - the java process terminates
normally. so something about the first call to "cmd -start" preventes
InputStream.read() from returning.

some questions:
* after Process.waitFor() returns, what could cause
Process.getInputStream().read() and Process.getErrorStream().read() to block
indefinitely? shouldn't they return -1 or receive IOExceptions almost
immediately?
* any thoughts on what this command line tool might be doing (perhaps not
closing its out/err streams gracefully when it terminates?) that would cause
these reads to block?

thanks.

-mike

import java.io.IOException;

public class cmdstart {
public static void main(String [] args) throws IOException,
InterruptedException {
String cmd = "cmd -start";
Process process = Runtime.getRuntime().exec(cmd);
MyReader outReader = new MyReader(process.getInputStream());
Thread outThread = new Thread(outReader);
outThread.start();
MyReader errReader = new MyReader(process.getErrorStream());
Thread errThread = new Thread(errReader);
errThread.start();
System.out.println("exit code is " + process.waitFor());
//process.destroy();
// process.getInputStream().close();
// process.getErrorStream().close();
outThread.join();
errThread.join();
System.out.println("joins completed");
}
}

import java.io.*;

public class MyReader implements Runnable {
private final InputStream is;

public MyReader(InputStream is) {
this.is = is;
}

public void run() {
// normally would read in while here, but these threads never get any
data -
// they are blocked on the first read() call forever.
int length;
try {
length = is.read();
if (length == -1) {
System.out.println("-1");
return;
}
if (length == 0) {
System.out.println("0");
return;
}
System.out.println("read something");
} catch (Throwable t) {
System.out.println(t);
}
}
}



 
Reply With Quote
 
 
 
 
Thomas Hawtin
Guest
Posts: n/a
 
      08-18-2005
Michael Grove wrote:
>
> * after Process.waitFor() returns, what could cause
> Process.getInputStream().read() and Process.getErrorStream().read() to block
> indefinitely? shouldn't they return -1 or receive IOExceptions almost
> immediately?
> * any thoughts on what this command line tool might be doing (perhaps not
> closing its out/err streams gracefully when it terminates?) that would cause
> these reads to block?


As a random guess forking (for instance executing another command)
and/or reopening stdout/stderr.

Apparently strace is the Linux command to use to see what a process is
doing. Never tried it myself.

Tom Hawtin
--
Unemployed English Java programmer
http://jroller.com/page/tackline/
 
Reply With Quote
 
 
 
 
Michael Grove
Guest
Posts: n/a
 
      08-19-2005
thanks for the response.

you're right that something like fork occurs. i haven't fully tracked it
down via strace, but i remembered that the first time "cmd -start" is run it
actually bootstraps the other process it normally connects to if it finds it
not running. that process ends up started with init (pid 1) as it's parent
process. i didn't see exactly how this happened via strace, but it seemed
to occur right around a clone() call, which i see is pretty similar to fork.
i think the cloned process might actually be the one that bootstraps the
other process before it terminates. the cloned process vanishes before i
can get a chance to see what it's doing.

that should at least help me duplicate this problem with a "cmd" program of
my own creation. however i still wonder - is it correct behavior in java
that after the process has terminated the streams are not closed? even if
that answer is yes, i'm still confused why a polling InputStream.available()
would fail with an IOException once i closed the Process streams manually
yet InputStream.read() would remain blocked.

i know Runtime.exec() is a bit of a crapshoot and the javadocs indicate it
may not work well in a variety of cases. wondering though if this behavior
is 1) expected/correct or 2) a Runtime.exec() limitation or 3) a JVM bug.

thanks.

-mike

"Thomas Hawtin" <(E-Mail Removed)> wrote in message
news:43051c2c$0$97138$(E-Mail Removed)...
> Michael Grove wrote:
> >
>> * after Process.waitFor() returns, what could cause
>> Process.getInputStream().read() and Process.getErrorStream().read() to
>> block indefinitely? shouldn't they return -1 or receive IOExceptions
>> almost immediately?
>> * any thoughts on what this command line tool might be doing (perhaps not
>> closing its out/err streams gracefully when it terminates?) that would
>> cause these reads to block?

>
> As a random guess forking (for instance executing another command) and/or
> reopening stdout/stderr.
>
> Apparently strace is the Linux command to use to see what a process is
> doing. Never tried it myself.
>
> Tom Hawtin
> --
> Unemployed English Java programmer
> http://jroller.com/page/tackline/



 
Reply With Quote
 
John C. Bollinger
Guest
Posts: n/a
 
      08-20-2005
Michael Grove wrote:
> you're right that something like fork occurs. i haven't fully tracked it
> down via strace, but i remembered that the first time "cmd -start" is run it
> actually bootstraps the other process it normally connects to if it finds it
> not running. that process ends up started with init (pid 1) as it's parent
> process. i didn't see exactly how this happened via strace, but it seemed
> to occur right around a clone() call, which i see is pretty similar to fork.
> i think the cloned process might actually be the one that bootstraps the
> other process before it terminates. the cloned process vanishes before i
> can get a chance to see what it's doing.
>
> that should at least help me duplicate this problem with a "cmd" program of
> my own creation. however i still wonder - is it correct behavior in java
> that after the process has terminated the streams are not closed? even if
> that answer is yes, i'm still confused why a polling InputStream.available()
> would fail with an IOException once i closed the Process streams manually
> yet InputStream.read() would remain blocked.
>
> i know Runtime.exec() is a bit of a crapshoot and the javadocs indicate it
> may not work well in a variety of cases. wondering though if this behavior
> is 1) expected/correct or 2) a Runtime.exec() limitation or 3) a JVM bug.


My guess would be that the observed behavior is more-or-less correct, in
that the OS-level streams are not being closed, with the result that the
Java-level streams are not closed either. This could be the case if the
Process' stdin and stdout are inherited by the chain of other processes
that are forked off, and not closed by those processes I suspect that's
what is happening. If so, it's a minor bug in the bootstrapper,
possibly duplicated in the other process, and it can probably be
addressed in one or both of those places.

--
John Bollinger
http://www.velocityreviews.com/forums/(E-Mail Removed)
 
Reply With Quote
 
Roedy Green
Guest
Posts: n/a
 
      08-22-2005
On Sat, 20 Aug 2005 22:44:46 GMT, Roedy Green
<(E-Mail Removed)> wrote or quoted :

>
>>that should at least help me duplicate this problem with a "cmd" program of
>>my own creation. however i still wonder - is it correct behavior in java
>>that after the process has terminated the streams are not closed? even if
>>that answer is yes, i'm still confused why a polling InputStream.available()
>>would fail with an IOException once i closed the Process streams manually
>>yet InputStream.read() would remain blocked.

>
>try looking at java.util.prefs instead.


I meant java.lang.ProcessBuilder. I was thinking of somenone else's
question, and came back to this.


 
Reply With Quote
 
Michael Grove
Guest
Posts: n/a
 
      08-23-2005
thanks for clarifying - i couldn't imagine what preferences you had in mind
. program has to run on 1.4, so ProcessBuilder is not available.

i've asked the authors of the "cmd" program to make certain streams are
properly being closed, and will post any solution to the newsgroup.

"Roedy Green" <(E-Mail Removed)> wrote in message
news:(E-Mail Removed)...
> On Sat, 20 Aug 2005 22:44:46 GMT, Roedy Green
> <(E-Mail Removed)> wrote or quoted :
>
>>
>>>that should at least help me duplicate this problem with a "cmd" program
>>>of
>>>my own creation. however i still wonder - is it correct behavior in java
>>>that after the process has terminated the streams are not closed? even
>>>if
>>>that answer is yes, i'm still confused why a polling
>>>InputStream.available()
>>>would fail with an IOException once i closed the Process streams manually
>>>yet InputStream.read() would remain blocked.

>>
>>try looking at java.util.prefs instead.

>
> I meant java.lang.ProcessBuilder. I was thinking of somenone else's
> question, and came back to this.
>
>



 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Site to open the blocked sites and blocked and encoded alagmy2030 Javascript 0 02-11-2011 11:54 PM
java calling rsync calling ssh: io threads blocked pvbemmel-at-xs4all-nl Java 7 10-29-2010 08:38 AM
using streams for communication between threads...? stefan.oedenkoven@gmx.de C++ 5 01-20-2006 01:22 PM
Threads over same socket/streams instances. udupi_mail@yahoo.com Java 2 02-21-2005 06:59 PM
streams and threads... why? Frank Java 2 01-29-2004 01:23 AM



Advertisments