Runtime.exec(String[]) Doesn't Always Work, bBut Runtime.exec(String) Does
 

I've included the method I'm having trouble with at the end of the post.

I was using Runtime.getRuntime().exec(String) to execute programs and had no
problem with it at all, but since I had to include an argument, I was
worried about problems with spaces in program names, so I switched
to .exec(String[]) instead of .exec(String). I figure this way when it
turns out I have to run a program with a space in the filename, there
should be no problem.

When I was specifying the program with only a String, every program I would
attempt would run. Now that I'm using String[], I have times where I run
one program and it works, then run the same program, with the same
arguments, a few seconds later, and waitFor() returns immediately and I get
no data from the ErrorStream or InputStream.

I tried, as an experiment, to put quotation marks around the filename in a
string so it would look like this:

"/usr/bin/myprogram" -myargument

And that would not work -- the programs would not run at all.

Also as an experiment, in the spot I marked with a comment, I tried a
sleep() call, just in case waiting had any effect. When I had it wait 100
milliseconds, every time I ran a program, I got data from the appropriate
InputStream or ErrorStream, but not all the data I should have (it was
lopped off at the beginning). I tried shorter times for sleep() and with
10 milliseconds, no programs ran.

So why is it that sometimes the program runs and I get input and sometimes
it doesn't? My best guess is that the program always runs, but that there
might be a timing issue that keeps me from reading the streams quickly
enough, but that doesn't sound likely.

Any help is appreciated.

Thanks!

Hal

------Code method-----
public boolean runFile()
{
stopFlag = false;
System.out.println("DEBUG: Starting to run file, Command: " + runLine);
try
{
Process procRun = Runtime.getRuntime().exec(cmdInfo);
iRead = new BufferedReader (new
InputStreamReader(procRun.getInputStream()));
eRead = new BufferedReader (new
InputStreamReader(procRun.getErrorStream()));
System.out.println("\tDEBUG: Buffers made");
//Tried to use Thread.sleep() here with different timings
//Thread to read Standard output
//Only exception thrown by BufferedReader.readLine() is EOF, which
//means we're at the end, so there's no need to do anything with it.
new Thread(
new Runnable() {
public void run() {
String sRead;
try {
while ((sRead = iRead.readLine()) != null) {
System.out.println("DEBUG: Standard:" + sRead);
outData = outData + sRead;
if (stopFlag)
break;
}
} catch (Exception eofE) {}
}
}
).start();

//Thread to read Error output
new Thread(
new Runnable() {
public void run() {
String sRead;
try {
while ((sRead = eRead.readLine()) != null) {
System.out.println("DEBUG: Error:" + sRead);
outError = outError + sRead;
if (stopFlag)
break;
}
} catch (Exception eofE) {}
}
}
).start();

procRun.waitFor();
stopFlag = true;
System.out.println("\tDEBUG: Run is complete");
iStream.close();
eStream.close();
}
catch (Exception e)
{
System.err.println("Could not execute IDFile: " + runLine);
return false;
}
System.out.println("\tDEBUG: Done running file");
return true;
}

Re: Runtime.exec(String[]) Doesn't Always Work, bBut Runtime.exec(String)Does
 

Hal Vaughan wrote:
> Also as an experiment, in the spot I marked with a comment, I tried a
> sleep() call, just in case waiting had any effect. When I had it wait 100
> milliseconds, every time I ran a program, I got data from the appropriate
> InputStream or ErrorStream, but not all the data I should have (it was
> lopped off at the beginning). I tried shorter times for sleep() and with
> 10 milliseconds, no programs ran.

The start of the data missing? That's very odd. It should drop anything
from the streams.

> procRun.waitFor();
> stopFlag = true;

You are instructing you child threads to stop before they have
necessarily read all of the data. (I assume stopFlag is volatile.) They
wont read the flag if there is no data to read and the streams are not
closed. A better idea would be to jsut let them complete normally.

> System.out.println("\tDEBUG: Run is complete");
> iStream.close();
> eStream.close();

Now you are attempting to close the streams, before they are necessarily
read.

Tom Hawtin
--
Unemployed English Java programmer
http://jroller.com/page/tackline/

Re: Runtime.exec(String[]) Doesn't Always Work, bBut Runtime.exec(String) Does
 

Thomas Hawtin wrote:

> Hal Vaughan wrote:
>> Also as an experiment, in the spot I marked with a comment, I tried a
>> sleep() call, just in case waiting had any effect. When I had it wait
>> 100 milliseconds, every time I ran a program, I got data from the
>> appropriate InputStream or ErrorStream, but not all the data I should
>> have (it was
>> lopped off at the beginning). I tried shorter times for sleep() and with
>> 10 milliseconds, no programs ran.
>
> The start of the data missing? That's very odd. It should drop anything
> from the streams.

I tried again, with a sleep() call. I may have gotten some output files
mixed up, but it DID lose data from the streams. It may not have been from
the beginning, but I'm pretty sure it was.

>> procRun.waitFor();
>> stopFlag = true;
>
> You are instructing you child threads to stop before they have
> necessarily read all of the data. (I assume stopFlag is volatile.) They
> wont read the flag if there is no data to read and the streams are not
> closed. A better idea would be to jsut let them complete normally.

So would I be better off just reading them continually until they throw an
EOFException? It seems that would be the best if that is the case.

> > System.out.println("\tDEBUG: Run is complete");
> > iStream.close();
> > eStream.close();
>
> Now you are attempting to close the streams, before they are necessarily
> read.

Is that so even if I'm closing them AFTER I'm using waitFor()?

Thanks!

Hal

Re: Runtime.exec(String[]) Doesn't Always Work, bBut Runtime.exec(String) Does
 

Thomas Hawtin wrote:

> Hal Vaughan wrote:
>> Also as an experiment, in the spot I marked with a comment, I tried a
>> sleep() call, just in case waiting had any effect. *When I had it wait
>> 100 milliseconds, every time I ran a program, I got data from the
>> appropriate InputStream or ErrorStream, but not all the data I should
>> have (it was
>> lopped off at the beginning). *I tried shorter times for sleep() and with
>> 10 milliseconds, no programs ran.
>
> The start of the data missing? That's very odd. It should drop anything
> from the streams.
>
>> * * * * * * * * procRun.waitFor();
>> * * * * * * * * stopFlag = true;
>
> You are instructing you child threads to stop before they have
> necessarily read all of the data. (I assume stopFlag is volatile.) They
> wont read the flag if there is no data to read and the streams are not
> closed. A better idea would be to jsut let them complete normally.
>
> *> * * * * * * * * System.out.println("\tDEBUG: Run is complete");
> *> * * * * * * * * iStream.close();
> *> * * * * * * * * eStream.close();
>
> Now you are attempting to close the streams, before they are necessarily
> read.
>
> Tom Hawtin

I made changes based on these suggestions and it works now,with only one
issue that I fixed with a hack I'm not thrilled with.

The streams are not closed at the end of the routine, but within the threads
that read from the streams. *When reading from a stream throws an error
(and the only documented error they throw is EOF), the stream is closed.

I still use waitFor() at the end to determine when the program finishes
running, but after that, I wait 100 milliseconds, to allow any extra data
to be pulled from the streams. *Then I return. *That's the hack I'm worried
about. *I can't wait too long, since I'm going through a number of files,
but without waiting, I sometimes don't get any data from the streams. *I'd
think there should be a better way to make sure all the output from the
program has been received, but when the streams are done (i.e. all the
program output has completed), they don't always throw an EOF.

I tried setting flags when each stream threw an error and waiting until both
streams had thrown an EOFException, but when I did that, it would freeze
and at least one of the streams never closed. *If there is another way to
handle this, other than waiting for some time to allow the streams to
receive all the data, I'd really like to hear about it.

Thanks for any help and suggestions.

Hal

Re: Runtime.exec(String[]) Doesn't Always Work, bBut Runtime.exec(String) Does
 

"Hal Vaughan" <hal@thresholddigital.com> wrote in message
news:rKGdnR0na7fkV-3ZnZ2dnUVZ_v6dnZ2d@comcast.com...
> Thomas Hawtin wrote:
>
>> Hal Vaughan wrote:
>>> Also as an experiment, in the spot I marked with a comment, I tried a
>>> sleep() call, just in case waiting had any effect. When I had it wait
>>> 100 milliseconds, every time I ran a program, I got data from the
>>> appropriate InputStream or ErrorStream, but not all the data I should
>>> have (it was
>>> lopped off at the beginning). I tried shorter times for sleep() and
>>> with
>>> 10 milliseconds, no programs ran.
>>
>> The start of the data missing? That's very odd. It should drop anything
>> from the streams.
>
> I tried again, with a sleep() call. I may have gotten some output files
> mixed up, but it DID lose data from the streams. It may not have been
> from
> the beginning, but I'm pretty sure it was.
>
>>> procRun.waitFor();
>>> stopFlag = true;
>>
>> You are instructing you child threads to stop before they have
>> necessarily read all of the data. (I assume stopFlag is volatile.) They
>> wont read the flag if there is no data to read and the streams are not
>> closed. A better idea would be to jsut let them complete normally.
>
> So would I be better off just reading them continually until they throw an
> EOFException? It seems that would be the best if that is the case.
>
>> > System.out.println("\tDEBUG: Run is complete");
>> > iStream.close();
>> > eStream.close();
>>
>> Now you are attempting to close the streams, before they are necessarily
>> read.
>
> Is that so even if I'm closing them AFTER I'm using waitFor()?
>
> Thanks!

I found this article quite helpful when I was messing with Runtime.exec() a
couple of years back. It may help you sort out your current problem or at
least understand better how it works:
http://www.javaworld.com/javaworld/j...9-traps_p.html.

--
Rhino

Re: Runtime.exec(String[]) Doesn't Always Work, bBut Runtime.exec(String) Does
 

On Sun, 21 May 2006 15:49:27 -0400, Hal Vaughan wrote:
> When I was specifying the program with only a String, every program
> I would attempt would run. Now that I'm using String[], I have times
> where I run one program and it works, then run the same program,
> with the same arguments, a few seconds later, and waitFor() returns
> immediately and I get no data from the ErrorStream or InputStream.
>
> I tried, as an experiment, to put quotation marks around the
> filename in a string so it would look like this:
>
> "/usr/bin/myprogram" -myargument
>
> And that would not work -- the programs would not run at all.

exec(String) simply tokenizes the command and invokes exec(String[]).
So if exec(String) works but exec(String[]) doesn't, any problems with
the latter are most likely due to your tokenization of the command
line.

exec(String[]) is, as you correctly observe, the right choice when
your command contains *significant* whitespace, because the
StringTokenizer used by exec(String) is not particularly smart.

The rule is simple: each single component in the command line, i.e.
the command name, an option flag, an argument, etc, is one String in
the command array.

Don't use any quotes or escape characters, because there is no shell
to further interpret the command line. Any such characters will be
treated literally by the system, so unless your program really is
spelled with those characters it won't be found, or the arguments will
be meaningless, etc.

> So why is it that sometimes the program runs and I get input and
> sometimes it doesn't? My best guess is that the program always runs,
> but that there might be a timing issue that keeps me from reading
> the streams quickly enough, but that doesn't sound likely.

There should not be any timing issues. Data written to either of the
streams stays there until you read it. And if you don't read it, the
child will eventually hang waiting for you to do so, as you've already
discovered. And you won't get EOF until you've emptied the streams and
the child has exited (or closed the streams).

I don't see BufferedReader.ready() or InputStream.available() in your
code, but use of those could give the results you describe (avoid
them).

However if waitFor() returns, that is a string indication that your
program has exited. What is returned by Process.exitValue()? What does
your command line look like?

/gordon

--
[ do not email me copies of your followups ]
g o r d o n + n e w s @ b a l d e r 1 3 . s e

Re: Runtime.exec(String[]) Doesn't Always Work, bBut Runtime.exec(String) Does
 

On Sun, 21 May 2006 18:26:10 -0400, Hal Vaughan wrote:
> I still use waitFor() at the end to determine when the program
> finishes running, but after that, I wait 100 milliseconds, to allow
> any extra data to be pulled from the streams. *Then I return.
> *That's the hack I'm worried about.

Simply read until readLine() returns null. That's BufferedReader's way
of indicating EOF.

/gordon

--
[ do not email me copies of your followups ]
g o r d o n + n e w s @ b a l d e r 1 3 . s e

Re: Runtime.exec(String[]) Doesn't Always Work, bBut Runtime.exec(String)Does
 

Hal Vaughan wrote:
>
> The streams are not closed at the end of the routine, but within the threads
> that read from the streams. When reading from a stream throws an error
> (and the only documented error they throw is EOF), the stream is closed.

Rather than trying to sleep, I'd suggest using Thread.join. You actually
only need one additional thread, as you are going to wait for the
process anyway.

I don't see any documentation that states that other types of
IOException are not thrown.

BTW: If your process returns many lines, the Java code will become
increasingly slow. Repeated string concatenation is bad - use
StringBuilder (or StringBuffer).

Tom Hawtin
--
Unemployed English Java programmer
http://jroller.com/page/tackline/

Re: Runtime.exec(String[]) Doesn't Always Work, bBut Runtime.exec(String) Does
 

Gordon Beaton wrote:

> On Sun, 21 May 2006 18:26:10 -0400, Hal Vaughan wrote:
>> I still use waitFor() at the end to determine when the program
>> finishes running, but after that, I wait 100 milliseconds, to allow
>> any extra data to be pulled from the streams. *Then I return.
>> That's the hack I'm worried about.
>
> Simply read until readLine() returns null. That's BufferedReader's way
> of indicating EOF.
>
> /gordon

If the process is still running but hasn't printed anything to output for a
while, what does readLine() return? An empty string instead of a null?

Thanks for the pointer. I'll add that to the routine.

An extra note: I did searches and found Sun's "official" method for reading
from a process and it does not take a lot of what has come up in this
thread into account.

Hal

Re: Runtime.exec(String[]) Doesn't Always Work, bBut Runtime.exec(String) Does
 

Thomas Hawtin wrote:

> Hal Vaughan wrote:
>>
>> The streams are not closed at the end of the routine, but within the
>> threads
>> that read from the streams. When reading from a stream throws an error
>> (and the only documented error they throw is EOF), the stream is closed.
>
> Rather than trying to sleep, I'd suggest using Thread.join. You actually
> only need one additional thread, as you are going to wait for the
> process anyway.

I had never thought of that -- I'm not familiar with it and the API docs
don't go into detail, so I'll read up on it.

> I don't see any documentation that states that other types of
> IOException are not thrown.

Thanks. That confirms what I could find..

> BTW: If your process returns many lines, the Java code will become
> increasingly slow. Repeated string concatenation is bad - use
> StringBuilder (or StringBuffer).
>
> Tom Hawtin

I'm not too worried. Basically, this is part of a file finder. It looks
for files to identify them. Then it either reads the file in to find
identifying text, or runs it to check the output. That means usually the
program is run with something like "--help" or "-version" or something
similar and all I'm doing is reading in the output to compare it against a
version string.

Hal