Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Java > Java Runtime.exec problem

Reply
Thread Tools

Java Runtime.exec problem

 
 
suru
Guest
Posts: n/a
 
      07-16-2009
HI,
I am using Runtime.exec to excute my unix (wrapper) commands.
These commands are fired consecutively.
But somtimes i get following exception,
java.io.IOException: Stream closed
at java.io.BufferedInputStream.getBufIfOpen(BufferedI nputStream.java:
145)
at java.io.BufferedInputStream.read(BufferedInputStre am.java:304)
at sun.nio.cs.StreamDecoder$CharsetSD.readBytes(Strea mDecoder.java:
411)
at sun.nio.cs.StreamDecoder$CharsetSD.implRead(Stream Decoder.java:
453)
at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:1 83)
at java.io.InputStreamReader.read(InputStreamReader.j ava:167)
at java.io.BufferedReader.fill(BufferedReader.java:13 6)
at java.io.BufferedReader.readLine(BufferedReader.jav a:299)
at java.io.BufferedReader.readLine(BufferedReader.jav a:362)
at com.vistaar.web.util.StreamGobbler.run(ToolkitAgen t.java:711)

Following is the code snippet:
//method from which exec is invoked
public static String processCommandWithNewLine(String
commandToExecute) {
Runtime systemShell = Runtime.getRuntime();
Process outputProcess = null;
String toolkitOutput = "";
try {
LogHelper.debug(lg, commandToExecute);
outputProcess = systemShell.exec(commandToExecute);

StreamGobbler errorGobbler = new StreamGobbler
(outputProcess.getErrorStream(), true, lg);
StreamGobbler outputGobbler = new StreamGobbler
(outputProcess.getInputStream(), true, lg);
errorGobbler.start();
outputGobbler.start();
outputProcess.waitFor();

toolkitOutput = toolkitOutput + outputGobbler.getOutput();
toolkitOutput = toolkitOutput + errorGobbler.getOutput();
LogHelper.debug(lg, toolkitOutput);
} catch (Throwable ex) {
LogHelper.printErrorTrace(lg, ex);
} finally {
closeProcessStreams(outputProcess);
}
return toolkitOutput.trim();
}

// steam gobbler implmentaion to read output of the command
class StreamGobbler extends Thread {

InputStream is;
String output = "";
boolean insertNewLine;
Logger lg;

StreamGobbler(InputStream is, boolean insertNewLine, Logger lg) {
this.is = is;
this.insertNewLine = insertNewLine;
this.lg = lg;
}

public void run() {
try {
InputStreamReader isr = new InputStreamReader(is);
BufferedReader br = new BufferedReader(isr);
String line = null;
if (insertNewLine) {
while ((line = br.readLine()) != null) {
output = output + line + "\n";
}
} else {
while ((line = br.readLine()) != null) {
output = output + line;
}
}
} catch (IOException ioe) {
LogHelper.printErrorTrace(lg, ioe);
}
}

public String getOutput() {
return output;
}
}


1. i m using thread to read output & error stream simultaneously
2. the above problem doesn't come consistently, but comes frequently

eventhough the program says "stream closed",
when i fire same command from shell it gives me proper output

any idea what is causing the problem?
or der there any better way to read output given by exec process?

thnx,
suraj
 
Reply With Quote
 
 
 
 
Andrew Thompson
Guest
Posts: n/a
 
      07-16-2009
On Jul 16, 5:19*pm, suru <(E-Mail Removed)> wrote:
>..
> Following is the code snippet:


For better chance of getting help, post an SSCCE.
<http://pscode.org/sscce.html>

--
Andrew T.
pscode.org
 
Reply With Quote
 
 
 
 
suru
Guest
Posts: n/a
 
      07-16-2009
On Jul 16, 3:48*pm, Steven Simpson <(E-Mail Removed)> wrote:
> suru wrote:
> > I am using Runtime.exec to excute my unix (wrapper) commands.
> > These commands are fired consecutively.
> > But somtimes i get following exception,
> > java.io.IOException: Stream closed

>
> Okay something closed your stream, but your snippet doesn't show anything
> explicitly doing that. *However...
>
>
>
>
>
> > * * * * * * outputProcess = systemShell.exec(commandToExecute);

>
> > * * * * * * StreamGobbler errorGobbler = new StreamGobbler
> > (outputProcess.getErrorStream(), true, lg);
> > * * * * * * StreamGobbler outputGobbler = new StreamGobbler
> > (outputProcess.getInputStream(), true, lg);
> > * * * * * * errorGobbler.start();
> > * * * * * * outputGobbler.start();
> > * * * * * * outputProcess.waitFor();

>
> > * * * * * * toolkitOutput = toolkitOutput + outputGobbler..getOutput();
> > * * * * * * toolkitOutput = toolkitOutput + errorGobbler.getOutput();
> > * * * * * * LogHelper.debug(lg, toolkitOutput);
> > * * * * } catch (Throwable ex) {
> > * * * * * * LogHelper.printErrorTrace(lg, ex);
> > * * * * } finally {
> > * * * * * * closeProcessStreams(outputProcess);

>
> Does this last line close the streams that your gobblers are gobbling
> on? *If so, they might still be gobbling after the process has finished
> (i.e. after waitFor() returns). *Before closing the streams, try waiting
> for the gobblers to complete (Thread#join()?), or get the gobblers to
> close their respective streams themselves.
>
> --
> ss at comp dot lancs dot ac dot uk- Hide quoted text -
>
> - Show quoted text -



really good opinion
wil try and let u know
 
Reply With Quote
 
suru
Guest
Posts: n/a
 
      07-16-2009
i tried with Thread.join approach,
for some calls it just hangs after join call
 
Reply With Quote
 
Lew
Guest
Posts: n/a
 
      07-16-2009
suru wrote:
> i [sic] tried with Thread.join approach,
> for some calls it just hangs after join call


Did the thread it was joining terminate? It sounds like it didn't.
The join() won't return until the thread it's joining terminates.

Also, join() is a blunt tool.

It's really not possible to fully answer your question without seeing
the code, though.

http://sscce.org/

--
Lew

 
Reply With Quote
 
Lew
Guest
Posts: n/a
 
      07-16-2009
On Jul 16, 10:29*am, Steven Simpson <(E-Mail Removed)> wrote:
> suru wrote:
> > i tried with Thread.join approach,
> > for some calls it just hangs after join call

>
> Where did you put the join()s? *Which one is hanging? *Can you determine
> whether the gobblers have completed? *What does your code look like
> now? *(Take note of others' comments about SSCCEs too.)
>
> From your original post:
>
> > * * * * * * outputProcess.waitFor();

>
> > * * * * * * toolkitOutput = toolkitOutput + outputGobbler..getOutput();
> > * * * * * * toolkitOutput = toolkitOutput + errorGobbler.getOutput();

>
> I think you're going to have to do some sort of waiting (e.g. join())
> between waitFor() and getOutput() anyway, as you need to make sure that
> your gobblers have finished by then. *You could make getOutput() block
> until the stream is finished, instead of using join().
>
> Get the gobblers (instead of anything else) to close the streams too.
> There's no reason for anything else to be dealing with the streams (is
> there?).
>


Also you might consider using an Executor and/or a Future
(FutureTask).

Thread coordination is a fairly advanced topic. After decades of
working with concurrent programming, I still keep researching,
studying and repeating that research and study, and I still feel that
I need to keep doing so.

I recommend the book /Java Concurrency in Practice/ by Brian Goetz, et
al. There are articles by Mr. Goetz available on the IBM
DeveloperWorks site that will help a lot. The Sun tutorials are a
decent starting point.

--
Lew
 
Reply With Quote
 
alexandre_paterson@yahoo.fr
Guest
Posts: n/a
 
      07-17-2009
Hi Suraj,

On Jul 16, 8:19 am, suru <(E-Mail Removed)> wrote:
....
> any idea what is causing the problem?


I've long stopped caring about such problems. Java's
runtime support as always been flakey and tricky (documented
bugs regarding sometimes magically disappearing slave processes
when using the Un*x pipe '|' symbol comes to mind).

Correctly running external processes was already a "Java Pitfall"
ten years ago, and it still is. Your post is a case in point.

The sheer number of posts in here with people having
trouble executing external processes is unusually high.

The number of framework and "helper" classes out there trying
to give a saner abstraction is also madenning (and directly
related to the immediate pain most people encounter when trying
to launch external processes from Java).

I've probably launched more external processes, on more
platforms/OSes/differents versions of these OSes, than
anyone in this group and I can give you the "trick" I'm
using since years.


> or der there any better way to read output given by exec process?


In my opinion yes, and it's trivially simple...

"free your mind"

Seen that correctly "Gobbling" or consuming stdout/stderr
from Java is tricky, I went on with a radical, ultra-simple,
solution : I wrap all my scripts in another wrapper script
that suppresses every stdout/stderr output and redirects them
to temporary files. In other words, I let the OS take care
of the correct gobbling/consuming of the script's output
by redirecting these to temporary files.

Then I simply parse these temporary files.

This is both radical and elegant and it works on Windows, OS X,
Linux, and other Un*x flavors.

The Un*x shell script (Bash in this case but YMMV) may
look like this (out of memory, to give an idea [both stdout
and stderr are redirected here to the same temp file]):

#!/bin/bash
TEMP_REDIR_FILE=$1
shift;
$@ > $TEMP_REDIR_FILE 2>&1

Brutally simple.

The equivalent for the various flavor of Windows is
easy too.

In case you need to specifically output to a non-tty
and be immune to hangups, then 'nohup' is useful
(man nohup and check your shell's nohup doc).

My actual scripts are a tad bit more advanced for in
the case of terminating external processes (most do
terminate, some are 'non-stop') I also append some
"magic" string to know, when parsing my file, that
the command is done, and exited without errors, etc.

I abstracted all this in a clean interface and handle
the result (ie the parsed stdout/stderr text) using a
callback. I implemented timeout facilities, watchdog
process kill -9'ing the watched process should it block
(a correctly implement Un*x OS makes guarantees regarding
kill -9's behavior [on Windows this was more complicated,
I had to resort to pskill.exe]), I also implemented retries,
etc.

It's just an opinion but I find this more advanced than what
you can do in Java (for example regarding the "kill -9"
guarantees [longstanding Java bugs where Process's destroy()
would not really release all the ressources], yadda yadda!) but
yet I find my abstraction much more simple to
use than the 'external processes Java SNAFU' (pardon, I meant
than the Java API dealing with external processes

I wrote this several years ago and it never failed on me,

Alex


--
"You see things; and you say, 'Why?
"But I dream things that never were; and I say, 'Why not?'"
 
Reply With Quote
 
suru
Guest
Posts: n/a
 
      07-21-2009
really nice opinion Alex,

i hv few queries for understanding,

1. do u know why exactly java hangs up? is der any posts/link
explaining the details?
Also, i have observed that sometimes the program hangs at
"br.readLine()" call

2. in ur suggestion, u have mentioned that, u redirect ur process
output to a temporary file,
is der simple way to keep temporary file name unique? since multiple
users may execute same commands (from java) hence output for each user
should be in different files.

thnx,
suraj
 
Reply With Quote
 
Andrew Thompson
Guest
Posts: n/a
 
      07-21-2009
On Jul 21, 4:40*pm, suru <(E-Mail Removed)> wrote:
> really nice opinion Alex,
>
> i hv .. u ... der .. ur ...


Do you expect anyone to read that rubbish?

How about you show your appreciation for the
help you've received by showing the extra effort
needed to type words properly (and in the process,
you might save yourself from seeming like a lazy
bozo).

--
Andrew T.
 
Reply With Quote
 
suru
Guest
Posts: n/a
 
      07-21-2009
apologies for using short words (got habit due to chat)


Following is the proper version,

Really nice opinion Alex,

I have few queries for understanding,


1. Do you know why exactly java hangs up? is there any posts/link
explaining the details?
Also, i have observed that sometimes the program hangs at
"br.readLine()" call


2. In your suggestion, you have mentioned that, you redirect your
process
output to a temporary file,
is there simple way to keep temporary file name unique? since
multiple
users may execute same commands (from java) hence output for each
user
should be in different files.


Thanks,
Suraj

 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Hot Requirements: 1.Sr Java Developer,2.Java Developer (Java with EJB) Isaac Java 0 01-20-2011 08:41 PM
hey i am just started java,, can anyone tell me the use ,application, why java , importance of java.. manish sahu Java 3 02-14-2008 12:00 AM
[JAVA] [EVALUATION] - The Java Failure (Sorry: The Java(tm) Failure) Ilias Lazaridis Java 0 02-01-2005 10:32 AM
Job to convert Java App 1.3.1 to Java Newest of Java Michael Kintner Java 0 11-30-2003 04:42 AM
my problem with Java Plugin 1.4.1_02 & java.awt.TextArea & Greek char Thang Nguyen Java 0 08-07-2003 09:02 AM



Advertisments