Velocity Reviews

Velocity Reviews (http://www.velocityreviews.com/forums/index.php)
-   Java (http://www.velocityreviews.com/forums/f30-java.html)
-   -   The program that would'nt Quit (http://www.velocityreviews.com/forums/t149670-the-program-that-wouldnt-quit.html)

Roedy Green 01-26-2006 02:34 AM

The program that would'nt Quit
 
I have I think discovered a strange bug in both JDK 1.5 and 1.6.

the program ends like this:

System.out.println( "done" );
// System.exit( 0 );
} // end main

Without that System.exit, the program keeps on running endlessly using
about 94% of my CPU time after printing "done".

The odd thing is this is not a gui and it has but a single thread. I
have no finalizers.

It works fine in Jet.

I don't know where to begin to track this down or how to reproduce it
for Sun. The jar is 271K and the data files it works on are 14 MB.

Any ideas on what could cause such behaviour?

--
Canadian Mind Products, Roedy Green.
http://mindprod.com Java custom programming, consulting and coaching.

Roedy Green 01-26-2006 02:57 AM

Re: The program that would'nt Quit
 
On Thu, 26 Jan 2006 02:34:42 GMT, Roedy Green
<my_email_is_posted_on_my_website@munged.invalid > wrote, quoted or
indirectly quoted someone who said :

>the program ends like this:
>
> System.out.println( "done" );
> // System.exit( 0 );
> } // end main


there in the class containing main. I don't see anything out of the
ordinary about it.

package com.mindprod.htmlmacros;
import java.io.File;
import java.io.FilenameFilter;
import java.util.ArrayList;

import com.mindprod.filter.ExtensionListFilter;

/**
* Feeds the entire Mindprod website to the Replacer macro mechanism
refreshing
* expansions of all macros. Expands HTML boilerplate refresher macros
e.g. <!--
* macro Include dogs.txt --> <!-- generated --> <img
* src="images/animals/dog.jpg" height="410" width="511" alt="dog">
<!--
* /generated by Include --> Allows arbitrary expander macro code.
Futures:
* Error messages give offset/line number, nearby text check for
generated by
* macro not just generated to remove old text. Can embed a " in a
parm by
* encoding it as &quot;
*/
public class ReplaceMindprod extends Replacer {

/**
* True if want debugging output, counts of files gathered from
different
* dirs.
*/
private static final boolean DEBUGGING = false;

/**
* total count of files gathered
*/
private static int gatherCount;

/**
* arrayList where we build list of files to process.
*/
static ArrayList<String> filesToProcess;

/**
* Estimated files will process.
*/
public static final int ESTIMATED_TOTAL_FILES = 4500;

/**
* Collect all files in the given dirName for processing.
*
* @param dirName
* Name of fully qualified directory where to look for the
html
* files.
* @param filter
* filter to use to decide while files in that dir to
process.
*/
static void gather ( String dirName, FilenameFilter filter )
{
String[] filenames = new File( dirName ).list( filter );
if ( filenames == null )
{
throw new IllegalArgumentException(
"com.mindprod.htmlmacros.ReplaceMindprod: No such dir:
"
+ dirName );
}
gatherCount += filenames.length;
if ( DEBUGGING )
{
System.out.println( filenames.length
+ " files gathered from "
+ dirName );
}
for ( String filename : filenames )
{
filesToProcess.add( dirName + "/" + filename );
}
}

/**
* Get fully qualified name of all the files to process
*/
public static void gatherFiles ()
{

// add *.html files in mindprod

filesToProcess = new ArrayList<String>( ESTIMATED_TOTAL_FILES
);
FilenameFilter f = new ExtensionListFilter( new String[] {
"html"
} );

gather( "e:/mindprod", f );
gather( "e:/mindprod/animalrights", f );
gather( "e:/mindprod/applets", f );
gather( "e:/mindprod/applications", f );
gather( "e:/mindprod/bgloss", f );
// bookcovers
gather( "e:/mindprod/contact", f );
gather( "e:/mindprod/deepthoughts", f );
// dvdcovers
gather( "e:/mindprod/environment", f );
gather( "e:/mindprod/esperanto", f );
gather( "e:/mindprod/ethics", f );
gather( "e:/mindprod/feedback", f );
gather( "e:/mindprod/ggloss", f );
gather( "e:/mindprod/humanrights", f );
// icon
// images
// include won't work since macros in wrong dir
gather( "e:/mindprod/jgloss", f );
// jgloss/include
// kjv, rarely changes, do manually when it needs it.
gather( "e:/mindprod/livinglove", f );
gather( "e:/mindprod/livinglove/methods", f );
gather( "e:/mindprod/money", f );
// pads
gather( "e:/mindprod/phone", f );
gather( "e:/mindprod/politics", f );
gather( "e:/mindprod/politics/laser", f );
// precis
gather( "e:/mindprod/projects", f );
gather( "e:/mindprod/quadra", f );

// redir, only makes sense to expand them in place. Don't
disturb copied
// versions
gather( "e:/mindprod/redir", f );
gather( "e:/mindprod/religion", f );
// renney
// replicator
gather( "e:/mindprod/reunion", f );
// screenshots
// snippets
// sound
gather( "e:/mindprod/utilities", f );
gather( "e:/mindprod/webstarts", f );
// zips

// all files are ready to go.

}

/**
* Expands macros written in Java in HTML files files.
*
* @param args
* names of files to process, no wildcards.
*/
public static void main ( String[] args )
{
System.out.println( "\nExpanding macros on entire Mindprod
website" );

Tools.setWebRoot( "e:\\mindprod" );
gatherFiles();

System.out.println( gatherCount
+ " files gathered for mindprod site macro expansion." );

assert gatherCount <= ESTIMATED_TOTAL_FILES;
for ( String fileToProcess : filesToProcess )
{
assert fileToProcess != null;
assert new File( fileToProcess ).exists() : "file to
process missing";
try
{
expandMacrosInFile( new File( fileToProcess ), true /*
quiet */);
}
catch ( Exception e )
{
System.err.println( e.getMessage()
+ " in file "
+ fileToProcess );
System.err.println();
e.printStackTrace();
// we don't exit. We keep on processing files.
}
} // end for each file loop
System.out.println( "done" );
// don't remove the System.exit( 0 ). It bypasses a Sun bug.
System.exit( 0 );
} // end main

} // end ReplaceMindProd

--
Canadian Mind Products, Roedy Green.
http://mindprod.com Java custom programming, consulting and coaching.

Stefan Ram 01-26-2006 03:12 AM

Re: The program that would'nt Quit
 
You could use Thread.activeCount() to confirm that there is
only one thread at the end.

>Tools.setWebRoot( "e:\\mindprod" ); (...)
>gatherFiles(); (...)
>for ( String fileToProcess : filesToProcess ) (...)
>expandMacrosInFile( new File( fileToProcess )


Comment out (hide) the call of "expandMacrosInFile" and see
whether the behavior still occurs.

If yes, remove "gatherFiles" and see whether ...

If no: The behavior might have been cause by
something in "expandMacrosInFile": So now, edit
this method, commenting out its final half or so.

Recursively repeat this until the cause is found.

(This approach is not always possible or might not
always work.)


abmccullough@gmail.com 01-26-2006 04:02 PM

Re: The program that would'nt Quit
 

Seems like taking a javacore after the "done" message is printed would
a)
tell you what thread is active and what it is doing and b)
provide the information Sun would need to fix it, if it is truly a bug.

-Andrew


Thomas Hawtin 01-26-2006 08:01 PM

Re: The program that would'nt Quit
 
Roedy Green wrote:
>
> Without that System.exit, the program keeps on running endlessly using
> about 94% of my CPU time after printing "done".
>
> The odd thing is this is not a gui and it has but a single thread. I
> have no finalizers.
>
> It works fine in Jet.
>
> I don't know where to begin to track this down or how to reproduce it
> for Sun. The jar is 271K and the data files it works on are 14 MB.
>
> Any ideas on what could cause such behaviour?


Have you tried using jstack (or similar) to see what is continuing to run?

Tom Hawtin
--
Unemployed English Java programmer
http://jroller.com/page/tackline/

opalpa@gmail.com opalinski from opalpaweb 01-26-2006 08:02 PM

Re: The program that would'nt Quit
 
Good day Roedy,

I empathize with this situation; I share an aversion for System.exit .
Some ideas:

There are situations where this occurs according to definition in
non-gui programs. For example when using a JMF player one needs to
stop(), deallocate(), and close() before graceful exit. Perhaps you
are inadvartanly and indirectly creating some event loop?
---
I know one program that exits gracefully after about ten seconds. Have
you waited this long? The general rule is if there is a thread that
only wakes up every X seconds then you might wait upto X seconds for it
to return from its run().
---
I believe Java spec says that JVM can choose to exit if only demon
threads areleft, but does not have to. I doubt this is the case
because Sun's jvm always seems to take the exit choice within a couple
of seconds.
---
Perhaps you should view running threads using JConsole. You will also
see each thread's state. There is a negative in that you will see
threads related to JConsole listed; this negative is not that large.

Hopefully this email is of some service

All the best,
Opalinski
opalpa@gmail.com
http://www.geocities.com/opalpaweb/


P.S. Jconsole takes a total of maybe five minutes to learn. Run your
executable with -Dcom.sun.management.jmxremote and seperately execute
jconsole found within java's bin directory. That's it

Here is some documenation:
http://java.sun.com/j2se/1.5.0/docs/.../jconsole.html


Roedy Green 01-26-2006 10:11 PM

Re: The program that would'nt Quit
 
On 26 Jan 2006 12:02:00 -0800, "opalpa@gmail.com opalinski from
opalpaweb" <opalpa@gmail.com> wrote, quoted or indirectly quoted
someone who said :

>There are situations where this occurs according to definition in
>non-gui programs. For example when using a JMF player one needs to
>stop(), deallocate(), and close() before graceful exit. Perhaps you
>are inadvartanly and indirectly creating some event loop?


This is weird. I am doing pure batch text processing. There are no
events or listeners in the whole program. I don't spawn any threads --
any of the usual suspects in refusal to shut down.

Keep going. I hope I have overlooked something.
--
Canadian Mind Products, Roedy Green.
http://mindprod.com Java custom programming, consulting and coaching.

Roedy Green 01-26-2006 10:22 PM

Re: The program that would'nt Quit
 
On 26 Jan 2006 12:02:00 -0800, "opalpa@gmail.com opalinski from
opalpaweb" <opalpa@gmail.com> wrote, quoted or indirectly quoted
someone who said :

>I know one program that exits gracefully after about ten seconds. Have
>you waited this long? The general rule is if there is a thread that
>only wakes up every X seconds then you might wait upto X seconds for it
>to return from its run().


There are no timers, execs or threads. About the only exotic feature
of this program is using Class.forName to load most of its classes.

Perhaps some sort of incomplete i/o? I do some auxiliary disk
processing and perhaps a failure is getting lost, maybe an unclosed
file somewhere. Unfortunately Java has no open method you can grep.
Also I don't know of a way of testing if and what files are still
open.
--
Canadian Mind Products, Roedy Green.
http://mindprod.com Java custom programming, consulting and coaching.

Roedy Green 01-26-2006 11:15 PM

Re: The program that would'nt Quit
 
On 26 Jan 2006 12:02:00 -0800, "opalpa@gmail.com opalinski from
opalpaweb" <opalpa@gmail.com> wrote, quoted or indirectly quoted
someone who said :

>Perhaps you should view running threads using JConsole. You will also
>see each thread's state. There is a negative in that you will see
>threads related to JConsole listed; this negative is not that large.


I'm running it now It turns out to be ten times easier to set up than
it first looks. I have documented this, as is my wont, at
http://mindprod.com/jgloss/jconsole.html

Surprises for me includes:

1. my "single thread' app had 18 threads.

3. garbage collections are extremely frequent. There over a 1000 of
them in the time it took to connect and start looking. However
JConsole said that only a few seconds were used in GC. I had assumed
they were much less frequent and far more painful.

4. the different aged collections of objects truly do behave
differently. the older collections are rock stable. The eden has a two
frequency periodicity, like a sound with harmonics.

5. My app uses about 15 MB of heap space by default. I had never done
much thinking about ram usage. It is a bit of a shock. In the 1970s I
think I would have pulled off an app like this in 200K including the
run time.

6. There is a quantum mechanical effect. Watching the apps with
JConsole makes it work!!
--
Canadian Mind Products, Roedy Green.
http://mindprod.com Java custom programming, consulting and coaching.

Roedy Green 01-26-2006 11:16 PM

Re: The program that would'nt Quit
 
On 26 Jan 2006 08:02:06 -0800, abmccullough@gmail.com wrote, quoted or
indirectly quoted someone who said :

>taking a javacore


What class/method is that?
--
Canadian Mind Products, Roedy Green.
http://mindprod.com Java custom programming, consulting and coaching.


All times are GMT. The time now is 07:38 AM.

Powered by vBulletin®. Copyright ©2000 - 2014, vBulletin Solutions, Inc.
SEO by vBSEO ©2010, Crawlability, Inc.