Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Java > Java 1.5_22 crash my server, please for help

Reply
Thread Tools

Java 1.5_22 crash my server, please for help

 
 
Lothar Kimmeringer
Guest
Posts: n/a
 
      01-26-2010
Sherkan wrote:

> After one day, two days of working in production, my server crash with
> the error:
>
> #
> # An unexpected error has been detected by HotSpot Virtual Machine:
> #
> # SIGSEGV (0xb) at pc=0xb7962d4f, pid=3307, tid=1355598736
> #
> # Java VM: Java HotSpot(TM) Server VM (1.5.0_22-b03 mixed mode)
> # Problematic frame:
> # V [libjvm.so+0x402d4f]

[...]
>
> 2 days ago I had java newest java 1.6 with same problem.
> I get still this message: # Problematic frame: # V [libjvm.so
> +0x402d4f]


Really? The value after libjvm.so is AFAIR the offset inside
the library. That would mean that the two shared libraries were
more or less identical after compilation.

When searching the web for Cent OS and JVM-crashes, you can
find a couple of discussions going on. Reinstalling the JDK
might be worth a try. Download it directly from SUN (not the
RPM but the selfextracting file) and set the JAVA_HOME-variable
to that directory explicitly to make sure that it's really used.


Regards, Lothar
--
Lothar Kimmeringer E-Mail: http://www.velocityreviews.com/forums/(E-Mail Removed)
PGP-encrypted mails preferred (Key-ID: 0x8BC3CD81)

Always remember: The answer is forty-two, there can only be wrong
questions!
 
Reply With Quote
 
 
 
 
Roedy Green
Guest
Posts: n/a
 
      01-28-2010
On Mon, 25 Jan 2010 09:26:42 -0800 (PST), Sherkan <(E-Mail Removed)>
wrote, quoted or indirectly quoted someone who said :

># An unexpected error has been detected by HotSpot Virtual Machine:


This sounds like the sort of thing only Sun can fix.

However, if this happens after only a few days, it could be some sort
of slow memory leak. Is there a way you could monitor free memory to
see if is gradually reducing?

Do you have an exerciser -- something that simulates a curious user
who exercises all your code. If it something unusual your code is
doing, that way you could out what it was, and find some alternative
way of doing it.

What is centos? Who the made the JVM?
--
Roedy Green Canadian Mind Products
http://mindprod.com
Computers are useless. They can only give you answers.
~ Pablo Picasso (born: 1881-10-25 died: 1973-04-08 at age: 91)
 
Reply With Quote
 
 
 
 
John B. Matthews
Guest
Posts: n/a
 
      01-28-2010
In article <(E-Mail Removed)>,
Roedy Green <(E-Mail Removed)> wrote:

> What is centos?


<http://www.centos.org/> is a RedHat clone with spotty maintenance:

<http://linux.slashdot.org/story/09/08/01/1443221/CentOS-Administrator-Reappears>

IMO, RedHat's low annual fees are a relative bargain:

<http://gigaom.com/2010/01/22/how-red-hat-routed-the-recession/>

--
John B. Matthews
trashgod at gmail dot com
<http://sites.google.com/site/drjohnbmatthews>
 
Reply With Quote
 
Lew
Guest
Posts: n/a
 
      01-29-2010
Sherkan wrote, quoted or indirectly quoted someone who said :
>> # An unexpected error has been detected by HotSpot Virtual Machine:
>> #
>> # SIGSEGV (0xb) at pc=0xb7962d4f, pid=3307, tid=1355598736
>> #
>> # Java VM: Java HotSpot(TM) Server VM (1.5.0_22-b03 mixed mode)
>> # Problematic frame:
>> # V [libjvm.so+0x402d4f]


Roedy Green wrote:
> This sounds like the sort of thing only Sun can fix.
>
> However, if this happens after only a few days, it could be some sort
> of slow memory leak. Is there a way you could monitor free memory to
> see if is gradually reducing?


SIGSEGV is not usually a result of a slow memory leak. It's an illegal access
to a section of memory, in this case by the JVM itself. Monitoring free
memory as seen by the application itself probably won't help, and I really
can't see how monitoring free memory in the OS would help either.

If the problem were a memory leak or insufficient memory allocated to the
heap, you wouldn't get a SIGSEGV.

The problem is nearly impossible to diagnose from Usenet, although if we knew
the code layout of libjvm.so and had source we could probably figure out which
instruction was trying to access memory illegally.

Sherkan wrote:
>> jvm_args:


Next time make it easier to read this by formatting it for Usenet. It's hard
to follow all these complicated options when line breaks interrupt in the
middle of an argument. (I tried to neaten up here for citation.)

>> -Djava.util.logging.manager=org.apache.juli.ClassLo aderLogManager
>> -Xms1024m -Xmx1024m -XXermSize=256m -XX:MaxPermSize=512m
>> -XX:MinHeapFreeRatio=40 -XX:MaxHeapFreeRatio=70 -XX:NewRatio=3
>> -XX:MaxNewSize=512m


Some interesting choices here for your JVM arguments. Why these values? In
particular, aren't 40 an 70 the default values for the min and max HeapFreeRatios?

>> -Dsun.rmi.dgc.server.gcInterval=3600000 -XX:+UseParNewGC
>> -XX:+CMSPermGenSweepingEnabled
>> -XX:+ClassUnloading -XX:+CMSClassUnloadingEnabled
>> -XX:CompileThreshold=100 -XX:CompileThreshold=100


Why is CompileThreshold here twice? Or at all?

>> -XX:+PrintGCDetails -verbose:gc -Xloggc:/serwer/jvm


"serwer" is an interesting spelling.

>> -Djava.util.logging.config.file=/serwer/tomcat6/conf/logging.properties
>> -Djava.endorsed.dirs=/serwer/tomcat6/endorsed
>> -Dcatalina.base=/serwer/tomcat6 -Dcatalina.home=/serwer/tomcat6


You don't actually need to specify $CATALINA_BASE if it's the same as
$CATALINA_HOME.

>> -Djava.io.tmpdir=/serwer/tomcat6/temp


Moving on:
> Environment Variables:
> LD_LIBRARY_PATH=/serwer/install/jdk1.5.0_22/jre/lib/i386/server:
> /serwer/install/jdk1.5.0_22/jre/lib/i386:
> /serwer/install/jdk1.5.0_22/jre/../lib/i386


I don't see the relevance of LD_LIBRARY_PATH. That's something for C
programs, not Java. Are you using JNI? If so, why, in a Tomcat context?

> 2 days ago I had java [sic] newest java 1.6 with same problem.
> I get still this message:
> # Problematic frame:
> # V [libjvm.so+0x402d4f]

(Line wrap fixed up - you really need to manage that.)

Somehow that's just waaaaay too much of a coincidence. You might be accessing
the wrong libjvm.so.

--
Lew
 
Reply With Quote
 
Kevin McMurtrie
Guest
Posts: n/a
 
      01-29-2010
In article
<(E-Mail Removed)>,
Sherkan <(E-Mail Removed)> wrote:

> -XX:CompileThreshold=100


To my knowledge, there is no version of Java 1.5 that is free of serious
HotSpot bugs. Setting the compilation threshold so low almost
guarantees that it's going to crash or unroll loops incorrectly.

Java 1.6 seems better (so far) but I still wouldn't set the compilation
threshold lower than 10000 for a server. With a value of 100, your 512MB
Perm Heap may be completely consumed during initialization, leaving your
main code paths interpreted.
--
I won't see Google Groups replies because I must filter them as spam
 
Reply With Quote
 
Sherkan
Guest
Posts: n/a
 
      01-29-2010
Hello guys,
Thank you for reply for my email.
I need to explain one think.
I got 3 apps on tomcat 6, based on:

first and second: Seam 2.0.1GA, Facelets, JSF, RichFaces, jdbc
third is on Seam 2.2.0GA, facelets, JSF, Richfaces, jdbc

All app jar libs are in each app WEB-INF/lib directory.
One month ago my first and second apps worked alone withouth third, on
newest centos 64bit, Core 2 Duo 3GHz, 4 gb ram, and postgres 8.3,
I had to change datacenter and it all works right now, on newest 32bit
centos on Quad Core 2,5Ghz and 8GB ram.
Don't ask me why not 64bit (this is long story)

The first time my apps worked on this configuration for over 2 years.
Server worked fine for months without rebooting. Right now on the same
configuration, same tomcat, same postgres and even driver this f***
server can't hold 2 days.
It crash randomly, 20minutes after start, 5 hours, 10 hours. I even
don't know what crashes my whole server so randomly.

Yesterday I loose whole day to profile my all apps, with yourkit
profiler, and it all looks fine without memory leaks, and also if
there could be memory leak, why server crash 20 minutes after start,
there could be huge leak, but it should get an error message like:
java heap space or perm gen space, not this one.

OK here are my changes:
1. I have checked it there is no new libraries of seam, jdbc etc.
2. I've changed JSF state in web.xml from server in all aps to client.
3. I've raised -XX:NewSize=512m -XX:MaxNewSize=512m"
from 256 to 512 and I see little difference.
Today server works for all day without problems, and I see in jvm
logs, my machine start FULL GC...

68504.347: [Full GC 68504.347: [ParNew: 1334K->24K(471872K), 0.0056640
secs] 215448K->214137K(996160K), 0.0058180 secs]

In previous logs I couldn't find such an FULL GC log.
but there is also new problem, why I get this FULL gc log sometimes?

72356.821: [Full GC 72356.821: [ParNew: 419456K->419456K(471872K),
0.0000080 secs] 634061K->634061K(996160K), 0.0000940 secs]
GC locker: Trying a full collection because scavenge failed
72356.821: [Full GC 72356.821: [Tenured: 214604K->217917K(524288K),
0.7640090 secs] 634061K->217917K(996160K), [Perm : 69328K->69328K
(262144K)], 0.7641500 secs]


Here are my complete server parameters:
JAVA_OPTS="$JAVA_OPTS -server -Xms1024m -Xmx1024m -XXermSize=256m -
XX:MaxPermSize=512m"
JAVA_OPTS="$JAVA_OPTS -XX:MinHeapFreeRatio=40 -
XX:MaxHeapFreeRatio=70"
JAVA_OPTS="$JAVA_OPTS -XX:NewRatio=3 -XX:NewSize=512m -
XX:MaxNewSize=512m"
JAVA_OPTS="$JAVA_OPTS -Dsun.rmi.dgc.server.gcInterval=3600000"
JAVA_OPTS="$JAVA_OPTS -XX:+UseParNewGC -XX:
+CMSPermGenSweepingEnabled"
JAVA_OPTS="$JAVA_OPTS -XX:+ClassUnloading -XX:
+CMSClassUnloadingEnabled -XX:CompileThreshold=100"
JAVA_OPTS="$JAVA_OPTS -XX:+PrintGCDetails -verbose:gc -Xloggc:/
serwer/jvm"
JAVA_OPTS="$JAVA_OPTS -Djava.net.preferIPv4Stack=true"

Yes there was twice compile. I set this to 100 because I found on one
web site author offer this to set to minimum to force to compile to
native code very fast classes.

JAVA_OPTS="$JAVA_OPTS -XX:+PrintGCDetails -verbose:gc -Xloggc:/
serwer/jvm"

/serwer/jvm - this is my centos jvm.log file path for gc logs.
OK i will remove compilethreeshold in the next time, and we so how it
works.
I hope all java opts don't break next time.
 
Reply With Quote
 
jolz
Guest
Posts: n/a
 
      01-29-2010
> If the problem were a memory leak or insufficient memory allocated to
> the heap, you wouldn't get a SIGSEGV.


If memory leaks in native code malloc will eventually return NULL (or
more likely bad_alloc will be thrown). In 32 bit system it may be really
fast, especially if Xmx is set to maximum possible value. Sun's JVM and
JRockit and possibly all other JVM ignore malloc failures and will
crash. So yes, memory leak may cause SIGSEGV.
 
Reply With Quote
 
jolz
Guest
Posts: n/a
 
      01-29-2010
> It crash randomly, 20minutes after start, 5 hours, 10 hours. I even
> don't know what crashes my whole server so randomly.


Are you sure it's a software problem? Very often faulty hardware
(possibly RAM) causes similar behaviour.
 
Reply With Quote
 
Sherkan
Guest
Posts: n/a
 
      01-30-2010
On 30 Sty, 00:30, jolz <(E-Mail Removed)> wrote:
> > It crash randomly, 20minutes after start, 5 hours, 10 hours. I even
> > don't know what crashes my whole server so randomly.

>
> Are you sure it's a software problem? Very often faulty hardware
> (possibly RAM) causes similar behaviour.


I really don't know
So why on 32bit system there will not be thrown exception java heap
space if there is memory leak instead of SIGSEGV?
 
Reply With Quote
 
jolz
Guest
Posts: n/a
 
      01-30-2010
> So why on 32bit system there will not be thrown exception java heap
> space if there is memory leak instead of SIGSEGV?


Sun's JVM will throw OutOfMemoryError if memory limit for java was hit
but will crash if memory limit for the process will be reached. On 64
bit system it's hard to reach that limit (unless the disc is full and
there's no place for swap), but on 32 bit system native code may have
very little memory to use (for example yours 1024m (-Xmx) + 512m
(-XX:MaxPermSize) + memory for used by JVM will leave very little from
2G (depends on the OS) available for a process). If memory leaks all
available memory will be used very soon.
 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Please please please help this guy with his open source java app casioculture@gmail.com Java 4 05-05-2005 08:24 AM
HELP! HELP! PLEASE, PLEASE, PLEASE tpg comcntr Computer Support 11 02-15-2004 06:22 PM
CRASH - DirectX End-User runtime - CRASH - What to to ? reply@newsgroup.please Computer Support 1 01-05-2004 02:55 PM
HELP PLEASE!!!! server crash now getting 400 bad request John ASP .Net 3 12-20-2003 02:54 PM
please help... ...me learn C++ please please please :) KK C++ 2 10-14-2003 02:08 PM



Advertisments