Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Java > java, multithreading, fork() - performance problem on Solaris

Reply
Thread Tools

java, multithreading, fork() - performance problem on Solaris

 
 
bugbear
Guest
Posts: n/a
 
      03-25-2008
I'm using tomcat on Solaris with around 100 threads.
This is perfectly normal and mundane, and works well.

However, I have now noticed that when the JSP's
call system() (either via runtime.exec or new processBuilder)
the whole thing slows down a lot, and the kernel
usage goes crazy. Idle time goes UP (which is odd).
JSP response tanks.

No, I can't stop shelling out from tomcat

I have traced this problem to fork(), to whit:

The fork() and fork1() functions suspend
all threads in the process before proceeding.

(ref:
http://docs.sun.com/app/docs/doc/816...6nd4n9e?a=view
)

They're all started up again after the fork(), of course, and
I have confirmed this diagnosis by crawling over an ENORMOUS
truss output log.

Now, with 100 tomcat threads, this is a bit of an overhead (*)

It appears that multithreaded Java and fork()
on Solaris have a rather deep incompatibility.

Has any body else seen/experienced this, and
has anybody found a solution/workround?

BugBear

(*) British understatement.
 
Reply With Quote
 
 
 
 
Tim Bradshaw
Guest
Posts: n/a
 
      03-26-2008
On Mar 25, 3:30 pm, bugbear <bugbear@trim_papermule.co.uk_trim> wrote:

> Has any body else seen/experienced this, and
> has anybody found a solution/workround?


It might be that posix_spawn has different behaviour, though this
seems unlikely to me. Otherwise the obvious thing would be to have a
separate, single-threaded, process which does the forks to which your
huge JVM can talk using some IPC mechanism.
 
Reply With Quote
 
 
 
 
Casper H.S. Dik
Guest
Posts: n/a
 
      03-26-2008
Tim Bradshaw <(E-Mail Removed)> writes:

>On Mar 25, 3:30 pm, bugbear <bugbear@trim_papermule.co.uk_trim> wrote:


>> Has any body else seen/experienced this, and
>> has anybody found a solution/workround?


>It might be that posix_spawn has different behaviour, though this
>seems unlikely to me. Otherwise the obvious thing would be to have a
>separate, single-threaded, process which does the forks to which your
>huge JVM can talk using some IPC mechanism.


System and posix_spawn both use vfork(); vfork() also needs to
suspend all threads, pretty much by definition.

Note that fork/fork1 are expensive in requiring all page
table entries for the process to be marked "copy on write".

vfork() is generally cheap because it only needs to suspend the
parent (but that is quite different when the parent has 100s
of threads). (While vfork() is documented as not being thread
safe, the restrictions on its use are known and fixed so we
do know how to safely use it in the implementation of system(),
popen() and posix_spawn)


The fork() process creation model and the POSIX threads in a single
address space model don't really work well together and the
combination should be avoided.

Casper
--
Expressed in this posting are my opinions. They are in no way related
to opinions held by my employer, Sun Microsystems.
Statements on Sun products included here are not gospel and may
be fiction rather than truth.
 
Reply With Quote
 
bugbear
Guest
Posts: n/a
 
      03-26-2008
Tim Bradshaw wrote:
> On Mar 25, 3:30 pm, bugbear <bugbear@trim_papermule.co.uk_trim> wrote:
>
>> Has any body else seen/experienced this, and
>> has anybody found a solution/workround?

>
> It might be that posix_spawn has different behaviour, though this
> seems unlikely to me. Otherwise the obvious thing would be to have a
> separate, single-threaded, process which does the forks to which your
> huge JVM can talk using some IPC mechanism.


Yes, thank you.

That solution is quite natural, but would complicate
out software architecture (and hence deployment).

I was hoping to avoid such a solution if possible...
(which it may not be )

It appears that (in theory) the problem
is independant of Java; any strongly multi threaded
code would have "issues" with shell out.

It's just that Java tends to *be* strongly
multi-threaded.

BugBear
 
Reply With Quote
 
Arne Vajh°j
Guest
Posts: n/a
 
      03-28-2008
bugbear wrote:
> I'm using tomcat on Solaris with around 100 threads.
> This is perfectly normal and mundane, and works well.
>
> However, I have now noticed that when the JSP's
> call system() (either via runtime.exec or new processBuilder)
> the whole thing slows down a lot, and the kernel
> usage goes crazy. Idle time goes UP (which is odd).
> JSP response tanks.
>
> No, I can't stop shelling out from tomcat


[other have commented on the fork part]

You may need to run external commands, but could
you instead of starting new processes have a
pool of permanent processes you just communicated
with ?

Arne
 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Performance Tutorials Services - Boosting Performance by DisablingUnnecessary Services on Windows XP Home Edition Software Engineer Javascript 0 06-10-2011 02:18 AM
python performance on Solaris inaf Python 10 10-17-2009 07:29 AM
Ruby on Solaris 10 performance problems Colin Mackenzie Ruby 15 01-30-2009 11:35 PM
Web Form Performance Versus Single File Performance jm ASP .Net 1 12-12-2003 11:14 PM
JVM performance Linux versus AIX/Solaris thomhashi Java 2 10-31-2003 08:02 AM



Advertisments