Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Java > NIO - OutOfMemoryException

Reply
Thread Tools

NIO - OutOfMemoryException

 
 
Camille
Guest
Posts: n/a
 
      11-15-2005
Hi,
first of all, i'm not sure of the newsgroup to post this message.
If i'm not in the good place, please tell me.

I'm using NIO in a client server project, and from a sudden, I get
OutOfMemoryException.
After some search on the web, I found that ByteBuffer.allocateDirect
was often a problem if the heap size was too low.

BUT, the problem is that I get this exception when writing on NIO, not
when reading.

Here's the code :

FIFOQueue m_responsesQueue = new FIFOQueue();
....
AKSResponse response = null;
SocketChannel client;
Charset charset = Charset.forName("UTF-8"); // client encoding

while (m_responsesQueue.size() > 0)
{
response = (AKSResponse) m_responsesQueue.getFirst();
client = response.getChannel();
if (client.isOpen())
{
if (!client.socket().isClosed())
{
if (!client.socket().isOutputShutdown())
{
client.write(charset.encode(response.getData()));
}
}
}
response.free();
response = null;
m_responsesQueue.dequeue();
}

And I get the following :

java.lang.OutOfMemoryError
at sun.misc.Unsafe.allocateMemory(Native Method)
at java.nio.DirectByteBuffer.<init>(DirectByteBuffer. java:99)
at java.nio.ByteBuffer.allocateDirect(ByteBuffer.java :285)
at sun.nio.ch.Util.getTemporaryDirectBuffer(Util.java :54)
at sun.nio.ch.IOUtil.write(IOUtil.java:69)
at
sun.nio.ch.SocketChannelImpl.write(SocketChannelIm pl.java:300)
at aks.gameServer.net.AKSWriteDaemon.run
(AKSWriteDaemon.java:102)

The line 102 being the client.write(charset.......

I presume I have to allocate more memory to my VM with the Xmx, Xmn,
X... settings, but I'm confused with all the possibilites : what about
XNewSize, XMaxNewSize, Xms, ...

Thanks in advance for your help,

Camille
--

 
Reply With Quote
 
 
 
 
Thomas Hawtin
Guest
Posts: n/a
 
      11-16-2005
Camille wrote:
> first of all, i'm not sure of the newsgroup to post this message.
> If i'm not in the good place, please tell me.


Looks good to me.

>
> I'm using NIO in a client server project, and from a sudden, I get
> OutOfMemoryException.
> After some search on the web, I found that ByteBuffer.allocateDirect
> was often a problem if the heap size was too low.


> java.lang.OutOfMemoryError
> at sun.misc.Unsafe.allocateMemory(Native Method)
> at java.nio.DirectByteBuffer.<init>(DirectByteBuffer. java:99)
> at java.nio.ByteBuffer.allocateDirect(ByteBuffer.java :285)
> at sun.nio.ch.Util.getTemporaryDirectBuffer(Util.java :54)
> at sun.nio.ch.IOUtil.write(IOUtil.java:69)
> at
> sun.nio.ch.SocketChannelImpl.write(SocketChannelIm pl.java:300)
> at aks.gameServer.net.AKSWriteDaemon.run
> (AKSWriteDaemon.java:102)


The code appears to cache direct buffers, if you are use non-direct
buffers. Possibly there is a significant delay between SoftReferences
getting cleared and reference handler cleansing the buffer. If you are
on Linux or Solaris, you might see the problem disappear when running as
root (I believe thread priorities actually work then). I don't know the
details of interaction between SoftReferences, finalisable objects, the
garbage collector and the reference handler. If it requires another
appropriate GC cycle to queue the finaliser after the SoftReference is
cleared, then that may well cause problems.

If you have many threads, then you can end up allocating a silly number
of these temporary direct buffers, as the cache is thread-local. Their
interaction with the garbage collector is such that you don't want to be
doing that.

The obvious solution is to use directly allocated buffers yourself. But
be sure they are long lived, rather than constantly allocated and collected.

> I presume I have to allocate more memory to my VM with the Xmx, Xmn,
> X... settings, but I'm confused with all the possibilites : what about
> XNewSize, XMaxNewSize, Xms, ...


In general you should set -Xmx to how much heap you want. Set the
minimum heap, -Xms, to the same in order to avoid unnecessary GC. If you
don't want to do that, I believe using -server changes SoftReference
calculations to use free heap space if maximum were allocated in place
of current free heap space. IIRC, there is a magic -XX option to change
the maximum space allocable for direct buffers, as a fraction of heap size.

Tom Hawtin
--
Unemployed English Java programmer
http://jroller.com/page/tackline/
 
Reply With Quote
 
 
 
 
camille.chafer@gmail.com
Guest
Posts: n/a
 
      12-05-2005
Hi folks,
Just a little message to say I finnaly found the solution.
The problem was the SocketChannel.write implementation.
Indeed, even if you don't use DirectByteBuffer in your code,
SocketChannel.write does it for you.
This means that every time you call the write method, a new
DirectByteBuffer is created. And since there is no real and effective
GC on this type of objects, on heavy load, your application simply
crash.

The solution ? I did create a big DirectBuffer by myself, copy the
encoded result into it, then use it to do the write stuff.
Here's the code :

ByteBuffer outputDirectBuffer =
ByteBuffer.allocateDirect(OUTPUT_DIRECTBUFFER_SIZE );
int dataSent, dataToSend;
while (m_responsesQueue.size() > 0) {
response = (AKSResponse) m_responsesQueue.getFirst();
client = response.getChannel();
if (client.isOpen()) {
if (!client.socket().isClosed()) {
if (!client.socket().isOutputShutdown()) {
outputDirectBuffer.clear();
outputDirectBuffer.put(charset.encode(response.get Data()));
outputDirectBuffer.flip();
dataToSend = outputDirectBuffer.limit() -
outputDirectBuffer.position();
dataSent = client.write(outputDirectBuffer);
if (dataSent != dataToSend) {
//.. not enough bytes were sent. Maybe the network heap full ?
logger.fatal ("Unable to send " + dataToSend + " bytes : " +
dataSend + " sent");
}
}
}
}
response.free();
response = null;
m_responsesQueue.dequeue();
}


Using this, I now achieve to let about 6.000 players join games on my
server, without any memory problem).
Thanks again for your help,

Camille

 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Exception of type System.OutOfMemoryException is thrown =?Utf-8?B?U3Rhbg==?= ASP .Net 11 12-28-2004 07:00 PM
NIO with timeouts != NIO? iksrazal Java 1 06-18-2004 02:28 PM
OutOfMemoryException Error: Garbage Collector doesn't release memory to OS Pyramis ASP .Net 0 01-25-2004 04:37 PM
OutOfMemoryException =?Utf-8?B?amF6?= ASP .Net 2 12-16-2003 12:41 AM
Base Exception:System.OutOfMemoryException error message =?Utf-8?B?UmlwdWwgSGFuZGE=?= ASP .Net 1 11-18-2003 07:52 AM



Advertisments