Question about non-blocking NIO and Selection Keys
 

I am new to non-blocking socket i/o so please go easy on my terrible
code below :)

That being said, I have a socket that must be used for reading and
writing, but nothing else. So I initialize my socket early on as:

asyncSelector = new Selector();
sockC.configureBlocking(false);
sockC.register(asyncSelector, SelectionKey.OP_WRITE |
SelectionKey.OP_READ);

Now, whenever I need to write to my socket, I do the following:

public void write(ByteBuffer buf) throws ServerException {
int bytesWritten = 0;
int bytesToWrite = buf.remaining();

while (bytesWritten < bytesToWrite) {
limitBuffer(buf, bytesToWrite, bytesWritten,
MAX_BYTES_PER_BUF_WRITE);
waitForOperation(SelectionKey.OP_WRITE);

int bytesWrittenThisTime = sockC.write(buf);
if (bytesWrittenThisTime == -1)
throw new ServerException("Remote host unexpectedly closed
the connection");
else
bytesWritten += bytesWrittenThisTime;
}
}

public void waitForOperation(int operation) throws ServerException {
while (true) {
try {
if (asyncSelector.select() == 0)
continue;
Set<SelectionKey> selected = asyncSelector.selectedKeys();
for (Iterator<SelectionKey> i = selected.iterator();
i.hasNext(); ) {
if ((i.next().readyOps() & operation) != 0)
return;
}
}
catch (IOException e) {
throw new ServerException("An error occured while performing
a socket operation.");
}
}
}


When I need to read from the socket, I do something very similar.

Now first let me say I know this is poorly designed, but all of this
code is contained in a class that was originally designed to
encapsulate a blocking socket. Then an entire application was built
around this class, and then a specific need was encountered to make it
non-blocking, and short of re-designing the entire application (which
is not an option at all) this seemed like the best way to go about
doing it.

The ultimate problem I was trying to solve is that I had a blocking
socket that would start sending tons of data, gigabytes even. At any
point in time, the client might say "I don't want this data anymore",
and instead of waiting for 17 gigabytes fo data to be sent, which
could take quite a while, it should be able to send the server a code
which would make the server stop sending data. Before we had this
need, a blocking socket was fine. But now, the server breaks the data
up into chunks and sends it in a tight loop. The first line of the
loop simply checks if SelectionKey.OP_READ is selected and if so, it
can read the code from the client and possibly stop sending data.

The problem I am having is that if I run a previous version of the
application it is much, much, much faster. I'm not sure how to tell
if this NIO is even the culprit, as a few other changes were made as
well, but this one is the most fundamental and highest visibility, and
as such I think it's most likely to be the culprit. Can anyone
provide any observations or suggestions as how to improve this code?

Thanks

Re: Question about non-blocking NIO and Selection Keys
 

Zachary Turner wrote:
> I am new to non-blocking socket i/o so please go easy on my terrible
> code below :)
>
> That being said, I have a socket that must be used for reading and
> writing, but nothing else. So I initialize my socket early on as:
>
> asyncSelector = new Selector();
> sockC.configureBlocking(false);
> sockC.register(asyncSelector, SelectionKey.OP_WRITE |
> SelectionKey.OP_READ);
>
> Now, whenever I need to write to my socket, I do the following:
>
> public void write(ByteBuffer buf) throws ServerException {
> int bytesWritten = 0;
> int bytesToWrite = buf.remaining();
>
> while (bytesWritten < bytesToWrite) {
> limitBuffer(buf, bytesToWrite, bytesWritten,
> MAX_BYTES_PER_BUF_WRITE);
> waitForOperation(SelectionKey.OP_WRITE);
>
> int bytesWrittenThisTime = sockC.write(buf);
> if (bytesWrittenThisTime == -1)
> throw new ServerException("Remote host unexpectedly closed
> the connection");
> else
> bytesWritten += bytesWrittenThisTime;
> }
> }
>
> public void waitForOperation(int operation) throws ServerException {
> while (true) {
> try {
> if (asyncSelector.select() == 0)
> continue;
> Set<SelectionKey> selected = asyncSelector.selectedKeys();
> for (Iterator<SelectionKey> i = selected.iterator();
> i.hasNext(); ) {
> if ((i.next().readyOps() & operation) != 0)
> return;
> }
> }
> catch (IOException e) {
> throw new ServerException("An error occured while performing
> a socket operation.");
> }
> }
> }
>
>

> as such I think it's most likely to be the culprit. Can anyone
> provide any observations or suggestions as how to improve this code?

No, but I'm no NIO expert so don't dispair yet. However I do have a
couple of observations.

First, writting a small number of bytes seems counter productive with
NIO. If you are blocking, it make sense to limit the number of bytes
written so you don't block too long. With NIO, this shouldn't be an
issue, so for efficiency (speed) you should just hand NIO as much data
as you have and let it deal with sending it out.

> limitBuffer(buf, bytesToWrite, bytesWritten,
> MAX_BYTES_PER_BUF_WRITE);

The above line makes no sense to me. Why limit the bytes if the
operation won't block?

Second, you say your IO needs to be non-blocking, but you have this
method call with the word "wait" in it right in the middle of your
routine, which seems suspect. If you don't want to block, why wait? It
makes no sense. In particular, this line:

> if (asyncSelector.select() == 0)

will block, so you might be slowing yourself down there. It's worth
looking into, I think.

Good luck.

Re: Question about non-blocking NIO and Selection Keys
 

On Jul 1, 1:03*pm, Mark Space <marksp...@sbc.global.net> wrote:
> Second, you say your IO needs to be non-blocking, but you have this
> method call with the word "wait" in it right in the middle of your
> routine, which seems suspect. *If you don't want to block, why wait? *It
> makes no sense. *In particular, this line:
>
> *> * * * * *if (asyncSelector.select() == 0)
>
> will block, so you might be slowing yourself down there. *It's worth
> looking into, I think.
>
> Good luck.- Hide quoted text -
>

Well, most of this is due to the fact that it was originally designed
around blocking sockets, and without a complete re-write it's hard to
do it correctly. The idea is that I want to wait *until the socket is
ready for writing*, then I want to write N bytes of data and return
immediately.

The sole reason this was changed from blocking to non-blocking is
because we need to support the case where the client can
asynchronously send a message to the server telling it to stop sending
more data, so the server needs to be able to do a non-blocking read,
because otherwise there's no way to say "is there data waiting? if so
read it, if not let me do other stuff".

I've looked into the possibility of converting everything back to
blocking sockets. Then I would have to have one thread writing and
one thread reading from the same SocketChannel instance, which is
apparently supported according to the docs. The problem though is now
I have all this complex synchronization between the two threads,
because after writing a certain sequence of bytes, I then need to read
a certain response, then send other bytes. Currently I just have
reads and writes interspersed sequentially in the order that I need
them, it becomes more difficult if I have to force this synchronicity
across multiple threads, but I guess it's an option if I have to.

Re: Question about non-blocking NIO and Selection Keys
 

Zachary Turner schrieb:
> I am new to non-blocking socket i/o so please go easy on my terrible
> code below :)
>
> That being said, I have a socket that must be used for reading and
> writing, but nothing else. So I initialize my socket early on as:
>
> asyncSelector = new Selector();
> sockC.configureBlocking(false);
> sockC.register(asyncSelector, SelectionKey.OP_WRITE |
> SelectionKey.OP_READ);
>
> Now, whenever I need to write to my socket, I do the following:
>
> public void write(ByteBuffer buf) throws ServerException {
> int bytesWritten = 0;
> int bytesToWrite = buf.remaining();
>
> while (bytesWritten < bytesToWrite) {
> limitBuffer(buf, bytesToWrite, bytesWritten,
> MAX_BYTES_PER_BUF_WRITE);
> waitForOperation(SelectionKey.OP_WRITE);
>
> int bytesWrittenThisTime = sockC.write(buf);
> if (bytesWrittenThisTime == -1)
> throw new ServerException("Remote host unexpectedly closed
> the connection");
> else
> bytesWritten += bytesWrittenThisTime;
> }
> }
>
> public void waitForOperation(int operation) throws ServerException {
> while (true) {
> try {
> if (asyncSelector.select() == 0)
> continue;
> Set<SelectionKey> selected = asyncSelector.selectedKeys();
> for (Iterator<SelectionKey> i = selected.iterator();
> i.hasNext(); ) {
> if ((i.next().readyOps() & operation) != 0)
> return;
> }
> }
> catch (IOException e) {
> throw new ServerException("An error occured while performing
> a socket operation.");
> }
> }
> }
>
>
> When I need to read from the socket, I do something very similar.
>
> Now first let me say I know this is poorly designed, but all of this
> code is contained in a class that was originally designed to
> encapsulate a blocking socket. Then an entire application was built
> around this class, and then a specific need was encountered to make it
> non-blocking, and short of re-designing the entire application (which
> is not an option at all) this seemed like the best way to go about
> doing it.
>
> The ultimate problem I was trying to solve is that I had a blocking
> socket that would start sending tons of data, gigabytes even. At any
> point in time, the client might say "I don't want this data anymore",
> and instead of waiting for 17 gigabytes fo data to be sent, which
> could take quite a while, it should be able to send the server a code
> which would make the server stop sending data. Before we had this
> need, a blocking socket was fine. But now, the server breaks the data
> up into chunks and sends it in a tight loop. The first line of the
> loop simply checks if SelectionKey.OP_READ is selected and if so, it
> can read the code from the client and possibly stop sending data.
>
> The problem I am having is that if I run a previous version of the
> application it is much, much, much faster. I'm not sure how to tell
> if this NIO is even the culprit, as a few other changes were made as
> well, but this one is the most fundamental and highest visibility, and
> as such I think it's most likely to be the culprit. Can anyone
> provide any observations or suggestions as how to improve this code?
>
> Thanks



If you need to send data while reading from the socket nothing stops you
from Using a second thread to write the stop signal. The writing on the
other side can then be interrupted by the reading thread.


Seems to be easier than what you did.
NIO is only really useful if you use just one thread for reading from
several sockets.. using one Thread per socket with NIO spoils the benefits.


Christian

also If you ever do a rewrite I just recently found a library by Apache
called Mina which seems very promissing and easy to use.

Re: Question about non-blocking NIO and Selection Keys
 

Lew wrote:
> Christian wrote:
>> NIO is only really useful if you use just one thread for reading from
>> several sockets.. using one Thread per socket with NIO spoils the
>> benefits.
>
> I thought NIO was for reading several connections to the same socket,
> using one thread per socket instead of the traditional one thread per
> connection.

Considering that a single ServerSocket(Channel) listen and
accepts multiple Socket(Channel) one for each connection, then
there are no difference.

Arne

Re: Question about non-blocking NIO and Selection Keys
 

Zachary Turner wrote:

> Well, most of this is due to the fact that it was originally designed
> around blocking sockets, and without a complete re-write it's hard to
> do it correctly. The idea is that I want to wait *until the socket is
> ready for writing*, then I want to write N bytes of data and return
> immediately.
>
> The sole reason this was changed from blocking to non-blocking is
> because we need to support the case where the client can
> asynchronously send a message to the server telling it to stop sending
> more data, so the server needs to be able to do a non-blocking read,
> because otherwise there's no way to say "is there data waiting? if so
> read it, if not let me do other stuff".

Isn't your communication protocol just broken, then? What you
seem to be saying is that sometimes you need write/reads to be
sequenced, but sometimes you don't... Possibly things could be improved
by doing one of the following:

- use a separate connection for the asynchronous control message
(obviously wasteful, but OK for a small number of connections)
- make everything synchronous: after every portion of data sent, you
expect a response from the client "OK got that" or "OK got that but
don't send more"; if there's no data for you to send in X seconds,
send some "dummy" message (and expect a response) just to check if
the client is wanting to stop communication
- Of course, the client could also signal to the server that it
didn't want more data by closing the connection... :-)

Otherwise, I'd suggest trying to write things so that you have
(a) a single selector thread, and (b) for each connection, a 'state'
object which encompasses things like the data you're currently
trying to send, and whether the next data you expect to read is
in response to previously sent data. You can then attach your state
object to the selection key (check the SelectionKey.attach() method if
you haven't come across it). In your single selector loop, when you
get a 'ready to write', you send the next bit of data from your
state object; when you get a 'ready to read', you check if you're
expecting a response and process it, else assume it's one of your
asynchronous control messages.

It's not entirely clear to me that this is any easier or more difficult
than the synchronising you'd need between threads (each time your
read/write thread came back, you'd effectively then synchronize
and update a similar state object...). But if you want to stick to
the selector route, I think it'll be easier sticking to the standard
single-selector pattern I've outlined above.

Neil

Re: Question about non-blocking NIO and Selection Keys
 

On Jul 1, 5:43*pm, Christian <fakem...@xyz.de> wrote:
> If you need to send data while reading from the socket nothing stops you
> from Using a second thread to write the stop signal. The writing on the
> other side can then be interrupted by the reading thread.
However, interrupting a blocked read operation causes the channel to
be closed if I understand correctly. This is not an option, as I am
not done with the channel. Furthermore, reads and writes have to be
sequenced with each other. I need to write something that tells the
client i'm about to send data, then only after that is done i have to
wait for the client to send me something saying ok, and only after
that is done do i have to actually send the data to the client. It
really seems easier just to use a non-blocking socket. I know that
NIO is "most useful" when used in conjuction with selecting off of
lots of different sockets and handling tons of connections at the same
time, but making a separate thread only to completely serialize 90% of
their execution with each other is just as using NIO the way I
proposed.

Of course, I could be misunderstanding something. If the write thread
really could interrupt the read thread and not have my SocketChannel
be closed as a result, a separate thread would trivialize this
approach and make it easily superior. This way I could block in read,
have the write thread interrupt it if it gets all the way to the end
of the file successfully, then continue about my work sending the next
file to the same client.

Re: Question about non-blocking NIO and Selection Keys
 

On Jul 2, 12:03*am, Neil Coffey <neil.cof...@french-linguistics.co.uk>
wrote:
> Zachary Turner wrote:
> > Well, most of this is due to the fact that it was originally designed
> > around blocking sockets, and without a complete re-write it's hard to
> > do it correctly. *The idea is that I want to wait *until the socket is
> > ready for writing*, then I want to write N bytes of data and return
> > immediately.
>
> > The sole reason this was changed from blocking to non-blocking is
> > because we need to support the case where the client can
> > asynchronously send a message to the server telling it to stop sending
> > more data, so the server needs to be able to do a non-blocking read,
> > because otherwise there's no way to say "is there data waiting? *if so
> > read it, if not let me do other stuff".
>
> Isn't your communication protocol just broken, then? What you
> seem to be saying is that sometimes you need write/reads to be
> sequenced, but sometimes you don't... Possibly things could be improved
> by doing one of the following:
>
> - use a separate connection for the asynchronous control message
> * *(obviously wasteful, but OK for a small number of connections)
This could be an option, I considered it initially but wrote it off
without much thought because non-blockign sockets seemed like the way
to go.

> - make everything synchronous: after every portion of data sent, you
> * *expect a response from the client "OK got that" or "OK got that but
> * *don't send more"; if there's no data for you to send in X seconds,
> * *send some "dummy" message (and expect a response) just to check if
> * *the client is wanting to stop communication
Actually tried this, but considering the volume of data being sent,
the constant round trips back to the server were having a noticeable
performance impact. I got about 10% higher throughput just by
removing a 4 byte acknowledgement from client -> server after every
chunk of data.

> - Of course, the client could also signal to the server that it
> * *didn't want more data by closing the connection... :-)
Well, it's really more just like "I don't want any more of THAT
data." But the server still has lots of other data to send.


The problem is that whatever I do has to fit nicely and transparently
into the implementation of a class that used to represent a blocking
socket, and is now scattered across 10s of thousands of lines of code
of our server. So what you said at the very beginning is pretty much
spot on. It needs to be blockign most of the time, but sometimes it
doesn't. I know it sounds stupid, but that's what happens when stuff
is designed poorly in the beginning and you just have to make
something work.

What I'm after is whatever is the closest I can possibly get to a
blocking socket with the added ability to "peek" at the socket's recv
buffer.

Re: Question about non-blocking NIO and Selection Keys
 

Zachary Turner schrieb:
> On Jul 1, 5:43 pm, Christian <fakem...@xyz.de> wrote:
>> If you need to send data while reading from the socket nothing stops you
>> from Using a second thread to write the stop signal. The writing on the
>> other side can then be interrupted by the reading thread.
> However, interrupting a blocked read operation causes the channel to
> be closed if I understand correctly. This is not an option, as I am
> not done with the channel. Furthermore, reads and writes have to be
> sequenced with each other. I need to write something that tells the
> client i'm about to send data, then only after that is done i have to
> wait for the client to send me something saying ok, and only after
> that is done do i have to actually send the data to the client. It
> really seems easier just to use a non-blocking socket. I know that
> NIO is "most useful" when used in conjuction with selecting off of
> lots of different sockets and handling tons of connections at the same
> time, but making a separate thread only to completely serialize 90% of
> their execution with each other is just as using NIO the way I
> proposed.
>
> Of course, I could be misunderstanding something. If the write thread
> really could interrupt the read thread and not have my SocketChannel
> be closed as a result, a separate thread would trivialize this
> approach and make it easily superior. This way I could block in read,
> have the write thread interrupt it if it gets all the way to the end
> of the file successfully, then continue about my work sending the next
> file to the same client.

No interrupting will close the socket.
Though what you can do is ser some com variable and check that. Like
this you can send some "stop after next read" .. or specify that the
next read data is actually already for the next destination file...

Christian

Re: Question about non-blocking NIO and Selection Keys
 

On Jul 2, 10:33*am, Christian <fakem...@xyz.de> wrote:
> Zachary Turner schrieb:
>
>
>
>
>
> > On Jul 1, 5:43 pm, Christian <fakem...@xyz.de> wrote:
> >> If you need to send data while reading from the socket nothing stops you
> >> from Using a second thread to write the stop signal. The writing on the
> >> other side can then be interrupted by the reading thread.
> > However, interrupting a blocked read operation causes the channel to
> > be closed if I understand correctly. *This is not an option, as I am
> > not done with the channel. *Furthermore, reads and writes have to be
> > sequenced with each other. *I need to write something that tells the
> > client i'm about to send data, then only after that is done i have to
> > wait for the client to send me something saying ok, and only after
> > that is done do i have to actually send the data to the client. *It
> > really seems easier just to use a non-blocking socket. *I know that
> > NIO is "most useful" when used in conjuction with selecting off of
> > lots of different sockets and handling tons of connections at the same
> > time, but making a separate thread only to completely serialize 90% of
> > their execution with each other is just as using NIO the way I
> > proposed.
>
> > Of course, I could be misunderstanding something. *If the write thread
> > really could interrupt the read thread and not have my SocketChannel
> > be closed as a result, a separate thread would trivialize this
> > approach and make it easily superior. *This way I could block in read,
> > have the write thread interrupt it if it gets all the way to the end
> > of the file successfully, then continue about my work sending the next
> > file to the same client.
>
> No interrupting will close the socket.
> Though what you can do is ser some com variable and check that. Like
> this you can send some "stop after next read" .. or specify that the
> next read data is actually already for the next destination file...
>
> Christian- Hide quoted text -
>
> - Show quoted text -

FWIW I fixed the performance problem (which only occured on windows
for some odd reason) by disabling nagle algorithm on the socket. I'm
at a little bit of a loss as to why this would fix anything. My
understanding of nagle is that it's good in a situation where you have
multiple small writes followed by a read. or even just lots of small
writes. But I don't, I have like one small write followed by lots of
huge writes, with no reads anywhere in sight.