Nonblocking Sockets
 

Is this the "standard" way to make a nonblocking Socket in Ruby?

require "socket"
require "fcntl"

server = TCPServer.new(...)
server.fcntl(Fcntl::F_SETFL, Fcntl::O_NONBLOCK)

__END__

Will that work on Unix and Windows?

If I accept() a connection from that, will the connecting socket be
nonblocking as well?

Thanks for the information.

James Edward Gray II

Re: Nonblocking Sockets
 

From: "James Edward Gray II" <james@grayproductions.net>
>
> Is this the "standard" way to make a nonblocking Socket in Ruby?
>
> require "socket"
> require "fcntl"
>
> server = TCPServer.new(...)
> server.fcntl(Fcntl::F_SETFL, Fcntl::O_NONBLOCK)
>
> __END__
>
> Will that work on Unix and Windows?

As far as I know it's the "standard" way (but that
just means it's the only way I know of. :)

It doesn't work on Windows. :(

I've ended up tacking on: ... if defined? Fcntl::O_NONBLOCK
onto the fcntl() line so that my scripts still run on
windows, even though they may block inappropriately.

Blocking I/O issues have been a thorny issue for me in
trying to write cross-platform network applications in
Ruby.

> If I accept() a connection from that, will the connecting socket be
> nonblocking as well?

I seem to recall discussion of a patch to fix something
related to nonblocking I/O and accept() within the last
year or so... but I could be wrong... (I couldn't find
it with Google.)

I tend to re-issue the nonblock fcntl() call before
every socket operation I perform.

Even so, I *still* hit this timeout occasionally:

if select([sock], nil, nil, UDP_RECV_TIMEOUT)
begin
timeout(UDP_RECV_TIMEOUT) {
sock.fcntl(Fcntl::F_SETFL, Fcntl::O_NONBLOCK) if defined? Fcntl::O_NONBLOCK
resp = sock.recvfrom(65536)
}
rescue Timeout::Error
$stderr.puts "Timeout::Error in sock.recvfrom !"
end
end

...Since select() said data was ready, AND since I'm
requesting a nonblocking operation... I have no idea
why #recvfrom sometimes hangs. It used to totally hang
my program (on linux), indefinitely, about once a day,
until I added the timeout().


Regards,

Bill

Re: Nonblocking Sockets
 

Hi,

At Sun, 17 Jul 2005 05:52:34 +0900,
James Edward Gray II wrote in [ruby-talk:148402]:
> Is this the "standard" way to make a nonblocking Socket in Ruby?

Though not a "standard" way, there is io/nonblock module in
"rough".
<http://www.ruby-lang.org/cgi-bin/cvsweb.cgi/rough/ext/io/nonblock/>

> Will that work on Unix and Windows?

Still not on Windows.

> If I accept() a connection from that, will the connecting socket be
> nonblocking as well?

It may be changed so in the near future.

--
Nobu Nakada

Re: Nonblocking Sockets
 

In article <01a001c58a52$13828780$6442a8c0@musicbox>,
"Bill Kelly" <billk@cts.com> writes:

> I tend to re-issue the nonblock fcntl() call before
> every socket operation I perform.
>
> Even so, I *still* hit this timeout occasionally:
>
> if select([sock], nil, nil, UDP_RECV_TIMEOUT)
> begin
> timeout(UDP_RECV_TIMEOUT) {
> sock.fcntl(Fcntl::F_SETFL, Fcntl::O_NONBLOCK) if defined? Fcntl::O_NONBLOCK
> resp = sock.recvfrom(65536)
> }
> rescue Timeout::Error
> $stderr.puts "Timeout::Error in sock.recvfrom !"
> end
> end
>
> ...Since select() said data was ready, AND since I'm
> requesting a nonblocking operation... I have no idea
> why #recvfrom sometimes hangs.

Hmm. Linux, UDP, readable by select, not readable by recvfrom.

It may be caused by wrong UDP checksum.

Linux-Kernel Archive: UDP recvmsg blocks after select(), 2.6 bug?
http://www.ussg.iu.edu/hypermail/lin...10.0/1372.html

Debian Bug report logs - #275585 - /usr/sbin/inetd: UDP builtins can be used to hang inetd
http://bugs.debian.org/cgi-bin/bugre...85&archive=yes

select(2):
Under Linux, select may report a socket file descriptor as "ready for
reading", while nevertheless a subsequent read blocks. This could for
example happen when data has arrived but upon examination has wrong
checksum and is discarded. There may be other circumstances. Thus it
may be safer to use O_NONBLOCK on sockets that should not block.

You can test UDP with wrong checksum by hping2.
See the debian bug report #275585.

> It used to totally hang
> my program (on linux), indefinitely, about once a day,
> until I added the timeout().

It seems that Ruby process doesn't hang because timeout works.
timeout is implemented by Ruby thread.

So your problem is IPSocket#recvfrom retry when EAGAIN.

You may need lower level method which makes EAGAIN user visible.
--
Tanaka Akira

Re: Nonblocking Sockets
 

James Edward Gray II <james@grayproductions.net> writes:

> server.fcntl(Fcntl::F_SETFL, Fcntl::O_NONBLOCK)

that should work in most cases, but perhaps you should have preserved
any existing values as well, something like:

server.fcntl(Fcntl::F_SETFL, server.fcntl(Fcntl::F_GETFL) |
Fcntl::O_NONBLOCK)

YS.

Re: Nonblocking Sockets
 

On Jul 16, 2005, at 9:16 PM, nobu.nokada@softhome.net wrote:

>> If I accept() a connection from that, will the connecting socket be
>> nonblocking as well?
>>
>
> It may be changed so in the near future.

So do I need to call it once on each Socket accept() spawns? Will
once be enough?

What is the nonblocking behavior of accept? Will it throw an
Exception, return nil, or what?

Thanks.

James Edward Gray II

Re: Nonblocking Sockets
 

Hi,

At Sun, 17 Jul 2005 13:10:03 +0900,
James Edward Gray II wrote in [ruby-talk:148440]:
> So do I need to call it once on each Socket accept() spawns? Will
> once be enough?

Once on each accepted sockets.

> What is the nonblocking behavior of accept? Will it throw an
> Exception, return nil, or what?

Errno::EWOULDBLOCK will be raised.

--
Nobu Nakada

Re: Nonblocking Sockets
 

From: "Tanaka Akira" <akr@m17n.org>
> In article <01a001c58a52$13828780$6442a8c0@musicbox>,
> "Bill Kelly" <billk@cts.com> writes:
>
> > ...Since select() said data was ready, AND since I'm
> > requesting a nonblocking operation... I have no idea
> > why #recvfrom sometimes hangs.
>
> Hmm. Linux, UDP, readable by select, not readable by recvfrom.
>
> It may be caused by wrong UDP checksum.
>
> Linux-Kernel Archive: UDP recvmsg blocks after select(), 2.6 bug?
> http://www.ussg.iu.edu/hypermail/lin...10.0/1372.html
>
> Debian Bug report logs - #275585 - /usr/sbin/inetd: UDP builtins can be used to hang inetd
> http://bugs.debian.org/cgi-bin/bugre...85&archive=yes

Thank you! A most interesting read.

> > It used to totally hang
> > my program (on linux), indefinitely, about once a day,
> > until I added the timeout().
>
> It seems that Ruby process doesn't hang because timeout works.
> timeout is implemented by Ruby thread.
>
> So your problem is IPSocket#recvfrom retry when EAGAIN.
>
> You may need lower level method which makes EAGAIN user visible.

Starting with
http://www.ussg.iu.edu/hypermail/lin...10.0/1837.html
there were some number of messages in the thread saying
they can't return EAGAIN in this situation because
POSIX forbids it. (But that's in the nonblocking case.)

Since I'm requesting O_NONBLOCK prior to recvfrom(),
shouldn't it ............ ahh "your problem is
IPSocket#recvfrom retry when EAGAIN." Hmm.....

Interesting, doesn't that imply IPSocket#recvfrom is
essentially broken for NONBLOCK semantics? What I mean
is, if the socket is in nonblocking mode, and recvfrom()
gets an EAGAIN, shouldn't it just return nil to the
caller? (Or whatever the TCP methods do in a NONBLOCK
situation?) Why should it ever "block" in a NONBLOCK
situation?


Thanks,

Regards,

Bill

Re: Nonblocking Sockets
 

On Sun, Jul 17, 2005 at 07:01:54AM +0900, Bill Kelly wrote:
> From: "James Edward Gray II" <james@grayproductions.net>
> >
> > Is this the "standard" way to make a nonblocking Socket in Ruby?
> >
> > require "socket"
> > require "fcntl"
> >
> > server = TCPServer.new(...)
> > server.fcntl(Fcntl::F_SETFL, Fcntl::O_NONBLOCK)
> >
> > __END__
> >
> > Will that work on Unix and Windows?
>
> As far as I know it's the "standard" way (but that
> just means it's the only way I know of. :)
>
> It doesn't work on Windows. :(
>
> I've ended up tacking on: ... if defined? Fcntl::O_NONBLOCK
> onto the fcntl() line so that my scripts still run on
> windows, even though they may block inappropriately.
>
> Blocking I/O issues have been a thorny issue for me in
> trying to write cross-platform network applications in
> Ruby.

Aside: would it be appropriate to use Ruby threads in your application? It
can hide a lot of socket blocking nastiness. And I presume the interpreter
copes with Windows.

Regards,

Brian.

Re: Nonblocking Sockets
 

On Jul 17, 2005, at 5:41 AM, Brian Candler wrote:

> Aside: would it be appropriate to use Ruby threads in your
> application? It
> can hide a lot of socket blocking nastiness. And I presume the
> interpreter
> copes with Windows.

This works for *some* applications. Unfortunately, I large write can
still stall the process. I wish it wasn't so, but I believe you
still need to go with nonblocking IO for a robust setup.

James Edward Gray II