Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Ruby > Nonblocking Sockets

Reply
Thread Tools

Nonblocking Sockets

 
 
James Edward Gray II
Guest
Posts: n/a
 
      07-16-2005
Is this the "standard" way to make a nonblocking Socket in Ruby?

require "socket"
require "fcntl"

server = TCPServer.new(...)
server.fcntl(Fcntl::F_SETFL, Fcntl::O_NONBLOCK)

__END__

Will that work on Unix and Windows?

If I accept() a connection from that, will the connecting socket be
nonblocking as well?

Thanks for the information.

James Edward Gray II



 
Reply With Quote
 
 
 
 
Bill Kelly
Guest
Posts: n/a
 
      07-16-2005
From: "James Edward Gray II" <(E-Mail Removed)>
>
> Is this the "standard" way to make a nonblocking Socket in Ruby?
>
> require "socket"
> require "fcntl"
>
> server = TCPServer.new(...)
> server.fcntl(Fcntl::F_SETFL, Fcntl::O_NONBLOCK)
>
> __END__
>
> Will that work on Unix and Windows?


As far as I know it's the "standard" way (but that
just means it's the only way I know of.

It doesn't work on Windows.

I've ended up tacking on: ... if defined? Fcntl::O_NONBLOCK
onto the fcntl() line so that my scripts still run on
windows, even though they may block inappropriately.

Blocking I/O issues have been a thorny issue for me in
trying to write cross-platform network applications in
Ruby.

> If I accept() a connection from that, will the connecting socket be
> nonblocking as well?


I seem to recall discussion of a patch to fix something
related to nonblocking I/O and accept() within the last
year or so... but I could be wrong... (I couldn't find
it with Google.)

I tend to re-issue the nonblock fcntl() call before
every socket operation I perform.

Even so, I *still* hit this timeout occasionally:

if select([sock], nil, nil, UDP_RECV_TIMEOUT)
begin
timeout(UDP_RECV_TIMEOUT) {
sock.fcntl(Fcntl::F_SETFL, Fcntl::O_NONBLOCK) if defined? Fcntl::O_NONBLOCK
resp = sock.recvfrom(65536)
}
rescue Timeout::Error
$stderr.puts "Timeout::Error in sock.recvfrom !"
end
end

...Since select() said data was ready, AND since I'm
requesting a nonblocking operation... I have no idea
why #recvfrom sometimes hangs. It used to totally hang
my program (on linux), indefinitely, about once a day,
until I added the timeout().


Regards,

Bill



 
Reply With Quote
 
 
 
 
nobu.nokada@softhome.net
Guest
Posts: n/a
 
      07-17-2005
Hi,

At Sun, 17 Jul 2005 05:52:34 +0900,
James Edward Gray II wrote in [ruby-talk:148402]:
> Is this the "standard" way to make a nonblocking Socket in Ruby?


Though not a "standard" way, there is io/nonblock module in
"rough".
<http://www.ruby-lang.org/cgi-bin/cvsweb.cgi/rough/ext/io/nonblock/>

> Will that work on Unix and Windows?


Still not on Windows.

> If I accept() a connection from that, will the connecting socket be
> nonblocking as well?


It may be changed so in the near future.

--
Nobu Nakada


 
Reply With Quote
 
Tanaka Akira
Guest
Posts: n/a
 
      07-17-2005
In article <01a001c58a52$13828780$6442a8c0@musicbox>,
"Bill Kelly" <(E-Mail Removed)> writes:

> I tend to re-issue the nonblock fcntl() call before
> every socket operation I perform.
>
> Even so, I *still* hit this timeout occasionally:
>
> if select([sock], nil, nil, UDP_RECV_TIMEOUT)
> begin
> timeout(UDP_RECV_TIMEOUT) {
> sock.fcntl(Fcntl::F_SETFL, Fcntl::O_NONBLOCK) if defined? Fcntl::O_NONBLOCK
> resp = sock.recvfrom(65536)
> }
> rescue Timeout::Error
> $stderr.puts "Timeout::Error in sock.recvfrom !"
> end
> end
>
> ...Since select() said data was ready, AND since I'm
> requesting a nonblocking operation... I have no idea
> why #recvfrom sometimes hangs.


Hmm. Linux, UDP, readable by select, not readable by recvfrom.

It may be caused by wrong UDP checksum.

Linux-Kernel Archive: UDP recvmsg blocks after select(), 2.6 bug?
http://www.ussg.iu.edu/hypermail/lin...10.0/1372.html

Debian Bug report logs - #275585 - /usr/sbin/inetd: UDP builtins can be used to hang inetd
http://bugs.debian.org/cgi-bin/bugre...85&archive=yes

select(2):
Under Linux, select may report a socket file descriptor as "ready for
reading", while nevertheless a subsequent read blocks. This could for
example happen when data has arrived but upon examination has wrong
checksum and is discarded. There may be other circumstances. Thus it
may be safer to use O_NONBLOCK on sockets that should not block.

You can test UDP with wrong checksum by hping2.
See the debian bug report #275585.

> It used to totally hang
> my program (on linux), indefinitely, about once a day,
> until I added the timeout().


It seems that Ruby process doesn't hang because timeout works.
timeout is implemented by Ruby thread.

So your problem is IPSocket#recvfrom retry when EAGAIN.

You may need lower level method which makes EAGAIN user visible.
--
Tanaka Akira


 
Reply With Quote
 
Yohanes Santoso
Guest
Posts: n/a
 
      07-17-2005
James Edward Gray II <(E-Mail Removed)> writes:

> server.fcntl(Fcntl::F_SETFL, Fcntl::O_NONBLOCK)


that should work in most cases, but perhaps you should have preserved
any existing values as well, something like:

server.fcntl(Fcntl::F_SETFL, server.fcntl(Fcntl::F_GETFL) |
Fcntl::O_NONBLOCK)

YS.


 
Reply With Quote
 
James Edward Gray II
Guest
Posts: n/a
 
      07-17-2005
On Jul 16, 2005, at 9:16 PM, http://www.velocityreviews.com/forums/(E-Mail Removed) wrote:

>> If I accept() a connection from that, will the connecting socket be
>> nonblocking as well?
>>

>
> It may be changed so in the near future.


So do I need to call it once on each Socket accept() spawns? Will
once be enough?

What is the nonblocking behavior of accept? Will it throw an
Exception, return nil, or what?

Thanks.

James Edward Gray II


 
Reply With Quote
 
nobu.nokada@softhome.net
Guest
Posts: n/a
 
      07-17-2005
Hi,

At Sun, 17 Jul 2005 13:10:03 +0900,
James Edward Gray II wrote in [ruby-talk:148440]:
> So do I need to call it once on each Socket accept() spawns? Will
> once be enough?


Once on each accepted sockets.

> What is the nonblocking behavior of accept? Will it throw an
> Exception, return nil, or what?


Errno::EWOULDBLOCK will be raised.

--
Nobu Nakada


 
Reply With Quote
 
Bill Kelly
Guest
Posts: n/a
 
      07-17-2005
From: "Tanaka Akira" <(E-Mail Removed)>
> In article <01a001c58a52$13828780$6442a8c0@musicbox>,
> "Bill Kelly" <(E-Mail Removed)> writes:
>
> > ...Since select() said data was ready, AND since I'm
> > requesting a nonblocking operation... I have no idea
> > why #recvfrom sometimes hangs.

>
> Hmm. Linux, UDP, readable by select, not readable by recvfrom.
>
> It may be caused by wrong UDP checksum.
>
> Linux-Kernel Archive: UDP recvmsg blocks after select(), 2.6 bug?
> http://www.ussg.iu.edu/hypermail/lin...10.0/1372.html
>
> Debian Bug report logs - #275585 - /usr/sbin/inetd: UDP builtins can be used to hang inetd
> http://bugs.debian.org/cgi-bin/bugre...85&archive=yes


Thank you! A most interesting read.

> > It used to totally hang
> > my program (on linux), indefinitely, about once a day,
> > until I added the timeout().

>
> It seems that Ruby process doesn't hang because timeout works.
> timeout is implemented by Ruby thread.
>
> So your problem is IPSocket#recvfrom retry when EAGAIN.
>
> You may need lower level method which makes EAGAIN user visible.


Starting with
http://www.ussg.iu.edu/hypermail/lin...10.0/1837.html
there were some number of messages in the thread saying
they can't return EAGAIN in this situation because
POSIX forbids it. (But that's in the nonblocking case.)

Since I'm requesting O_NONBLOCK prior to recvfrom(),
shouldn't it ............ ahh "your problem is
IPSocket#recvfrom retry when EAGAIN." Hmm.....

Interesting, doesn't that imply IPSocket#recvfrom is
essentially broken for NONBLOCK semantics? What I mean
is, if the socket is in nonblocking mode, and recvfrom()
gets an EAGAIN, shouldn't it just return nil to the
caller? (Or whatever the TCP methods do in a NONBLOCK
situation?) Why should it ever "block" in a NONBLOCK
situation?


Thanks,

Regards,

Bill





 
Reply With Quote
 
Brian Candler
Guest
Posts: n/a
 
      07-17-2005
On Sun, Jul 17, 2005 at 07:01:54AM +0900, Bill Kelly wrote:
> From: "James Edward Gray II" <(E-Mail Removed)>
> >
> > Is this the "standard" way to make a nonblocking Socket in Ruby?
> >
> > require "socket"
> > require "fcntl"
> >
> > server = TCPServer.new(...)
> > server.fcntl(Fcntl::F_SETFL, Fcntl::O_NONBLOCK)
> >
> > __END__
> >
> > Will that work on Unix and Windows?

>
> As far as I know it's the "standard" way (but that
> just means it's the only way I know of.
>
> It doesn't work on Windows.
>
> I've ended up tacking on: ... if defined? Fcntl::O_NONBLOCK
> onto the fcntl() line so that my scripts still run on
> windows, even though they may block inappropriately.
>
> Blocking I/O issues have been a thorny issue for me in
> trying to write cross-platform network applications in
> Ruby.


Aside: would it be appropriate to use Ruby threads in your application? It
can hide a lot of socket blocking nastiness. And I presume the interpreter
copes with Windows.

Regards,

Brian.


 
Reply With Quote
 
James Edward Gray II
Guest
Posts: n/a
 
      07-17-2005
On Jul 17, 2005, at 5:41 AM, Brian Candler wrote:

> Aside: would it be appropriate to use Ruby threads in your
> application? It
> can hide a lot of socket blocking nastiness. And I presume the
> interpreter
> copes with Windows.


This works for *some* applications. Unfortunately, I large write can
still stall the process. I wish it wasn't so, but I believe you
still need to go with nonblocking IO for a robust setup.

James Edward Gray II


 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
nonblocking sockets nooneinparticular314159@yahoo.com Java 2 04-08-2006 09:41 AM
nonblocking sockets, select, & OpenSSL rakaur Ruby 4 02-14-2006 12:20 AM
strange problem with nonblocking sockets anki C Programming 3 03-01-2005 05:26 AM
nonblocking read() Peter Ammon Python 3 11-17-2004 12:54 AM
setSoTimeout in nonblocking mode? KimTaehwan Java 0 11-16-2003 01:51 AM



Advertisments