Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Ruby > When threads block

Reply
Thread Tools

When threads block

 
 
Hans Fugal
Guest
Posts: n/a
 
      09-24-2003
It's difficult to do any serious multi-threaded network programming when
the whole process blocks on system calls like read, write, and select. Is
this on a TODO list somewhere, or should I just accept it as a fact of
life? (and that may mean using a different language.)
 
Reply With Quote
 
 
 
 
nobu.nokada@softhome.net
Guest
Posts: n/a
 
      09-24-2003
Hi,

At Wed, 24 Sep 2003 10:16:53 +0900,
Hans Fugal wrote:
> It's difficult to do any serious multi-threaded network programming when
> the whole process blocks on system calls like read, write, and select. Is
> this on a TODO list somewhere, or should I just accept it as a fact of
> life? (and that may mean using a different language.)


What's your platform?

If you use Windows, it's a known problem but hard to fix with
current thread implementation.

--
Nobu Nakada

 
Reply With Quote
 
 
 
 
Thomas Sondergaard
Guest
Posts: n/a
 
      09-24-2003
> If you use Windows, it's a known problem but hard to fix with
> current thread implementation.


When will we get a new thread implementation, then?

Seriously, could you outline how these things are implemented on other
platforms to avoid blocking the entire process? Why can't this trick be used
on the windows platform? I'd like to help if I can in any way.

Another question - is true thread safety on the TODO list for ruby? Oh and
this begs the third question: Is there a release plan for ruby? I would love
to know what kind of goodies I can expect for future releases?

Cheers,

Thomas


 
Reply With Quote
 
Hans Fugal
Guest
Posts: n/a
 
      09-24-2003
On Tue, 23 Sep 2003 18:45:40 -0600, Hans Fugal wrote:

> It's difficult to do any serious multi-threaded network programming when
> the whole process blocks on system calls like read, write, and select. Is
> this on a TODO list somewhere, or should I just accept it as a fact of
> life? (and that may mean using a different language.)


Ok, I jumped the gun just a little. On linux /most/ things don't block.
But /some/ things do. e.g.

thread = Thread.new do
fifo = File.open("fifo","r")
l = fifo.readline
puts "#{Time.new} #{l}"
end

puts "#{Time.new} bar"
thread.join

Will block the entire process until the fifo is open. The following
variation does not block the entire process, though:

fifo = File.open("fifo","r")
thread = Thread.new do
l = fifo.readline
puts "#{Time.new} #{l}"
end

puts "#{Time.new} bar"
thread.join

I say "most" because that's what the wise ones on #ruby-lang said, and
indeed were surprised to see I had a counter example.

Is there a list of these sorts of things? I need to know what to expect if
I'm going to be able to use ruby for this (soft) real-time application.
 
Reply With Quote
 
gabriele renzi
Guest
Posts: n/a
 
      09-24-2003
il Wed, 24 Sep 2003 11:26:47 +0200, "Thomas Sondergaard"
<(E-Mail Removed)> ha scritto::

>> If you use Windows, it's a known problem but hard to fix with
>> current thread implementation.

>
>When will we get a new thread implementation, then?
>
>Seriously, could you outline how these things are implemented on other
>platforms to avoid blocking the entire process? Why can't this trick be used
>on the windows platform? I'd like to help if I can in any way.


The problem should be related to windows' select().
On *nix IO stuff is transparently put in a select, and ruby takes care
of giving a little time to every thread when select gives them output.
On windows this won't work cause select() just accepts socket
descriptor. I wonder if these works in python.
And is WaitForMultipleObjects() useful someway ?

>Another question - is true thread safety on the TODO list for ruby? Oh and
>this begs the third question: Is there a release plan for ruby? I would love
>to know what kind of goodies I can expect for future releases?


wait for Rite (ruby 2.0)


 
Reply With Quote
 
nobu.nokada@softhome.net
Guest
Posts: n/a
 
      09-24-2003
Hi,

At Wed, 24 Sep 2003 22:38:58 +0900,
Hans Fugal wrote:
> Ok, I jumped the gun just a little. On linux /most/ things don't block.
> But /some/ things do. e.g.
>
> thread = Thread.new do
> fifo = File.open("fifo","r")
> l = fifo.readline
> puts "#{Time.new} #{l}"
> end
>
> puts "#{Time.new} bar"
> thread.join


You can use IO::NONBLOCK with IO::RDONLY instead of "r".

> Is there a list of these sorts of things? I need to know what to expect if
> I'm going to be able to use ruby for this (soft) real-time application.


File#flock, and some Socket methods do. Use resolv-replace.rb
for the latter.

--
Nobu Nakada

 
Reply With Quote
 
Lothar Scholz
Guest
Posts: n/a
 
      09-25-2003
Hello gabriele,

Wednesday, September 24, 2003, 5:39:20 PM, you wrote:

gr> The problem should be related to windows' select().
gr> On *nix IO stuff is transparently put in a select, and ruby takes care
gr> of giving a little time to every thread when select gives them output.
gr> On windows this won't work cause select() just accepts socket
gr> descriptor. I wonder if these works in python.
gr> And is WaitForMultipleObjects() useful someway ?

It's a joke right ?

Every serious windows programming books warns you to not use 'select'
on a window plattform. Using WASSelect is the normal way. It is much
better then the UNIX API and integrates perfect into the windows
environment. But a new implementation should not use them either.
The best is to use asynchron system calls that calls back when data
is available, it is much faster then the LINUX ip stack implementation.

--
Best regards,
Lothar (E-Mail Removed)


 
Reply With Quote
 
nobu.nokada@softhome.net
Guest
Posts: n/a
 
      09-25-2003
Hi,

At Thu, 25 Sep 2003 09:24:45 +0900,
Nathaniel Talbott wrote:
> Is this intentional?
>
> irb(main):001:0> require 'socket'
> => true
> irb(main):002:0> TCPServer::new(55)
> => #<TCPServer:0x2b9ef50>
> irb(main):003:0> require 'resolv-replace'
> => true
> irb(main):004:0> TCPServer::new(55)
> ArgumentError: wrong number of arguments(1 for 2)
> from (irb):4:in `new'
> from (irb):4


No, resolv-replace.rb seems not to be 1.8 compliant.

> Because of this, using DRb with resolv-replace doesn't work very well. Any
> idea as to what is wrong?


Does this work?


Index: lib/resolv-replace.rb
================================================== =================
RCS file: /cvs/ruby/src/ruby/lib/resolv-replace.rb,v
retrieving revision 1.1
diff -u -2 -p -r1.1 resolv-replace.rb
--- lib/resolv-replace.rb 30 May 2001 09:10:26 -0000 1.1
+++ lib/resolv-replace.rb 25 Sep 2003 01:49:23 -0000
@@ -1,2 +1,3 @@
+require 'socket'
require 'resolv'

@@ -4,5 +5,5 @@ class BasicSocket
alias original_resolv_send send
def send(mesg, flags, *rest)
- rest[0] = Resolv.getaddress(rest[0]).to_s if 0 < rest.length
+ rest[0] = Resolv.getaddress(rest[0]).to_s unless rest.empty?
original_resolv_send(mesg, flags, *rest)
end
@@ -16,17 +17,18 @@ class << IPSocket
end

-class << TCPSocket
- alias original_resolv_new new
- def new(host, service)
- original_resolv_new(Resolv.getaddress(host).to_s, service)
- end
-
- alias original_resolv_open open
- def open(host, service)
- original_resolv_open(Resolv.getaddress(host).to_s, service)
+class TCPSocket
+ alias original_resolv_initialize initialize
+ def initialize(host, serv, *rest)
+ rest[0] = Resolv.getaddress(rest[0]).to_s unless rest.empty?
+ original_resolv_initialize(Resolv.getaddress(host) .to_s, serv, *rest)
end
end

class UDPSocket
+ alias original_resolv_bind bind
+ def bind(host, port)
+ original_resolv_bind(Resolv.getaddress(host).to_s, port)
+ end
+
alias original_resolv_connect connect
def connect(host, port)
@@ -36,6 +38,13 @@ class UDPSocket
alias original_resolv_send send
def send(mesg, flags, *rest)
- rest[0] = Resolv.getaddress(rest[0]).to_s if 0 < rest.length
+ rest[0] = Resolv.getaddress(rest[0]).to_s unless rest.empty?
original_resolv_send(mesg, flags, *rest)
end
end
+
+class SOCKSSocket
+ alias original_resolv_initialize initialize
+ def initialize(host, serv)
+ original_resolv_initialize(Resolv.getaddress(host) .to_s, port)
+ end
+end if defined? SOCKSSocket


--
Nobu Nakada

 
Reply With Quote
 
gabriele renzi
Guest
Posts: n/a
 
      09-25-2003
il Thu, 25 Sep 2003 09:15:50 +0900, Lothar Scholz
<(E-Mail Removed)> ha scritto::

>gr> And is WaitForMultipleObjects() useful someway ?
>
>It's a joke right ?
>
>Every serious windows programming books warns you to not use 'select'
>on a window plattform.


That's sure. I never ever read a windows programming book, this may
matter
Actually, I just wrote some win code for an OS course and at most used
3 syscall ..

>Using WASSelect is the normal way.

Is'nt WSASelect even stuck to FD_SET==array of sockets?

> It is much
>better then the UNIX API and integrates perfect into the windows
>environment.


Sure it does, but well, ruby is mostly developed on unix :/


anyway, I was simply referring to Message-Id:
<(E-Mail Removed). tochigi.jp>
 
Reply With Quote
 
Tanaka Akira
Guest
Posts: n/a
 
      09-26-2003
In article <(E-Mail Removed). tochigi.jp>,
http://www.velocityreviews.com/forums/(E-Mail Removed) writes:

> No, resolv-replace.rb seems not to be 1.8 compliant.


Please commit it.
--
Tanaka Akira

 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Fo:Block can you check to see if a block contains any text by using the block id? morrell XML 1 10-10-2006 07:18 PM
TB View, Threads, Threads with unread The Invisible Man Firefox 1 03-20-2006 02:09 AM
Standard Threads vs Weightless Threads yoda Python 2 08-01-2005 09:12 PM
threads without threads sindica@gmail.com C Programming 4 08-27-2004 09:25 PM



Advertisments