Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Ruby > Problem in Unit Testing Methods that start new threads

Reply
Thread Tools

Problem in Unit Testing Methods that start new threads

 
 
Hemant Kumar
Guest
Posts: n/a
 
      01-03-2007
I have a bit of doubt, in Unit Testing Programs that start new threads.
Please have a look at the code below:


class Foobar

def hello_world
p "Hello World"
@thread_status = false
end

def new_thread_start
Thread.new do
sleep(100)
@thread_status = true
end
end

end

# here goes the lame test case
require "test/unit"

module Test::Unit::Assertions
def assert_false(t_object,message=nil)
boolean = !t_object
full_message = build_message(message,'<?> Object is not
false',boolean)
assert_block(full_message) { boolean}
end
end

class TestFoobar < Test::Unit::TestCase
def setup
@foo = Foobar.new
class << @foo
def ivar var
instance_variable_get(:"@#{var}")
end
end
end

def test_hello_world
@foo.hello_world
assert_false @foo.ivar(:thread_status)
end


def test_new_thread
# sorry for a bit of not so DRY thingy
@foo.hello_world
assert_false @foo.ivar(:thread_status)
@foo.new_thread_start

# next assert is true because method was started in a new thread
# and control came back immediately, what i would probably want is
# to wait here so that i can have proper check on the method and
# state of the program, but i am not sure, if that's exactly
# approach i should take.

assert_false @foo.ivar(:thread_status)
end
end


Now, as someone suggested on IRC, I can do a join and wait for the
thread to finish. But the problem is, I don't exactly have an instance
to the thread, because its managed by a plugin and i am just using the
plugin to do stuff.

Any ideas/suggestions are more than welcome.


gnufied


 
Reply With Quote
 
 
 
 
Hemant Kumar
Guest
Posts: n/a
 
      01-03-2007
On Wed, 2007-01-03 at 22:30 +0900, Hemant Kumar wrote:
> I have a bit of doubt, in Unit Testing Programs that start new threads.
> Please have a look at the code below:
>
>
> class Foobar
>
> def hello_world
> p "Hello World"
> @thread_status = false
> end
>
> def new_thread_start
> Thread.new do
> sleep(100)
> @thread_status = true
> end
> end
>
> end
>
> # here goes the lame test case
> require "test/unit"
>
> module Test::Unit::Assertions
> def assert_false(t_object,message=nil)
> boolean = !t_object
> full_message = build_message(message,'<?> Object is not
> false',boolean)
> assert_block(full_message) { boolean}
> end
> end
>
> class TestFoobar < Test::Unit::TestCase
> def setup
> @foo = Foobar.new
> class << @foo
> def ivar var
> instance_variable_get(:"@#{var}")
> end
> end
> end
>
> def test_hello_world
> @foo.hello_world
> assert_false @foo.ivar(:thread_status)
> end
>
>
> def test_new_thread
> # sorry for a bit of not so DRY thingy
> @foo.hello_world
> assert_false @foo.ivar(:thread_status)
> @foo.new_thread_start
>
> # next assert is true because method was started in a new thread
> # and control came back immediately, what i would probably want is
> # to wait here so that i can have proper check on the method and
> # state of the program, but i am not sure, if that's exactly
> # approach i should take.
>
> assert_false @foo.ivar(:thread_status)
> end
> end
>
>
> Now, as someone suggested on IRC, I can do a join and wait for the
> thread to finish. But the problem is, I don't exactly have an instance
> to the thread, because its managed by a plugin and i am just using the
> plugin to do stuff.
>
> Any ideas/suggestions are more than welcome.


Or is it this a good idea?

Thread.list.each { |t| t.join }




 
Reply With Quote
 
 
 
 
ara.t.howard@noaa.gov
Guest
Posts: n/a
 
      01-03-2007
On Wed, 3 Jan 2007, Hemant Kumar wrote:

> I have a bit of doubt, in Unit Testing Programs that start new threads.
> Please have a look at the code below:
>
> class Foobar
>
> def hello_world
> p "Hello World"
> @thread_status = false
> end
>
> def new_thread_start
> Thread.new do
> sleep(100)
> @thread_status = true
> end
> end
>
> end
>
> # here goes the lame test case
> require "test/unit"
>
> module Test::Unit::Assertions
> def assert_false(t_object,message=nil)
> boolean = !t_object
> full_message = build_message(message,'<?> Object is not
> false',boolean)
> assert_block(full_message) { boolean}
> end
> end
>
> class TestFoobar < Test::Unit::TestCase
> def setup
> @foo = Foobar.new
> class << @foo
> def ivar var
> instance_variable_get(:"@#{var}")
> end
> end
> end
>
> def test_hello_world
> @foo.hello_world
> assert_false @foo.ivar(:thread_status)
> end
>
>
> def test_new_thread
> # sorry for a bit of not so DRY thingy
> @foo.hello_world
> assert_false @foo.ivar(:thread_status)
> @foo.new_thread_start
>
> # next assert is true because method was started in a new thread
> # and control came back immediately, what i would probably want is
> # to wait here so that i can have proper check on the method and
> # state of the program, but i am not sure, if that's exactly
> # approach i should take.
>
> assert_false @foo.ivar(:thread_status)
> end
> end
>
>
> Now, as someone suggested on IRC, I can do a join and wait for the
> thread to finish. But the problem is, I don't exactly have an instance
> to the thread, because its managed by a plugin and i am just using the
> plugin to do stuff.
>
> Any ideas/suggestions are more than welcome.


it seems that all you've managed to do is write a very long race condition.
i'm not one of those people who think the mere mention of the word
'unit-testing' bestows any sort of robustness on code. for example, the
testing of 'thread_status' in your test means nothing, as it's the source of
the race condition: you need to wrap setting and reading this var with a
semaphore. this shows why:

harp:~ > cat a.rb
class C
attr :thread
def initialize
@thread = nil
end
def new_thread
Thread.new{ @thread = Thread.current }
end
end

4242.times{|i| raise "race condition @ loopno #{ i }!" unless((c = C.new) and (Thread === c.new_thread) and c.thread) }


harp:~ > ruby a.rb
a.rb:11: race condition @ loopno 942! (RuntimeError)
from a.rb:11


regarding your specific question though, you need to verify that a thread is
created and that it's status is running. even if you don't have a handle on
the thread you can set things up in your unit test to get one. something like

harp:~ > cat a.rb
def tracking_threads &b
before = Thread.list
yield
after = Thread.list
return after - before
end

threads = tracking_threads{ 2.times{ Thread.new{ sleep } } }

threads.each{|t| p t.status}


harp:~ > ruby a.rb
"sleep"
"sleep"


cool. now we've managed to shows that Thread.new works!

i wouldn't bother with this at all unless i could also test that the created
thread did the right thing.

my 2 cts.

kind regards.

-a
--
if you find yourself slandering anybody, first imagine that your mouth is
filled with excrement. it will break you of the habit quickly enough. - the
dalai lama

 
Reply With Quote
 
Mat Schaffer
Guest
Posts: n/a
 
      01-03-2007

On Jan 3, 2007, at 8:30 AM, Hemant Kumar wrote:

> I have a bit of doubt, in Unit Testing Programs that start new
> threads.
> Please have a look at the code below:
>
>
> class Foobar
>
> def hello_world
> p "Hello World"
> @thread_status = false
> end
>
> def new_thread_start
> Thread.new do
> sleep(100)
> @thread_status = true
> end
> end
>
> end
>
> # here goes the lame test case
> require "test/unit"
>
> module Test::Unit::Assertions
> def assert_false(t_object,message=nil)
> boolean = !t_object
> full_message = build_message(message,'<?> Object is not
> false',boolean)
> assert_block(full_message) { boolean}
> end
> end
>
> class TestFoobar < Test::Unit::TestCase
> def setup
> @foo = Foobar.new
> class << @foo
> def ivar var
> instance_variable_get(:"@#{var}")
> end
> end
> end
>
> def test_hello_world
> @foo.hello_world
> assert_false @foo.ivar(:thread_status)
> end
>
>
> def test_new_thread
> # sorry for a bit of not so DRY thingy
> @foo.hello_world
> assert_false @foo.ivar(:thread_status)
> @foo.new_thread_start
>
> # next assert is true because method was started in a new thread
> # and control came back immediately, what i would probably want is
> # to wait here so that i can have proper check on the method and
> # state of the program, but i am not sure, if that's exactly
> # approach i should take.
>
> assert_false @foo.ivar(:thread_status)
> end
> end
>
>
> Now, as someone suggested on IRC, I can do a join and wait for the
> thread to finish. But the problem is, I don't exactly have an instance
> to the thread, because its managed by a plugin and i am just using the
> plugin to do stuff.
>
> Any ideas/suggestions are more than welcome.



Ara's right on this one. It's a race condition. But it looks like
you're interested in checking that the thread has indeed started. If
I were to write something like this I would probably have the unit
test run @foo.new_thread_start and immediately sleep waiting for a
signal from the child thread. The child thread would then signal
back to the any waiting threads that it had set it's thread status to
true. Altough the more I ponder it, the more I feel like I've just
moved the race condition to the sleep call. I'll have to review my
concurrent programming texts a bit. Seems like you should do the same.

Either way, check out Monitor for some insight:
http://www.ruby-doc.org/stdlib/libdo...doc/index.html

-Mat

 
Reply With Quote
 
ara.t.howard@noaa.gov
Guest
Posts: n/a
 
      01-03-2007
On Thu, 4 Jan 2007, Mat Schaffer wrote:

>
> Ara's right on this one. It's a race condition. But it looks like you're
> interested in checking that the thread has indeed started. If I were to
> write something like this I would probably have the unit test run
> @foo.new_thread_start and immediately sleep waiting for a signal from the
> child thread. The child thread would then signal back to the any waiting
> threads that it had set it's thread status to true. Altough the more I
> ponder it, the more I feel like I've just moved the race condition to the
> sleep call. I'll have to review my concurrent programming texts a bit.
> Seems like you should do the same.
>
> Either way, check out Monitor for some insight:
> http://www.ruby-doc.org/stdlib/libdo...doc/index.html
>
> -Mat


good idea mat. it's easy in ruby too:

require 'thread'

q = Queue.new

Thread.new{
q.push :running
# run
}

q.pop # wait for thread to start...

regards.

-a
--
if you find yourself slandering anybody, first imagine that your mouth is
filled with excrement. it will break you of the habit quickly enough. - the
dalai lama

 
Reply With Quote
 
Hemant Kumar
Guest
Posts: n/a
 
      01-03-2007
On Thu, 2007-01-04 at 01:04 +0900, http://www.velocityreviews.com/forums/(E-Mail Removed) wrote:
> > Now, as someone suggested on IRC, I can do a join and wait for the
> > thread to finish. But the problem is, I don't exactly have an instance
> > to the thread, because its managed by a plugin and i am just using the
> > plugin to do stuff.
> >
> > Any ideas/suggestions are more than welcome.

>
> it seems that all you've managed to do is write a very long race condition.
> i'm not one of those people who think the mere mention of the word
> 'unit-testing' bestows any sort of robustness on code. for example, the
> testing of 'thread_status' in your test means nothing, as it's the source of
> the race condition: you need to wrap setting and reading this var with a
> semaphore. this shows why:
>
> harp:~ > cat a.rb
> class C
> attr :thread
> def initialize
> @thread = nil
> end
> def new_thread
> Thread.new{ @thread = Thread.current }
> end
> end
>
> 4242.times{|i| raise "race condition @ loopno #{ i }!" unless((c = C.new) and (Thread === c.new_thread) and c.thread) }
>
>
> harp:~ > ruby a.rb
> a.rb:11: race condition @ loopno 942! (RuntimeError)
> from a.rb:11
>
>


Thanks for the insight Ara, although my class is singleton so i
shouldn't get the problem you have described above.

However can you show me a code sample, that doesn't cause race condition
in above code. I tried using Mutex on above code,

require "thread"
class C
attr :thread
def initialize
mutex = Mutex.new
mutex.lock
@thread = nil
mutex.unlock
end

def new_thread
Thread.new{
mutex = Mutex.new
mutex.lock
@thread = Thread.current
mutex.unlock
}
end

end

4242.times{|i| raise "race condition @ loopno #{ i }!" unless((c =
C.new) and (Thread === c.new_thread) and c.thread) }

And I am still getting a race condition. Using MonitorMixin to signal
execution seems like an overkill to me in above code.

So can you show me a way of making above code not race.


> regarding your specific question though, you need to verify that a thread is
> created and that it's status is running. even if you don't have a handle on
> the thread you can set things up in your unit test to get one. something like
>
> harp:~ > cat a.rb
> def tracking_threads &b
> before = Thread.list
> yield
> after = Thread.list
> return after - before
> end
>
> threads = tracking_threads{ 2.times{ Thread.new{ sleep } } }
>
> threads.each{|t| p t.status}
>
>
> harp:~ > ruby a.rb
> "sleep"
> "sleep"
>
>


Cool thanks.


 
Reply With Quote
 
Pit Capitain
Guest
Posts: n/a
 
      01-03-2007
Hemant Kumar schrieb:
> I have a bit of doubt, in Unit Testing Programs that start new threads.
> Please have a look at the code below:
> (...)


Hemant, in addition to what the others said, it's not clear to me what
you really want to test: whether a new thread is started, whether the
instance variable changed after a certain amount of time, whether the
instance variable changed at the end of the new thread, ...

I'm a fan of black box unit tests, so for me, the tests should specify
what you expect from the *interface* of the object under test. I
wouldn't test for values of certain instance variables or for threads
created internally, unless those are part of the desired interface of
your objects.

Regards,
Pit

 
Reply With Quote
 
Kenosis
Guest
Posts: n/a
 
      01-03-2007

Hemant Kumar wrote:
> On Thu, 2007-01-04 at 01:04 +0900, (E-Mail Removed) wrote:
> > > Now, as someone suggested on IRC, I can do a join and wait for the
> > > thread to finish. But the problem is, I don't exactly have an instance
> > > to the thread, because its managed by a plugin and i am just using the
> > > plugin to do stuff.
> > >
> > > Any ideas/suggestions are more than welcome.

> >
> > it seems that all you've managed to do is write a very long race condition.
> > i'm not one of those people who think the mere mention of the word
> > 'unit-testing' bestows any sort of robustness on code. for example, the
> > testing of 'thread_status' in your test means nothing, as it's the source of
> > the race condition: you need to wrap setting and reading this var with a
> > semaphore. this shows why:
> >
> > harp:~ > cat a.rb
> > class C
> > attr :thread
> > def initialize
> > @thread = nil
> > end
> > def new_thread
> > Thread.new{ @thread = Thread.current }
> > end
> > end
> >
> > 4242.times{|i| raise "race condition @ loopno #{ i }!" unless((c = C.new) and (Thread === c.new_thread) and c.thread) }
> >
> >
> > harp:~ > ruby a.rb
> > a.rb:11: race condition @ loopno 942! (RuntimeError)
> > from a.rb:11
> >
> >

>
> Thanks for the insight Ara, although my class is singleton so i
> shouldn't get the problem you have described above.
>
> However can you show me a code sample, that doesn't cause race condition
> in above code. I tried using Mutex on above code,
>
> require "thread"
> class C
> attr :thread
> def initialize
> mutex = Mutex.new
> mutex.lock
> @thread = nil
> mutex.unlock
> end
>
> def new_thread
> Thread.new{
> mutex = Mutex.new
> mutex.lock
> @thread = Thread.current
> mutex.unlock
> }
> end
>
> end
>
> 4242.times{|i| raise "race condition @ loopno #{ i }!" unless((c =
> C.new) and (Thread === c.new_thread) and c.thread) }
>
> And I am still getting a race condition. Using MonitorMixin to signal
> execution seems like an overkill to me in above code.
>
> So can you show me a way of making above code not race.
>
>
> > regarding your specific question though, you need to verify that a thread is
> > created and that it's status is running. even if you don't have a handle on
> > the thread you can set things up in your unit test to get one. something like
> >
> > harp:~ > cat a.rb
> > def tracking_threads &b
> > before = Thread.list
> > yield
> > after = Thread.list
> > return after - before
> > end
> >
> > threads = tracking_threads{ 2.times{ Thread.new{ sleep } } }
> >
> > threads.each{|t| p t.status}
> >
> >
> > harp:~ > ruby a.rb
> > "sleep"
> > "sleep"
> >
> >

>
> Cool thanks.


Try removing the "mutex = Mutex.new" from "new_thread" That's causing a
thread local mutex to be created that's only being used by that thread
and none of the others, because they have their own mutex. The threads
need to share the SAME mutex in order to enable mutual exclusion
between them, ie, the mutex you create in initialize(). That said, us
better understanding what you actually want to unit test would help us
to help you.

Cheers,

Ken

 
Reply With Quote
 
Hemant Kumar
Guest
Posts: n/a
 
      01-04-2007
On Thu, 2007-01-04 at 06:57 +0900, Pit Capitain wrote:
> Hemant Kumar schrieb:
> > I have a bit of doubt, in Unit Testing Programs that start new threads.
> > Please have a look at the code below:
> > (...)

>
> Hemant, in addition to what the others said, it's not clear to me what
> you really want to test: whether a new thread is started, whether the
> instance variable changed after a certain amount of time, whether the
> instance variable changed at the end of the new thread, ...
>
> I'm a fan of black box unit tests, so for me, the tests should specify
> what you expect from the *interface* of the object under test. I
> wouldn't test for values of certain instance variables or for threads
> created internally, unless those are part of the desired interface of
> your objects.
>
> Regards,
> Pit
>


Agreed Pit, but basically I want to test state of my program through
Unit Tests and whether each method modifies state of program as it was
intended?

Now, unit testing methods that return something on invocation is easy
and i don't need to go around running asserts on instance variables.

But I have some doubt regarding testing methods that don't return
anything explicitly and rather update instance variables.

Earlier, I was writing Unit Tests for a networking application which i
wrote using EventMachine. Now since, EM is completely based on
callbacks, you can't do a check on return values of methods. Hence I had
to rely on doing asserts on instance variables.

I would love to know, how do i go about unit testing in such cases. I
can't yet grasp concept of code that can be unit tested easily i guess.



gnufied


 
Reply With Quote
 
ara.t.howard@noaa.gov
Guest
Posts: n/a
 
      01-04-2007
On Thu, 4 Jan 2007, Hemant Kumar wrote:

>
> Agreed Pit, but basically I want to test state of my program through
> Unit Tests and whether each method modifies state of program as it was
> intended?
>
> Now, unit testing methods that return something on invocation is easy
> and i don't need to go around running asserts on instance variables.
>
> But I have some doubt regarding testing methods that don't return
> anything explicitly and rather update instance variables.
>
> Earlier, I was writing Unit Tests for a networking application which i
> wrote using EventMachine. Now since, EM is completely based on
> callbacks, you can't do a check on return values of methods. Hence I had
> to rely on doing asserts on instance variables.
>
> I would love to know, how do i go about unit testing in such cases. I
> can't yet grasp concept of code that can be unit tested easily i guess.
>


often methods should do one of two things:

- succeed or throw an exception

- return a value indicating success

in your case the former should be sufficient - you can assert that nothing is
raised and move on. if that's not sufficient consider changing the way your
method works: testing internal state is, at least, going to make maintaining
your tests very very hard since they're so cozy with your impl.

cheers.

-a
--
if you find yourself slandering anybody, first imagine that your mouth is
filled with excrement. it will break you of the habit quickly enough. - the
dalai lama

 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
problem in running a basic code in python 3.3.0 that includes HTML file Satabdi Mukherjee Python 1 04-04-2013 07:48 PM
unit-profiling, similar to unit-testing Ulrich Eckhardt Python 6 11-18-2011 02:00 AM
Unit testing errors (testing the platform module) John Maclean Python 1 04-13-2010 02:11 PM
Test::Unit - Ruby Unit Testing Framework Questions Bill Mosteller Ruby 0 10-22-2009 02:02 PM
Problem testing certain Thread methods with Test::Unit Daniel Berger Ruby 2 12-07-2007 10:21 PM



Advertisments