Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Ruby > Is it possible to dynamically extend Test::Unit test cases?

Reply
Thread Tools

Is it possible to dynamically extend Test::Unit test cases?

 
 
David Mitchell
Guest
Posts: n/a
 
      07-15-2008
Hello list,

I've got some "normal" test cases that look like this


class Testcases < Test::Unit::TestCase

def setup
...
end
def teardown
...
end
def test_1
...
end
def test_2
...
end
end


The test cases run as expected, but now I need to add some new test
cases at run time. What I think I need to do is something like this


require 'testcases'
t = Testcases.new
t.send(:define_method, "test_defined_at_run_time", "")


but I'm getting "wrong number of arguments (0 for 1)" when I try to
create the instance of Testcases.

Is there some way around this?

Thanks in advance

David Mitchell

 
Reply With Quote
 
 
 
 
phlip
Guest
Posts: n/a
 
      07-15-2008
David Mitchell wrote:

> The test cases run as expected, but now I need to add some new test
> cases at run time.


Why? Just curious...

> What I think I need to do is something like this


Try this:

class MySuite < Test::Unit::TestCase

[:concept_1, :concept_2, :concept_3].each do |thang|
define_method "test_#{thang}" do
assert_concept thang
end
end

def assert_concept(thang)
# now convert the thang symbol to a real thing and test it
end

end

> t.send(:define_method, "test_defined_at_run_time", "")


You are trying too hard. define_method is easier than that to call.

And note I put most of the processing _outside_ the define_method. Its only job
is to construct a test case name and pass in a thang. This helps make
assert_concept() more comprehensible.

--
Phlip
 
Reply With Quote
 
 
 
 
David Mitchell
Guest
Posts: n/a
 
      07-15-2008
Thanks Phlip,

The reason I need to do this is that we've got a small Watir-based DSL
written to allow us to drive an app through code that looks sort of
like:
login("fred", "password")
click_tab("Reports")
click_drilldown("Asia")
open_report("Some report title")
...

It essentially lets us construct test cases in something like plain English.

We built a few test cases using the DSL with a "normal"
Test::Unit::TestCase approach, then showed them to our testers.
Everyone was pretty excited about it; we can generate our own test
data using the DSL, the testers can comprehend the DSL without having
to dig into the nuts and bolts of the application itself, we can
finally build a full regression test suite for an application that's
basically a pig to drive using normal automation testing tools like
QTP, the scripts we write using the DSL are easy to maintain over
time, and everyone's happy.

Once our testers got a look at that, they pointed out what should've
been obvious all along: we can now get actual business users to write
a lot of the test cases using the DSL, rather than using specialist
testers. Rather than writing huge business requirements documents
that have a habit of getting misinterpreted, we can get the business
users to create what are essentially test cases using the DSL, and
that gives the developers a reasonably unambiguous description of how
things are supposed to work - we'll save a whole lot of time and money
we're currently wasting translating between business-speak and
developer-speak.

The only problem was the "scaffolding" code; apparently business users
are incapable of writing/extending code that looks like

class Testcases < Test::Unit::TestCase
def test_1
<<DSL stuff here>>
end
def test_2
<<more DSL stuff here>>
end
...
end

but they are capable of creating a bunch of test cases in individual
files that contain nothing but the DSL commands. They'll use e.g.
Notepad to create test cases in individual files that look like:
login('fred', 'password')
click_tab('Reports')
...

Fine with me - I just work here...

So now I've got a situation where we're going to have business users
generating loads of test cases using our DSL (without any of that
nasty complicated Test::Unit::TestCase stuff), saving them in flat
files, and we need to run be able to run some unspecified number of
test cases that will change over time. What I need to be able to do
is something like:

Dir.glob("app_test_cases/**/*.app).each do |test_script_file|
<<grab the content of each file, build a new Test::Unit::TestCase
wrapper around it and eval it>>
end

That's no problem - I've got most of this working already; all I
needed was the way to dynamically add new test cases, and you've now
given me a way to do that. I'll have a play with it tomorrow.

Thanks again

David Mitchell

2008/7/15 phlip <(E-Mail Removed)>:
> David Mitchell wrote:
>
>> The test cases run as expected, but now I need to add some new test
>> cases at run time.

>
> Why? Just curious...
>
>> What I think I need to do is something like this

>
> Try this:
>
> class MySuite < Test::Unit::TestCase
>
> [:concept_1, :concept_2, :concept_3].each do |thang|
> define_method "test_#{thang}" do
> assert_concept thang
> end
> end
>
> def assert_concept(thang)
> # now convert the thang symbol to a real thing and test it
> end
>
> end
>
>> t.send(:define_method, "test_defined_at_run_time", "")

>
> You are trying too hard. define_method is easier than that to call.
>
> And note I put most of the processing _outside_ the define_method. Its only
> job is to construct a test case name and pass in a thang. This helps make
> assert_concept() more comprehensible.
>
> --
> Phlip
>
>


 
Reply With Quote
 
Phlip
Guest
Posts: n/a
 
      07-15-2008
David Mitchell wrote:

> The reason I need to do this is that we've got a small Watir-based DSL
> written to allow us to drive an app through code that looks sort of
> like:
> login("fred", "password")
> click_tab("Reports")
> click_drilldown("Asia")
> open_report("Some report title")
> ..
>
> It essentially lets us construct test cases in something like plain English.


Do your programmers run that after every edit?

> We built a few test cases using the DSL with a "normal"
> Test::Unit::TestCase approach, then showed them to our testers.
> Everyone was pretty excited about it; we can generate our own test
> data using the DSL, the testers can comprehend the DSL without having
> to dig into the nuts and bolts of the application itself, we can
> finally build a full regression test suite for an application that's
> basically a pig to drive using normal automation testing tools like
> QTP, the scripts we write using the DSL are easy to maintain over
> time, and everyone's happy.


Asking the question another way - do your developers write any tests?

> Once our testers got a look at that, they pointed out what should've
> been obvious all along: we can now get actual business users to write
> a lot of the test cases using the DSL, rather than using specialist
> testers. Rather than writing huge business requirements documents
> that have a habit of getting misinterpreted, we can get the business
> users to create what are essentially test cases using the DSL, and
> that gives the developers a reasonably unambiguous description of how
> things are supposed to work - we'll save a whole lot of time and money
> we're currently wasting translating between business-speak and
> developer-speak.


Awesome! Now, can your business side actually run the tests - such as thru a web
site with a "what if" interface?

> The only problem was the "scaffolding" code; apparently business users
> are incapable of writing/extending code that looks like
>
> class Testcases < Test::Unit::TestCase
> def test_1
> <<DSL stuff here>>
> end
> def test_2
> <<more DSL stuff here>>
> end
> ...
> end
>
> but they are capable of creating a bunch of test cases in individual
> files that contain nothing but the DSL commands. They'll use e.g.
> Notepad to create test cases in individual files that look like:
> login('fred', 'password')
> click_tab('Reports')


That sounds like Fitnesse's territory. It does the Notepad thing, but with a
real GUI around it. Your customer team writes the test criteria in a DSL, and
FIT acts as a test runner.

> Fine with me - I just work here...
>
> So now I've got a situation where we're going to have business users
> generating loads of test cases using our DSL (without any of that
> nasty complicated Test::Unit::TestCase stuff), saving them in flat
> files, and we need to run be able to run some unspecified number of
> test cases that will change over time. What I need to be able to do
> is something like:
>
> Dir.glob("app_test_cases/**/*.app).each do |test_script_file|
> <<grab the content of each file, build a new Test::Unit::TestCase
> wrapper around it and eval it>>
> end


Now the slight problem is you are using TestCase as your runner, when it's full
of features you don't need, and thin on features you actually do need. More below.

> That's no problem - I've got most of this working already; all I
> needed was the way to dynamically add new test cases, and you've now
> given me a way to do that. I'll have a play with it tomorrow.


I use define_method, in Rails, like this:

[all this controller's actions].each do
define_method test_one_action_#{action} do
# test one common thing
end
end

The first thing you need to look for is if your failures are humane, or if they
are a huge mass of developer-friendly diagnostics and stack traces. The great
thing about a DSL (per RSpec) is (reputedly!) that faults can lead with clear
English too: "The frob should have returned 42 but it returned 43".

--
Phlip
 
Reply With Quote
 
Bret Pettichord
Guest
Posts: n/a
 
      07-15-2008
[Note: parts of this message were removed to make it a legal post.]

David,

test/spec or shoulda or rspec can be used to provide a more "dsl-ish" front
end for your tests.

Bret

On Tue, Jul 15, 2008 at 7:39 AM, David Mitchell <(E-Mail Removed)> wrote:

> Thanks Phlip,
>
> The reason I need to do this is that we've got a small Watir-based DSL
> written to allow us to drive an app through code that looks sort of
> like:
> login("fred", "password")
> click_tab("Reports")
> click_drilldown("Asia")
> open_report("Some report title")
> ...
>
> It essentially lets us construct test cases in something like plain
> English.
>
> We built a few test cases using the DSL with a "normal"
> Test::Unit::TestCase approach, then showed them to our testers.
> Everyone was pretty excited about it; we can generate our own test
> data using the DSL, the testers can comprehend the DSL without having
> to dig into the nuts and bolts of the application itself, we can
> finally build a full regression test suite for an application that's
> basically a pig to drive using normal automation testing tools like
> QTP, the scripts we write using the DSL are easy to maintain over
> time, and everyone's happy.
>
> Once our testers got a look at that, they pointed out what should've
> been obvious all along: we can now get actual business users to write
> a lot of the test cases using the DSL, rather than using specialist
> testers. Rather than writing huge business requirements documents
> that have a habit of getting misinterpreted, we can get the business
> users to create what are essentially test cases using the DSL, and
> that gives the developers a reasonably unambiguous description of how
> things are supposed to work - we'll save a whole lot of time and money
> we're currently wasting translating between business-speak and
> developer-speak.
>
> The only problem was the "scaffolding" code; apparently business users
> are incapable of writing/extending code that looks like
>
> class Testcases < Test::Unit::TestCase
> def test_1
> <<DSL stuff here>>
> end
> def test_2
> <<more DSL stuff here>>
> end
> ...
> end
>
> but they are capable of creating a bunch of test cases in individual
> files that contain nothing but the DSL commands. They'll use e.g.
> Notepad to create test cases in individual files that look like:
> login('fred', 'password')
> click_tab('Reports')
> ...
>
> Fine with me - I just work here...
>
> So now I've got a situation where we're going to have business users
> generating loads of test cases using our DSL (without any of that
> nasty complicated Test::Unit::TestCase stuff), saving them in flat
> files, and we need to run be able to run some unspecified number of
> test cases that will change over time. What I need to be able to do
> is something like:
>
> Dir.glob("app_test_cases/**/*.app).each do |test_script_file|
> <<grab the content of each file, build a new Test::Unit::TestCase
> wrapper around it and eval it>>
> end
>
> That's no problem - I've got most of this working already; all I
> needed was the way to dynamically add new test cases, and you've now
> given me a way to do that. I'll have a play with it tomorrow.
>
> Thanks again
>
> David Mitchell
>
> 2008/7/15 phlip <(E-Mail Removed)>:
> > David Mitchell wrote:
> >
> >> The test cases run as expected, but now I need to add some new test
> >> cases at run time.

> >
> > Why? Just curious...
> >
> >> What I think I need to do is something like this

> >
> > Try this:
> >
> > class MySuite < Test::Unit::TestCase
> >
> > [:concept_1, :concept_2, :concept_3].each do |thang|
> > define_method "test_#{thang}" do
> > assert_concept thang
> > end
> > end
> >
> > def assert_concept(thang)
> > # now convert the thang symbol to a real thing and test it
> > end
> >
> > end
> >
> >> t.send(:define_method, "test_defined_at_run_time", "")

> >
> > You are trying too hard. define_method is easier than that to call.
> >
> > And note I put most of the processing _outside_ the define_method. Its

> only
> > job is to construct a test case name and pass in a thang. This helps make
> > assert_concept() more comprehensible.
> >
> > --
> > Phlip
> >
> >

>
>



--
Bret Pettichord
CTO, WatirCraft LLC, http://www.watircraft.com
Lead Developer, Watir, http://wtr.rubyforge.org
Blog (Essays), http://www.io.com/~wazmo/blog
MiniBlog (Links), http://feeds.feedburner.com/bretshotlist

 
Reply With Quote
 
aidy
Guest
Posts: n/a
 
      07-15-2008
On Jul 15, 7:33*am, Bret Pettichord <(E-Mail Removed)> wrote:
> [Note: *parts of this message were removed to make it a legal post.]
>
> David,
>
> test/spec or shoulda or rspec can be used to provide a more "dsl-ish" front
> end for your tests.
>
> Bret


Bret's right.

I would also argue that the xUnit narrative for workflow tests is
inadequate.

You could object model your AUT

g_mail.goto_url
g_mail.welcome_page.sign_up_for_google_mail_link.c lick
gmail.create_account_page.first_name_field.set('ai dy')
gmail.create_account_page.last_name_field.set('lew is')

example: http://wiki.openqa.org/display/WTR/Example+Frameworks

You can then wrap this up in Rspec story runner tests

acceptance test:

Given a user at gmail
When clicks to create a account
And enters first and second name
Then #....

steps

Given "a user at gmail" do
g_mail.goto_url
end

When "clicks to create a account" do
g_mail.welcome_page.sign_up_for_google_mail_link.c lick
end

When "enters first and second name" do
gmail.create_account.page.first_name_field.set('ai dy')
gmail.create_account.page.last_name_field.set('lew is')
end

....

I will example this and post on Watir site

Aidy
 
Reply With Quote
 
David Mitchell
Guest
Posts: n/a
 
      07-15-2008
Thanks Bret and Aidy

I'll check out the options you've suggested today. I've only ever
needed test/unit in the past, so have never bothered to look into
alternate test frameworks.

Regards

David Mitchell

2008/7/16 aidy <(E-Mail Removed)>:
> On Jul 15, 7:33 am, Bret Pettichord <(E-Mail Removed)> wrote:
>> [Note: parts of this message were removed to make it a legal post.]
>>
>> David,
>>
>> test/spec or shoulda or rspec can be used to provide a more "dsl-ish" front
>> end for your tests.
>>
>> Bret

>
> Bret's right.
>
> I would also argue that the xUnit narrative for workflow tests is
> inadequate.
>
> You could object model your AUT
>
> g_mail.goto_url
> g_mail.welcome_page.sign_up_for_google_mail_link.c lick
> gmail.create_account_page.first_name_field.set('ai dy')
> gmail.create_account_page.last_name_field.set('lew is')
>
> example: http://wiki.openqa.org/display/WTR/Example+Frameworks
>
> You can then wrap this up in Rspec story runner tests
>
> acceptance test:
>
> Given a user at gmail
> When clicks to create a account
> And enters first and second name
> Then #....
>
> steps
>
> Given "a user at gmail" do
> g_mail.goto_url
> end
>
> When "clicks to create a account" do
> g_mail.welcome_page.sign_up_for_google_mail_link.c lick
> end
>
> When "enters first and second name" do
> gmail.create_account.page.first_name_field.set('ai dy')
> gmail.create_account.page.last_name_field.set('lew is')
> end
>
> ....
>
> I will example this and post on Watir site
>
> Aidy
>
>


 
Reply With Quote
 
Tachikoma
Guest
Posts: n/a
 
      07-16-2008
On Jul 15, 3:19*pm, David Mitchell <(E-Mail Removed)> wrote:
> Hello list,
>
> I've got some "normal" test cases that look like this
>
> class Testcases < Test::Unit::TestCase
>
> * def setup
> * * ...
> * end
> * def teardown
> * * ...
> * end
> * def test_1
> * * ...
> * end
> * def test_2
> * * ...
> * end
> end
>
> The test cases run as expected, but now I need to add some new test
> cases at run time. *What I think I need to do is something like this
>
> require 'testcases'
> t = Testcases.new
> t.send(:define_method, "test_defined_at_run_time", "")
>
> but I'm getting "wrong number of arguments (0 for 1)" when I try to
> create the instance of Testcases.
>
> Is there some way around this?
>
> Thanks in advance
>
> David Mitchell


I took the following test:

require 'test/unit/ui/console/testrunner'
require "test/unit"
class TestCases < Test::Unit::TestCase
def test_case_1
p "case 1"
end
end
eval "class TestCase2 < TestCases;def test_case_2;p 'case 2';end;end;"
Test::Unit::UI::Console::TestRunner.run(TestCases)
Test::Unit::UI::Console::TestRunner.run(TestCase2)

It gives:
Loaded suite TestCases
Started
"case 1"
.
Finished in 0.000433 seconds.

1 tests, 0 assertions, 0 failures, 0 errors
Loaded suite
Started
"case 1"
."case 2"
.
Finished in 0.001912 seconds.

2 tests, 0 assertions, 0 failures, 0 errors

It works well.and following is the same

require 'test/unit/ui/console/testrunner'
require "test/unit"
class TestCases < Test::Unit::TestCase
def test_case_1
p "case 1"
end
end
test_case_2 = Class.new(TestCases) do
def test_case_2
p "case 2"
end
end
Test::Unit::UI::Console::TestRunner.run(TestCases)
Test::Unit::UI::Console::TestRunner.run(test_case_ 2)

if you take a look into the testcase.rb
def initialize(test_method_name)
so,I wrote the following code:

require 'test/unit/ui/console/testrunner'
require "test/unit"
class TestCases < Test::Unit::TestCase
def test_case_1
p "case 1"
end
def test_case_2
p "case 2"
end
end
test_case_2 = TestCases.new(:test_case_2)
Test::Unit::UI::Console::TestRunner.run(TestCases)
Test::Unit::UI::Console::TestRunner.run(test_case_ 2)

the result:
Loaded suite TestCases
Started
"case 1"
."case 2"
.
Finished in 0.00051 seconds.

2 tests, 0 assertions, 0 failures, 0 errors
Loaded suite test_case_2(TestCases)
Started
"case 2"
.
Finished in 0.000314 seconds.

1 tests, 0 assertions, 0 failures, 0 errors

so the initialize function takes a methods name,and only the method is
running if you run the instance
but if you run the Class,the test* methods will all run through the
TestCase.suite [testcase.rb]

I hope this will help
 
Reply With Quote
 
Nick Sieger
Guest
Posts: n/a
 
      07-16-2008
On Tue, Jul 15, 2008 at 7:39 AM, David Mitchell <(E-Mail Removed)> wrote:
>
> The only problem was the "scaffolding" code; apparently business users
> are incapable of writing/extending code that looks like
>
> class Testcases < Test::Unit::TestCase
> def test_1
> <<DSL stuff here>>
> end
> def test_2
> <<more DSL stuff here>>
> end
> ...
> end
>
> but they are capable of creating a bunch of test cases in individual
> files that contain nothing but the DSL commands. They'll use e.g.
> Notepad to create test cases in individual files that look like:
> login('fred', 'password')
> click_tab('Reports')
> ...
>
> Fine with me - I just work here...
>
> So now I've got a situation where we're going to have business users
> generating loads of test cases using our DSL (without any of that
> nasty complicated Test::Unit::TestCase stuff), saving them in flat
> files, and we need to run be able to run some unspecified number of
> test cases that will change over time. What I need to be able to do
> is something like:
>
> Dir.glob("app_test_cases/**/*.app).each do |test_script_file|
> <<grab the content of each file, build a new Test::Unit::TestCase
> wrapper around it and eval it>>
> end
>
> That's no problem - I've got most of this working already; all I
> needed was the way to dynamically add new test cases, and you've now
> given me a way to do that. I'll have a play with it tomorrow.


If you haven't already, you might look at systir [1] -- does almost
exactly what you're describing. It's a little old, and perhaps not
maintained much, but it could give you an idea of how to do the
test/unit testcase definition more cleanly.

/Nick

[1]: http://www.atomicobject.com/pages/Sy...esting+in+Ruby

 
Reply With Quote
 
aidy
Guest
Posts: n/a
 
      07-16-2008
Hi Nick

> If you haven't already, you might look at systir [1] -- does almost
> exactly what you're describing. It's a little old, and perhaps not
> maintained much, but it could give you an idea of how to do the
> test/unit testcase definition more cleanly.
>
> /Nick
>
> [1]:http://www.atomicobject.com/pages/Sy...sting+in+Ruby- Hide quoted text -
>
> - Show quoted text -


The issues I have with the more procedural frameworks is that you are
increasing your code base and thus maintenance.

For example

set_terms_of_use_checkbox
clear_terms_of_use_checkbox

Will wrap the same object in two methods

Whereas this:

gmail.sign_up_page.terms_of_use_checkbox.set
gmail.sign_up_page.terms_of_use_checkbox.clear

Is one object with two different methods operated upon it.

Aidy
 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Is it possible (and wise) to extend the None-type ? Stef Mientki Python 3 11-27-2008 10:17 AM
Re: Is it possible (and wise) to extend the None-type ? Chris Rebert Python 0 11-26-2008 06:05 PM
Possible to extend my wireless network to about 1/2 to 3/4 of a mile. Leo Violette Wireless Networking 4 11-13-2006 04:09 PM
is it possible to make superclass extend another class ? krislioe@gmail.com Java 5 10-07-2006 03:09 AM
test test test test test test test Computer Support 2 07-02-2003 06:02 PM



Advertisments