Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Python > Creating unit tests on the fly

Reply
Thread Tools

Creating unit tests on the fly

 
 
Roy Smith
Guest
Posts: n/a
 
      04-08-2011
I've got a suite of unit tests for a web application. There's an
(abstract) base test class from which all test cases derive:

class BaseSmokeTest(unittest.TestCase):

BaseSmokeTest.setUpClass() fetches a UR (from a class attribute
"route", which must be defined in the derived classes), and there's a
number of test methods which do basic tests like checking for
reaonable-looking HTML (parsed with lxml), scanning the server log
file to make sure there's no error messages or stack dumps, etc.

Many of the test cases are nothing more than running the base test
methods on a particular route, i.e.

class Test_Page_About(BaseSmokeTest):
route = 'page/about'

Now, I want to do something a little fancier. I want to get a
particular page, parse the HTML to find anchor tags containing
additional URLs which I want to test. It's easy enough to pull out
the anchors I'm interested in with lxml:

selector = CSSSelector('div.st_info .st_name > a')
for anchor in selector(self.tree):
print anchor.get('href')

I can even create new test cases from these on the fly with something
like:

newClass = type("newClass", (BaseSmokeTest,), {'route': '/my/newly/
discovered/anchor'})

(credit to http://jjinux.blogspot.com/2005/03/p...ss-on-fly.html
for that neat little trick). The only thing I don't see is how I can
now get unittest.main(), which is already running, to notice that a
new test case has been created and add it to the list of test cases to
run. Any ideas on how to do that?

I suppose I don't strictly need to go the "create a new TestCase on
the fly" route, but I've already got a fair amount of infrastructure
set up around that, which I don't want to redo.
 
Reply With Quote
 
 
 
 
Raymond Hettinger
Guest
Posts: n/a
 
      04-08-2011
On Apr 8, 12:10*pm, Roy Smith <(E-Mail Removed)> wrote:
> I can even create new test cases from these on the fly with something
> like:
>
> *newClass = type("newClass", (BaseSmokeTest,), {'route': '/my/newly/
> discovered/anchor'})
>
> (credit tohttp://jjinux.blogspot.com/2005/03/python-create-new-class-on-fly.html
> for that neat little trick). *The only thing I don't see is how I can
> now get unittest.main(), which is already running, to notice that a
> new test case has been created and add it to the list of test cases to
> run. *Any ideas on how to do that?


The basic unittest.main() runner isn't well suited to this task. It
flows in a pipeline of discovery -> test_suite -> test_runner.

I think you're going to need a queue of tests, with your own test
runner consuming the queue, and your on-the-fly test creator running
as a producer thread.

Writing your own test runner isn't difficult. 1) wait on the queue
for a new test case. 2) invoke test_case.run() with a TestResult
object to hold the result 3) accumulate or report the results 4)
repeat forever.

Raymond

twitter: @raymondh

 
Reply With Quote
 
 
 
 
Roy Smith
Guest
Posts: n/a
 
      04-09-2011
In article <(E-Mail Removed)>,
Ben Finney <(E-Mail Removed)> wrote:

> Raymond Hettinger <(E-Mail Removed)> writes:
>
> > I think you're going to need a queue of tests, with your own test
> > runner consuming the queue, and your on-the-fly test creator running
> > as a producer thread.

>
> I have found the ‘testscenarios’ library very useful for this: bind a
> sequence of (name, dict) tuples to the test case class, and each tuple
> represents a scenario of data fixtures that will be applied to every
> test case function in the class.
>
> <URL:http://pypi.python.org/pypi/test-scenarios>
>
> You (the OP) will also find the ‘testing-in-python’ discussion forum
> <URL:http://lists.idyll.org/listinfo/testing-in-python> useful for this
> topic.


That link doesn't work, I assume you meant

http://pypi.python.org/pypi/testscenarios/0.2

This is interesting, and a bunch to absorb. Thanks. It might be what
I'm looking for. For the moment, I'm running the discovery then doing
something like

class_name = 'Test_DiscoveredRoute_%s' % cleaned_route_name
g = globals()
g[class_name] = type(class_name, bases, new_dict)

on each discovered route, and calling unittest.main() after I'm done
doing all that. It's not quite what I need however, so something like
testscenarios or raymondh's test queue idea might be where this needs to
go.
 
Reply With Quote
 
Roy Smith
Guest
Posts: n/a
 
      04-11-2011
In article
<(E-Mail Removed)>,
Raymond Hettinger <(E-Mail Removed)> wrote:

> I think you're going to need a queue of tests, with your own test
> runner consuming the queue, and your on-the-fly test creator running
> as a producer thread.
>
> Writing your own test runner isn't difficult. 1) wait on the queue
> for a new test case. 2) invoke test_case.run() with a TestResult
> object to hold the result 3) accumulate or report the results 4)
> repeat forever.


OK, this is working out pretty nicely. The main loop is shaping up to
to be something like:

def go(self):
result = unittest.TestResult()
while not self.queue.empty():
route, depth = self.queue.get()
test_case = self.make_test_case(route)
suite = unittest.defaultTestLoader. \
loadTestsFromTestCase(test_case)
suite.run(result)
if result.wasSuccessful():
print "passed"
else:
for case, trace in result.failures:
print case.id()
d = case.shortDescription()
if d:
print d
print trace
print '------------------------------------'

It turns out there's really no reason to put the test runner in its own
thread. Doing it all in one thread works fine; make_test_case() passes
self.queue to the newly created TestCase as part of the class dict, and
one of the test methods in my BaseSmokeTest pushes newly discovered
route onto the queue. Perhaps not the most efficient way to do things,
but since most of the clock time is spent waiting for the HTTP server to
serve up a page, it doesn't matter, and this keeps it simple.

Thanks for your help!

PS: After having spent the last 6 years of my life up to my navel in
C++, it's incredibly liberating to be creating classes on the fly in
user code
 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
“test what you fly and fly what you test” Lawrence D'Oliveiro NZ Computing 0 06-05-2009 02:06 AM
THE FLY & THE FLY II teem DVD Video 0 12-10-2005 09:31 PM
Creating module skeleton from unit tests Peter Maas Python 4 03-08-2005 11:07 AM
To Fly or not to fly? Should I move from Mozilla 1.5 to T-bird andF-bird? Daniel Steinberg Firefox 7 11-06-2003 11:31 AM
To Fly or not to fly? Should I move from Mozilla 1.5 to T-bird andF-bird? Daniel Steinberg Firefox 5 11-05-2003 06:23 AM



Advertisments