Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Python > Python Code Auditing Tool

Reply
Thread Tools

Python Code Auditing Tool

 
 
Robey Holderith
Guest
Posts: n/a
 
      02-02-2005
Does anybody know of a tool that can tell me all possible exceptions that
might occur in each line of code? What I'm hoping to find is something
like the following:

given all necessary python source and a given line ( my.py:40 ) it would
generate a list of possible exception classes sorted by function
(preferably in a tree).

Example:
------------------

my.py:40 | parsestring(genstring())

Possible Exceptions:

-def parsestring():
InvalidCharacterException
EmptyStringException
-class string, def split():
(All Exceptions that might occur directly in string.split() that are
not caught by parsestring())
(All functions called by string.split() and their exceptions and sub-
functions)
-def genstring():
SomeException
...

--------------------

This would be extremely useful for deciding how to write try: except
blocks and in figuring out what all possible errors that might occur would
be.

-Robey Holderith

 
Reply With Quote
 
 
 
 
Paul Rubin
Guest
Posts: n/a
 
      02-02-2005
Robey Holderith <(E-Mail Removed)> writes:
> Does anybody know of a tool that can tell me all possible exceptions that
> might occur in each line of code? What I'm hoping to find is something
> like the following:


That is impossible. The parameter to the raise statement is a class
object, which can be anything. I.e. you could say:

class ex1: pass
class ex2: pass

if something(): my_ex = ex1
else: my_ex = ex2

raise my_ex # static tool can't know what exception gets raised here.
 
Reply With Quote
 
 
 
 
Robey Holderith
Guest
Posts: n/a
 
      02-02-2005
On Tue, 01 Feb 2005 21:52:28 -0800, Paul Rubin wrote:

> Robey Holderith <(E-Mail Removed)> writes:
>> Does anybody know of a tool that can tell me all possible exceptions that
>> might occur in each line of code? What I'm hoping to find is something
>> like the following:

>
> That is impossible. The parameter to the raise statement is a class
> object, which can be anything. I.e. you could say:
>
> class ex1: pass
> class ex2: pass
>
> if something(): my_ex = ex1
> else: my_ex = ex2
>
> raise my_ex # static tool can't know what exception gets raised here.


I suppose that I am willing to lessen my expectations from _all_ to most.
Regarding your example I could also do:

if something():
def nothing(): return 0
else:
def nothing(): return 1

But this doesn't stop IDEs from attempting to do auto-completion. I'm not
trying to find hidden exceptions... just trying to easily get an idea of
what could go wrong on each line of code.

-Robey Holderith

 
Reply With Quote
 
Diez B. Roggisch
Guest
Posts: n/a
 
      02-02-2005
> I suppose that I am willing to lessen my expectations from _all_ to most.
> Regarding your example I could also do:
>
> if something():
> def nothing(): return 0
> else:
> def nothing(): return 1
>
> But this doesn't stop IDEs from attempting to do auto-completion. I'm not
> trying to find hidden exceptions... just trying to easily get an idea of
> what could go wrong on each line of code.


There is AFAIK only one language that this can de accomplished - java, and
that's because of these checked exceptions of theirs. But checked
exceptions are considered harmful:

http://www.gcek.net/ref/books/sw/ooad/tip/#_Toc41169682

I totally agree with that - in java, I tend to throw SystemExceptions to rid
myself of endless try/catch clauses that obscure the real problem.

So - there is no way of knowing this. The only thing I can think of is to
keep some docs around that specify what exceptions to be expected, and that
tool of yours could try and see if it can identify a function/method by
name and notify you of the possible exceptions thrown. Might actually work
out quite well for the standardlib, if one does the work for annotating all
functions/methods properly.

--
Regards,

Diez B. Roggisch
 
Reply With Quote
 
Neil Benn
Guest
Posts: n/a
 
      02-02-2005
Diez B. Roggisch wrote:

>>I suppose that I am willing to lessen my expectations from _all_ to most.
>> Regarding your example I could also do:
>>
>><
>>
>>

<snip>

>>There is AFAIK only one language that this can de accomplished - java, and
>>that's because of these checked exceptions of theirs. But checked
>>exceptions are considered harmful:
>>
>>http://www.gcek.net/ref/books/sw/ooad/tip/#_Toc41169682
>>
>>I totally agree with that - in java, I tend to throw SystemExceptions to rid
>>myself of endless try/catch clauses that obscure the real problem.
>>
>>
>>

<snip>
Hello,

I'm afraid that the only reliable way to gather what exceptions are
raised is to read docs and/or come up with test cases. This has been a
bugbear of mine in Python as it's not common to find a nice :Exceptions:
IOError <desc>, IllegalArgumentError <desc> type of description in the docs.

However if you want an incomplete test, you could parse the code and
check for raises and retrieve the class name of the exception - however
this would be patchy at best. Therefore it would sort of negate the
point of doing the analysis in the first place.

Even in Java you cannot find every exception that will be
thrown, only 'checked' exceptions but this is a percentage of all the
exceptions (BTW why do you throw SystemException - it's a CORBA
exception! OK, it's a runtime exception but why not just simply extend
RuntimeException?). Also, if someone ever puts - catch (Exception e){}
in their code they deserve to be kneecapped, IMHO the fault is with
sloppy coding not with the supplied tools.

Unfortunately its docs and testing again, that's why we get paid (if
you're doing a job) or not paid (if you're doing it for fun!). Although
one language which comes closer is Eiffel which has require and ensure
clauses on every method (following Meyer's Programming by contract
philosophy).

Cheers,

Neil

--

Neil Benn
Senior Automation Engineer
Cenix BioScience
BioInnovations Zentrum
Tatzberg 46
D-01307
Dresden
Germany

Tel : +49 (0)351 4173 154
e-mail : http://www.velocityreviews.com/forums/(E-Mail Removed)
Cenix Website : http://www.cenix-bioscience.com

 
Reply With Quote
 
Diez B. Roggisch
Guest
Posts: n/a
 
      02-02-2005
Hi,

> I'm afraid that the only reliable way to gather what exceptions are
> raised is to read docs and/or come up with test cases. This has been a
> bugbear of mine in Python as it's not common to find a nice :Exceptions:
> IOError <desc>, IllegalArgumentError <desc> type of description in the
> docs.
>
> However if you want an incomplete test, you could parse the code and
> check for raises and retrieve the class name of the exception - however
> this would be patchy at best. Therefore it would sort of negate the
> point of doing the analysis in the first place.


I don't want that - the OP wants. I agree with you.

> Even in Java you cannot find every exception that will be
> thrown, only 'checked' exceptions but this is a percentage of all the
> exceptions (BTW why do you throw SystemException - it's a CORBA
> exception! OK, it's a runtime exception but why not just simply extend
> RuntimeException?). Also, if someone ever puts - catch (Exception e){}
> in their code they deserve to be kneecapped, IMHO the fault is with
> sloppy coding not with the supplied tools.


Most probably I throw RuntimeException - that was out of my head, I luckily
I haven't been coding java too much lately

> Unfortunately its docs and testing again, that's why we get paid (if
> you're doing a job) or not paid (if you're doing it for fun!). Although
> one language which comes closer is Eiffel which has require and ensure
> clauses on every method (following Meyer's Programming by contract
> philosophy).


Full ack again.

--
Regards,

Diez B. Roggisch
 
Reply With Quote
 
System Administrator
Guest
Posts: n/a
 
      02-02-2005

>> Does anybody know of a tool that can tell me all possible exceptions
>> that might occur in each line of code? What I'm hoping to find is
>> something like the following:


Paul> That is impossible. The parameter to the raise statement is a
Paul> class object, which can be anything.

Sure, but in all but the rarest of cases the first arg to raise is a
specific exception, probably one of the standard exceptions. In the Python
code in the distribution (ignoring the test directory where all sorts of
mischief is done to stress things), here are the most common words following
"raise" where "raise" is the first word on the line:

% find . -name '*.py' \
> | egrep -v '\./test' \
> | xargs egrep '^ *raise ' \
> | awk '{print $3}' \
> | sed -e 's/[(,].*//' \
> | sort \
> | uniq -c \
> | sort -rn \
> | head -15

246 ValueError
227 aetools.Error
216 Error
124 TypeError
101 error
75 RuntimeError
53 IOError
36 NotImplementedError
36 ImportError
36 EOFError
31 SyntaxError
23 KeyError
23 AttributeError
22 DistutilsPlatformError
21 UnicodeError

Without checking, my guess is that #5 ("error") is one of a handful of
exception classes defined at module scope (ftplib, anydbm, sre_constants,
poplib, among others all define such an exception class), and not a variable
that accepts multiple values as in your example.

In short, while not perfect, simply grepping for '^ *(class|def|raise) ' and
printing the first and second words of each output line would probably give
you a pretty good idea of what gets raised where.

Skip
 
Reply With Quote
 
Roy Smith
Guest
Posts: n/a
 
      02-02-2005
In article <(E-Mail Removed)>,
System Administrator <(E-Mail Removed)> wrote:
>
> >> Does anybody know of a tool that can tell me all possible exceptions
> >> that might occur in each line of code? What I'm hoping to find is
> >> something like the following:

>
> Paul> That is impossible. The parameter to the raise statement is a
> Paul> class object, which can be anything.
>
>Sure, but in all but the rarest of cases the first arg to raise is a
>specific exception, probably one of the standard exceptions. In the Python
>code in the distribution (ignoring the test directory where all sorts of
>mischief is done to stress things), here are the most common words following
>"raise" where "raise" is the first word on the line:
>
> % find . -name '*.py' \
> > | egrep -v '\./test' \
> > | xargs egrep '^ *raise ' \
> > | awk '{print $3}' \
> > | sed -e 's/[(,].*//' \
> > | sort \
> > | uniq -c \
> > | sort -rn \
> > | head -15

> 246 ValueError
> 227 aetools.Error
> 216 Error
> 124 TypeError
> 101 error
> 75 RuntimeError
> 53 IOError
> 36 NotImplementedError
> 36 ImportError
> 36 EOFError
> 31 SyntaxError
> 23 KeyError
> 23 AttributeError
> 22 DistutilsPlatformError
> 21 UnicodeError


It's kind of interesting (scarry?) that in roughly 20% of the cases
nothing more specific than Error is raised.
 
Reply With Quote
 
Peter Hansen
Guest
Posts: n/a
 
      02-02-2005
Roy Smith wrote:
> Skip Montanaro wrote:
>> 246 ValueError
>> 227 aetools.Error
>> 216 Error
>> 124 TypeError
>> 101 error
>> 75 RuntimeError
>> 53 IOError
>> 36 NotImplementedError
>> 36 ImportError
>> 36 EOFError
>> 31 SyntaxError
>> 23 KeyError
>> 23 AttributeError
>> 22 DistutilsPlatformError
>> 21 UnicodeError

>
> It's kind of interesting (scarry?) that in roughly 20% of the cases
> nothing more specific than Error is raised.


(In case someone reading doesn't know) there isn't actually
an "Error" in the standard set of exceptions. It seems
likely that pretty much all of those uses are actually
module-specific Errors, and as such they are probably much
more specific than the unadorned name might imply.

Also, when one is trying to pick an appropriate exception
to raise, it is often the case that none of the standard
exceptions seems appropriate. In those cases (although I
personally would prefer a different name than "Error")
there's often no good alternative to making your own
unique module-specific Exception subclass.

-Peter
 
Reply With Quote
 
Skip Montanaro
Guest
Posts: n/a
 
      02-02-2005

>> 246 ValueError
>> 227 aetools.Error
>> 216 Error
>> 124 TypeError
>> 101 error
>> 75 RuntimeError
>> 53 IOError
>> 36 NotImplementedError
>> 36 ImportError
>> 36 EOFError
>> 31 SyntaxError
>> 23 KeyError
>> 23 AttributeError
>> 22 DistutilsPlatformError
>> 21 UnicodeError


Roy> It's kind of interesting (scarry?) that in roughly 20% of the cases
Roy> nothing more specific than Error is raised.

Not really. You might have code in a module named mod like this:

class Error(Exception): pass

then later:

def foo(...):
... blah blah blah ...
if condition:
raise Error, "hey dummy!"

The caller might look like:

import mod

...

try:
mod.foo()
except mod.Error, msg:
print msg

That said, the tendency for many newer modules seems to be to discriminate
exceptions based on type (look at urllib2 for example) while older modules
tended to have just a single exception they raised (look at ftplib). I
suspect that has something to do with whether the module was originally
written before or after Python introduced class exceptions.

Skip
 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
code auditing tools for dot.net? Elhanan ASP .Net 0 10-27-2009 08:01 PM
IT auditing tool SC Magazine Review maria.wyne@gmail.com Wireless Networking 0 08-16-2008 07:37 AM
Auditing C code CptDondo C Programming 9 09-24-2006 07:13 AM
Auditing question for the 70-270 MS book jones_net MCSE 2 10-22-2003 08:20 AM



Advertisments