Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Python > Destructors and exceptions

Reply
Thread Tools

Destructors and exceptions

 
 
David Turner
Guest
Posts: n/a
 
      06-07-2004
Hi all

I noticed something interesting while testing some RAII concepts
ported from C++ in Python. I haven't managed to find any information
about it on the web, hence this post.

The problem is that when an exception is raised, the destruction of
locals appears to be deferred to program exit. Am I missing
something? Is this behaviour by design? If so, is there any reason
for it? The only rationale I can think of is to speed up exception
handling; but as this approach breaks many safe programming idioms, I
see it as a poor trade.

Here is the code in question:

------------------------------------------
class Foo:
def __init__(self):
print "--foo %s created" % id(self)
def __del__(self):
print "--foo %s destroyed" % id(self)

def normal_exit():
print "normal_exit starts"
x = Foo()
print "normal_exit ends"

def premature_exit():
print "premature_exit starts"
x = Foo()
return 0
print "premature_exit ends"

def exceptional_exit():
print "exceptional_exit starts"
x = Foo()
raise "oops"
print "exceptional_exit ends"

if __name__ == "__main__":
print "main block starts"
try:
normal_exit()
premature_exit()
exceptional_exit()
except:
print "exception"
print "main block ends"
------------------------------------------

The output I get is:

------------------------------------------
main block starts
normal_exit starts
--foo 141819532 created
normal_exit ends
--foo 141819532 destroyed
premature_exit starts
--foo 141819532 created
--foo 141819532 destroyed
exceptional_exit starts
--foo 141819532 created
exception
main block ends
--foo 141819532 destroyed
------------------------------------------

....which indicates to me that the destruction of the local in
exceptional_exit() only happens when the program exits. Surely it
should occur at the point at which the exception is raised?

Regards
David Turner
 
Reply With Quote
 
 
 
 
Terry Reedy
Guest
Posts: n/a
 
      06-07-2004

"David Turner" <(E-Mail Removed)> wrote in message
news:(E-Mail Removed) om...
> The problem is that when an exception is raised, the destruction of
> locals appears to be deferred to program exit. Am I missing
> something?


When the last reference to an object disappears, the interpreter *may* but
is *not required* to forget or destruct the object, if indeed the concept
is meaningful. What happens is implementation dependent. CPython usually
cleans up immediately, which is sooner than with Jython. I am not sure
what human interpreters do. Write once, never erase storage of objects
(keeping a complete audit trail of objects created) would also be legal.

If you want to force the issue, give your class a close method, as with
file objects, and call it explicitly. Then code in try:finally:, which is
designed for this purpose.
try:
process
finally:
cleanup

Terry J. Reedy




 
Reply With Quote
 
 
 
 
Humpty Dumpty
Guest
Posts: n/a
 
      06-08-2004

"David Turner" <(E-Mail Removed)> wrote in message
news:(E-Mail Removed) om...
>


> The problem is that when an exception is raised, the destruction of
> locals appears to be deferred to program exit. Am I missing
> something? Is this behaviour by design? If so, is there any reason
> for it? The only rationale I can think of is to speed up exception
> handling; but as this approach breaks many safe programming idioms, I
> see it as a poor trade.



There is no Python equivalent of C++'s "destructor garanteed to be called
upon scope exit", for a couple of reasons: scope exit only destroys
references to objects, not the objects themselves; destruction of objects is
left to the garbage collector and you have no influence on it. In
particular, the gc is not required to release resources, so finalizers (the
__del__ method, closest to C++'s destructor) may not get called. This means
__del__ is pretty much useless (AFAIMC), and you can't rely on them being
called before program exit (or ever, for that matter).

I agree, it is a real pitty that Python doesn't have a way of doing what you
mention, other than try-finally, which makes code more difficult to read. A
new specifier, e.g. "scoped", would be a required addtion to Python:
interpreter would garantee that __del__ of scoped objects would be called on
scope exit, and raise an exception if attempt to alias.

Oliver


 
Reply With Quote
 
David Turner
Guest
Posts: n/a
 
      06-08-2004
Hi

> If you want to force the issue, give your class a close method, as with
> file objects, and call it explicitly. Then code in try:finally:, which is
> designed for this purpose.
> try:
> process
> finally:
> cleanup
>


That's a great shame. "Finally" is of course susceptible to the
"forgetful programmer" syndrom. It also pollutes the code with what I
consider to be implementation details -- for example, the fact that a
file must be closed is a characteristic of the file object, and only
accidentally relevant to the _use_ of files. Python would be a much
stronger language if it could guarantee deterministic destruction of
at least those objects which have a __del__ method. Can anyone point
out any technical reasons why such a feature should not be included in
the language?

At any rate, I wonder if there is an another solution to the problem
of resource management. Perhaps something along the lines of Ruby
closures? Any suggestions? There has to be something more elegant
(and safer!) than "finally".

Regards
David Turner
 
Reply With Quote
 
Dominic
Guest
Posts: n/a
 
      06-08-2004
David Turner wrote:
> Hi
>
>
>>If you want to force the issue, give your class a close method, as with
>>file objects, and call it explicitly. Then code in try:finally:, which is
>>designed for this purpose.
>>try:
>> process
>>finally:
>> cleanup
>>

>
>
> That's a great shame. "Finally" is of course susceptible to the
> "forgetful programmer" syndrom. It also pollutes the code with what I
> consider to be implementation details -- for example, the fact that a
> file must be closed is a characteristic of the file object, and only
> accidentally relevant to the _use_ of files. Python would be a much
> stronger language if it could guarantee deterministic destruction of

Well, there had been some discussion before. You can
wrap the open/close actions in a closure and write
something like with_file_do(myFunction).

Ciao,
Dominic

> at least those objects which have a __del__ method. Can anyone point
> out any technical reasons why such a feature should not be included in
> the language?
>
> At any rate, I wonder if there is an another solution to the problem
> of resource management. Perhaps something along the lines of Ruby
> closures? Any suggestions? There has to be something more elegant
> (and safer!) than "finally".
>
> Regards
> David Turner

 
Reply With Quote
 
Christos TZOTZIOY Georgiou
Guest
Posts: n/a
 
      06-08-2004
On 7 Jun 2004 07:51:23 -0700, rumours say that http://www.velocityreviews.com/forums/(E-Mail Removed)
(David Turner) might have written:

>Hi all
>
>I noticed something interesting while testing some RAII concepts
>ported from C++ in Python. I haven't managed to find any information
>about it on the web, hence this post.
>
>The problem is that when an exception is raised, the destruction of
>locals appears to be deferred to program exit. Am I missing
>something? Is this behaviour by design? If so, is there any reason
>for it? The only rationale I can think of is to speed up exception
>handling; but as this approach breaks many safe programming idioms, I
>see it as a poor trade.


[snip code]

ISTR that when an exception is raised, there is a reference to the code
frame (and therefore to its locals) kept somewhere; you can access this
info through the sys.exc_info call. Have you tried running
sys.exc_clear after the exception was raised?

Apart from that, advice from others not to rely on finalisation of
objects to clean up still applies.
--
TZOTZIOY, I speak England very best,
"I have a cunning plan, m'lord" --Sean Bean as Odysseus/Ulysses
 
Reply With Quote
 
David Turner
Guest
Posts: n/a
 
      06-08-2004
Hi

>
> There is no Python equivalent of C++'s "destructor garanteed to be called
> upon scope exit", for a couple of reasons: scope exit only destroys
> references to objects, not the objects themselves; destruction of objects is
> left to the garbage collector and you have no influence on it. In
> particular, the gc is not required to release resources, so finalizers (the
> __del__ method, closest to C++'s destructor) may not get called. This means
> __del__ is pretty much useless (AFAIMC), and you can't rely on them being
> called before program exit (or ever, for that matter).
>


In the documentation for the "gc" module, it states that objects with
a __del__ method are not subject to garbage collection; they are
collected using reference counting. Which means that one can rely on
locals (to which there is only one reference) being destroyed in a
predictable fashion -- but there's an exception. Section 3.1 of the
Python Reference Manual specifies that raising exceptions may keep
objects alive. What I take this to mean from an implementation point
of view is that the exception mechanism doesn't unwind scope.
Therefore, the destruction of locals in the presence of an exception
is deferred to global clean-up time when the program exits.

I can't think of any technical objections to having the exception
mechanism also release references to locals that are going to go out
of scope (unless one was planning to support resuming). Can you?

Regards
David Turner
 
Reply With Quote
 
Christos TZOTZIOY Georgiou
Guest
Posts: n/a
 
      06-08-2004
On 8 Jun 2004 02:44:04 -0700, rumours say that (E-Mail Removed)
(David Turner) might have written:

>I can't think of any technical objections to having the exception
>mechanism also release references to locals that are going to go out
>of scope (unless one was planning to support resuming). Can you?


Debugging.
--
TZOTZIOY, I speak England very best,
"I have a cunning plan, m'lord" --Sean Bean as Odysseus/Ulysses
 
Reply With Quote
 
Nick Jacobson
Guest
Posts: n/a
 
      06-08-2004
You know, I noticed this in the Python Reference Manual, p. 13, and
have been wondering about it.

"...note that catching an exception with a 'try...except' statement
may keep objects alive..."

No explanation is given, and I don't know why that's the case either.
But at least they're aware of it...HTH

--Nick

(E-Mail Removed) (David Turner) wrote in message news:<(E-Mail Removed). com>...
> Hi all
>
> I noticed something interesting while testing some RAII concepts
> ported from C++ in Python. I haven't managed to find any information
> about it on the web, hence this post.
>
> The problem is that when an exception is raised, the destruction of
> locals appears to be deferred to program exit. Am I missing
> something? Is this behaviour by design? If so, is there any reason
> for it? The only rationale I can think of is to speed up exception
> handling; but as this approach breaks many safe programming idioms, I
> see it as a poor trade.
>
> Here is the code in question:
>
> ------------------------------------------
> class Foo:
> def __init__(self):
> print "--foo %s created" % id(self)
> def __del__(self):
> print "--foo %s destroyed" % id(self)
>
> def normal_exit():
> print "normal_exit starts"
> x = Foo()
> print "normal_exit ends"
>
> def premature_exit():
> print "premature_exit starts"
> x = Foo()
> return 0
> print "premature_exit ends"
>
> def exceptional_exit():
> print "exceptional_exit starts"
> x = Foo()
> raise "oops"
> print "exceptional_exit ends"
>
> if __name__ == "__main__":
> print "main block starts"
> try:
> normal_exit()
> premature_exit()
> exceptional_exit()
> except:
> print "exception"
> print "main block ends"
> ------------------------------------------
>
> The output I get is:
>
> ------------------------------------------
> main block starts
> normal_exit starts
> --foo 141819532 created
> normal_exit ends
> --foo 141819532 destroyed
> premature_exit starts
> --foo 141819532 created
> --foo 141819532 destroyed
> exceptional_exit starts
> --foo 141819532 created
> exception
> main block ends
> --foo 141819532 destroyed
> ------------------------------------------
>
> ...which indicates to me that the destruction of the local in
> exceptional_exit() only happens when the program exits. Surely it
> should occur at the point at which the exception is raised?
>
> Regards
> David Turner

 
Reply With Quote
 
Duncan Booth
Guest
Posts: n/a
 
      06-08-2004
(E-Mail Removed) (Nick Jacobson) wrote in
news:(E-Mail Removed) om:

> You know, I noticed this in the Python Reference Manual, p. 13, and
> have been wondering about it.
>
> "...note that catching an exception with a 'try...except' statement
> may keep objects alive..."
>
> No explanation is given, and I don't know why that's the case either.
> But at least they're aware of it...HTH
>


When an exception is handled, you can access the stack traceback. This
contains references to all the local variables which were in scope at the
point when the exception was thrown. Normally these variables would be
destroyed when the functions return, but if there is a traceback
referencing them they stay alive as long as the traceback is accessible
which is usually until the next exception is thrown.

If you really want to be sure your objects are destroyed, then use 'del' in
the finally suite of a try..finally. This ensures that the traceback can't
be used to reference the objects that you deleted.
 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Exceptions in destructors Rennie deGraaf C++ 5 04-30-2007 09:07 AM
exceptions in diamond's destructors vgrebinski@gmail.com C++ 17 11-17-2005 10:54 PM
Checked exceptions vs unchecked exceptions Ahmed Moustafa Java 5 07-14-2004 01:46 PM
Custom exceptions -- inherit from exceptions.Exception? Paul Miller Python 3 11-12-2003 09:24 AM
C++ exceptions and destructors Magalie C++ 9 09-04-2003 09:23 PM



Advertisments