Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Python > does python have useless destructors?

Reply
Thread Tools

does python have useless destructors?

 
 
Michael P. Soulier
Guest
Posts: n/a
 
      06-09-2004
Greetings,

I recently had a friend tell me that because the Python spec does not
guarantee that an object will necessary be destroyed when all references
to it are gone, that desctructors in Python are not reliably called, and
thus useless. As I found it difficult to believe that Python would
include destructors for apparently no reason, I thought I should ask for
clarification here.

Can anyone say yay or nay on this issue?

I believe his comments stem from a couple of references.

http://docs.python.org/ref/customization.html#l2h-174
http://docs.python.org/ref/objects.html#l2h-18

Specifically, this paragraph in the second link:

Some objects contain references to ``external'' resources such as
open files or windows. It is understood that these resources are
freed when the object is garbage-collected, but since garbage
collection is not guaranteed to happen, such objects also provide an
explicit way to release the external resource, usually a close()
method. Programs are strongly recommended to explicitly close such
objects. The `try...finally' statement provides a convenient way to
do this.

So putting a close of some external resource in a destructor would be a
bad idea, apparently. As this is the kind of thing I typically use
destructors in OO programming for, I found this quite surprising.

Regards,
Mike

--
Michael P. Soulier <msoulier@digitaltorqueNOSPAM_.ca>
The major advances in civilization are processes that all but wreck the
societies in which they occur.
-- Albert North Whitehead
 
Reply With Quote
 
 
 
 
Donn Cave
Guest
Posts: n/a
 
      06-09-2004
In article <(E-Mail Removed) >,
"Michael P. Soulier" <(E-Mail Removed)._nospam> wrote:

> I recently had a friend tell me that because the Python spec does not
> guarantee that an object will necessary be destroyed when all references
> to it are gone, that desctructors in Python are not reliably called, and
> thus useless. As I found it difficult to believe that Python would
> include destructors for apparently no reason, I thought I should ask for
> clarification here.
>
> Can anyone say yay or nay on this issue?
>
> I believe his comments stem from a couple of references.
>
> http://docs.python.org/ref/customization.html#l2h-174
> http://docs.python.org/ref/objects.html#l2h-18
>
> Specifically, this paragraph in the second link:
>
> Some objects contain references to ``external'' resources such as
> open files or windows. It is understood that these resources are
> freed when the object is garbage-collected, but since garbage
> collection is not guaranteed to happen, such objects also provide an
> explicit way to release the external resource, usually a close()
> method. Programs are strongly recommended to explicitly close such
> objects. The `try...finally' statement provides a convenient way to
> do this.
>
> So putting a close of some external resource in a destructor would be a
> bad idea, apparently. As this is the kind of thing I typically use
> destructors in OO programming for, I found this quite surprising.


To answer your question, no, I don't think anyone can say
yea or nay on this issue.

There are two not-guaranteed issues with finalization.

- Reference cycles can prevent immediate finalization.
If an object holds a reference to itself, however
indirectly, it won't be deleted automatically and has
to be recovered through garbage collection, and garbage
collection won't delete instances with a __del__ method,
among other things.

- The Java implementation can't implement immediate
finalization because Java doesn't have it. In general,
Python's storage model isn't fully defined and its semantics
may vary according to the host language.

On the other hand, while we can't completely resolve this
problem, I think the text you quota above errs in trying
to paint it as a non-problem. try..finally is certainly
not a convenient substitute for guaranteed finalization.
The immediate finalization normally supported in Python
is a very powerful, elegant programming feature. For just
the same reasons that automatic control of memory allocation
is powerful and elegant -- memory is one of the resources
your objects use.

So yes, you can depend on immediate finalization in many
common situations, but you have to accept the liability of
a dependence on reference non-circularity and on the C Python.

Donn Cave, http://www.velocityreviews.com/forums/(E-Mail Removed)
 
Reply With Quote
 
 
 
 
John J. Lee
Guest
Posts: n/a
 
      06-09-2004
Donn Cave <(E-Mail Removed)> writes:
[...snip excellent answer...]
> In article <(E-Mail Removed) >,
> So yes, you can depend on immediate finalization in many
> common situations, but you have to accept the liability of
> a dependence on reference non-circularity and on the C Python.


Adding __del__ methods also, IIRC, has the disadvantage that it messes
up the GC that does reference-cycle collection. When __del__ was
introduced, Python had no automatic cycle GC.

On balance, considering all these points, I never write __del__
methods any more, and I always (well, almost always) explictly close
files and other external resources. Other people disagree, preferring
to avoid reference cycles (eg. using the weakref module) and to rule
out Jython.


John
 
Reply With Quote
 
Michael P. Soulier
Guest
Posts: n/a
 
      06-10-2004
On Wed, 09 Jun 2004 15:00:32 -0700, Donn Cave <(E-Mail Removed)>
wrote:
>
> On the other hand, while we can't completely resolve this
> problem, I think the text you quota above errs in trying
> to paint it as a non-problem. try..finally is certainly
> not a convenient substitute for guaranteed finalization.
> The immediate finalization normally supported in Python
> is a very powerful, elegant programming feature. For just
> the same reasons that automatic control of memory allocation
> is powerful and elegant -- memory is one of the resources
> your objects use.
>
> So yes, you can depend on immediate finalization in many
> common situations, but you have to accept the liability of
> a dependence on reference non-circularity and on the C Python.


As soon as I heard about not being able to trust destructors, I was
shocked, because I do rely on guaranteed finalization, and I also
consider it to be very elegant. If the only situations where I cannot
trust it are with Jython, and circular references, then I have no issue.

However, consider the code...

myfile = open("myfilepath", "w")
myfile.write(reallybigbuffer)
myfile.close()

If the write fails due to a lack of disk space, the exception will
prevent the explicit close() from happening. Now, if myfile goes out of
scope, I would hope that the file object is destroyed and thus the file
is closed, but after reading the python documentation, it seems that
the only way to guarantee closure of the file is using the mentioned
try/finally construct...

try:
myfile = open("myfilepath", "w")
myfile.write(reallybigbuffer)
finally:
myfile.close()

Also, there is the case of Python not guaranteeing the __del__ method of
an object being called if the object is still alive when the process
exits. For files, I don't care, since the filehandles will be returned
by the OS, but for some external resources, this would be a big issue.
FIFOs and shared memory segments, for example.

Mike

--
Michael P. Soulier <(E-Mail Removed)>
The major advances in civilization are processes that all but wreck the
societies in which they occur.
-- Albert North Whitehead
 
Reply With Quote
 
Peter Hansen
Guest
Posts: n/a
 
      06-10-2004
Michael P. Soulier wrote:

I'll limit my response to point out that this code:

> myfile = open("myfilepath", "w")
> myfile.write(reallybigbuffer)
> myfile.close()


.... immediately raises a warning flag in the mind of an
experienced Python programmer.

This code, on the other hand:

> try:
> myfile = open("myfilepath", "w")
> myfile.write(reallybigbuffer)
> finally:
> myfile.close()


.... "feels" just right. This is how you do it when you
want to be sure the file is closed, regardless of
other considerations like garbage collection, etc.

It is simple, clean, and most of all, very explicit.

-Peter
 
Reply With Quote
 
Michael Geary
Guest
Posts: n/a
 
      06-10-2004
Peter Hansen wrote:
> I'll limit my response to point out that this code:
>
> > myfile = open("myfilepath", "w")
> > myfile.write(reallybigbuffer)
> > myfile.close()

>
> ... immediately raises a warning flag in the mind of an
> experienced Python programmer.
>
> This code, on the other hand:
>
> > try:
> > myfile = open("myfilepath", "w")
> > myfile.write(reallybigbuffer)
> > finally:
> > myfile.close()

>
> ... "feels" just right. This is how you do it when you
> want to be sure the file is closed, regardless of
> other considerations like garbage collection, etc.
>
> It is simple, clean, and most of all, very explicit.


And it has a bug.

What if "myfilepath" is not writable? The open() call raises an exception,
which jumps to the finally: block, where myfile.close() fails because myfile
is undefined.

What would be the cleanest way to code this to handle that error situation?

-Mike


 
Reply With Quote
 
Terry Reedy
Guest
Posts: n/a
 
      06-10-2004

"Donn Cave" <(E-Mail Removed)> wrote in message
news:(E-Mail Removed)...
> In article <(E-Mail Removed) >,
> "Michael P. Soulier" <(E-Mail Removed)._nospam> wrote:
>
> > I recently had a friend tell me that because the Python spec does not
> > guarantee that an object will necessary be destroyed when all

references
> > to it are gone, that desctructors in Python are not reliably called,

and
> > thus useless.


To me, this is useless over-dramatization Is your car useless because
there is no guarantee that the engine will never quit? or the brakes fail?

While aspects of memory management are system dependent and outside of
Python's control, programs running on standard hardware with the CPython
interpreter do what one can expect: memory is freed for reuse by the
interpreter (but not necessarily for resuse by other programs, as some
mistakenly expect).

>> As I found it difficult to believe that Python would
> > include destructors for apparently no reason,


And they are not. Under the conditions stated above, deletion works. But
Guido did not want to restrict Python to C implementation on standard
systems.

> - Reference cycles can prevent immediate finalization.
> If an object holds a reference to itself, however
> indirectly, it won't be deleted automatically and has
> to be recovered through garbage collection, and garbage
> collection won't delete instances with a __del__ method,
> among other things.


This is admittedly a nuisance. Howver, the original solution was reference
counting only. Programmers had to explictly break loops to reclaim
resources. This option remains available.

When cyclic gc was introduced, I believe there was no such restriction.
The result was indeterminate chaos and sometime bad behavior. After trying
several fixes, the present rule was introduced. Of course, this made
cyclic gc less useful than intended, but seg faults are a no-no. No one
has volunteered anything better since.

Terry J. Reedy




 
Reply With Quote
 
Donn Cave
Guest
Posts: n/a
 
      06-10-2004
Quoth "Michael P. Soulier" <(E-Mail Removed)._nospam>:
....
| However, consider the code...
|
| myfile = open("myfilepath", "w")
| myfile.write(reallybigbuffer)
| myfile.close()
|
| If the write fails due to a lack of disk space, the exception will
| prevent the explicit close() from happening. Now, if myfile goes out of
| scope, I would hope that the file object is destroyed and thus the file
| is closed, but after reading the python documentation, it seems that
| the only way to guarantee closure of the file is using the mentioned
| try/finally construct...
|
| try:
| myfile = open("myfilepath", "w")
| myfile.write(reallybigbuffer)
| finally:
| myfile.close()

I think here we're talking about the problem where an exception takes
a reference to local variables. The other solution, a bit of a hack
but it could be more convenient depending on your application, is to
release the traceback (sys.exc_traceback = None) in the handler.

Donn Cave, (E-Mail Removed)
 
Reply With Quote
 
Hannu Kankaanp??
Guest
Posts: n/a
 
      06-10-2004
Peter Hansen <(E-Mail Removed)> wrote in message news:<(E-Mail Removed)>...
> This code, on the other hand:
>
> > try:
> > myfile = open("myfilepath", "w")
> > myfile.write(reallybigbuffer)
> > finally:
> > myfile.close()

>
> ... "feels" just right. This is how you do it when you
> want to be sure the file is closed, regardless of
> other considerations like garbage collection, etc.
>
> It is simple, clean, and most of all, very explicit.


It's not that simple when you compare it to C++ RAII idiom,
and the above code is actually wrong. If open() raises an
exception, myfile hasn't yet been assigned and myfile.close()
will raise another, unwanted exception of type "NameError".
The correct idiom is:

myfile = file("myfilepath", "w")
try:
myfile.write(reallybigbuffer)
finally:
myfile.close()

I'd very much prefer something simpler that leaves no room
for errors, e.g.

with myfile = file("myfilepath", "w"):
myfile.write(reallybigbuffer)

(this, and more generic solutions with code blocks have been
suggested here many times)
 
Reply With Quote
 
Michael Geary
Guest
Posts: n/a
 
      06-10-2004
Hannu Kankaanp??
> The correct idiom is:
>
> myfile = file("myfilepath", "w")
> try:
> myfile.write(reallybigbuffer)
> finally:
> myfile.close()
>
> I'd very much prefer something simpler that leaves no room
> for errors, e.g.
>
> with myfile = file("myfilepath", "w"):
> myfile.write(reallybigbuffer)
>
> (this, and more generic solutions with code blocks have been
> suggested here many times)


That's just like the way you would code it in Ruby:

File.open( 'myfilepath', 'w' ) do |myfile|
myfile.write( reallybigbuffer )
end

-Mike


 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Useless thread about some useless statistics Daniel Nogradi Python 0 11-15-2006 11:33 PM
RE: does python have useless destructors? Tim Peters Python 7 06-18-2004 08:34 AM
RE: does python have useless destructors? Delaney, Timothy C (Timothy) Python 4 06-14-2004 06:38 PM
RE: does python have useless destructors? Tim Peters Python 2 06-12-2004 12:08 AM
RE: does python have useless destructors? Robert Brewer Python 1 06-10-2004 04:29 PM



Advertisments