Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Python > Re: threading support in python

Reply
Thread Tools

Re: threading support in python

 
 
Jean-Paul Calderone
Guest
Posts: n/a
 
      09-06-2006
On 05 Sep 2006 17:31:11 -0700, Paul Rubin <"http://phr.cx"@nospam.invalid> wrote:
>
> def func(filename):
> with open(filename) as f:
> do_something_with(f)
> # f definitely gets closed when the "with" block exits
>
>which more explicitly shows the semantics actually desired. Not that
>"huge" a benefit as far as I can tell. Lisp programmers have gotten
>along fine without it for 40+ years...


Uh yea. No lisp programmer has ever written a with-* function... ever.

Jean-Paul
 
Reply With Quote
 
 
 
 
Paul Rubin
Guest
Posts: n/a
 
      09-06-2006
Jean-Paul Calderone <(E-Mail Removed)> writes:
> >which more explicitly shows the semantics actually desired. Not that
> >"huge" a benefit as far as I can tell. Lisp programmers have gotten
> >along fine without it for 40+ years...

>
> Uh yea. No lisp programmer has ever written a with-* function... ever.


The context was Lisp programmers have gotten along fine without
counting on the refcounting GC semantics that sjdevnull advocates
Python stay with. GC is supposed to make it look like every object
stays around forever, and any finalizer that causes an explicit
internal state change in an extant object (like closing a file or
socket) is not in the GC spirit to begin with.
 
Reply With Quote
 
 
 
 
sjdevnull@yahoo.com
Guest
Posts: n/a
 
      09-06-2006
Paul Rubin wrote:
> Jean-Paul Calderone <(E-Mail Removed)> writes:
> > >which more explicitly shows the semantics actually desired. Not that
> > >"huge" a benefit as far as I can tell. Lisp programmers have gotten
> > >along fine without it for 40+ years...

> >
> > Uh yea. No lisp programmer has ever written a with-* function... ever.

>
> The context was Lisp programmers have gotten along fine without
> counting on the refcounting GC semantics that sjdevnull advocates
> Python stay with. GC is supposed to make it look like every object
> stays around forever, and any finalizer that causes an explicit
> internal state change in an extant object (like closing a file or
> socket) is not in the GC spirit to begin with.


I disagree, strongly. If you want "every object stays around forever"
semantics, you can just not free anything. GC is actually supposed to
free things that are unreachable at least when memory becomes tight,
and nearly every useful garbage collected language allows destructors
that could have effects visible to the rest of the program. Reference
counting allows more deterministic semantics that can eliminate
repeating scope information multiple times.

 
Reply With Quote
 
Steve Holden
Guest
Posts: n/a
 
      09-06-2006
http://www.velocityreviews.com/forums/(E-Mail Removed) wrote:
> Paul Rubin wrote:
>
>>Jean-Paul Calderone <(E-Mail Removed)> writes:
>>
>>>>which more explicitly shows the semantics actually desired. Not that
>>>>"huge" a benefit as far as I can tell. Lisp programmers have gotten
>>>>along fine without it for 40+ years...
>>>
>>>Uh yea. No lisp programmer has ever written a with-* function... ever.

>>
>>The context was Lisp programmers have gotten along fine without
>>counting on the refcounting GC semantics that sjdevnull advocates
>>Python stay with. GC is supposed to make it look like every object
>>stays around forever, and any finalizer that causes an explicit
>>internal state change in an extant object (like closing a file or
>>socket) is not in the GC spirit to begin with.

>
>
> I disagree, strongly. If you want "every object stays around forever"
> semantics, you can just not free anything. GC is actually supposed to
> free things that are unreachable at least when memory becomes tight,
> and nearly every useful garbage collected language allows destructors
> that could have effects visible to the rest of the program. Reference
> counting allows more deterministic semantics that can eliminate
> repeating scope information multiple times.
>

Clearly you guys are determined to disagree. It seemed obvious to me
that Paul's reference to making it "look like every object stays around
forever" doesn't exclude their being garbage-collected once the program
no longer contains any reference to them.

You simplify the problems involved with GC-triggered destructors to the
point of triviality. There are exceedingly subtle and difficult issues
here: read some of the posts to the python-dev list about such issues
and then see if you still feel the same way.

regards
Steve
--
Steve Holden +44 150 684 7255 +1 800 494 3119
Holden Web LLC/Ltd http://www.holdenweb.com
Skype: holdenweb http://holdenweb.blogspot.com
Recent Ramblings http://del.icio.us/steve.holden

 
Reply With Quote
 
sjdevnull@yahoo.com
Guest
Posts: n/a
 
      09-06-2006
Steve Holden wrote:
> (E-Mail Removed) wrote:
> > Paul Rubin wrote:
> >
> >>Jean-Paul Calderone <(E-Mail Removed)> writes:
> >>
> >>>>which more explicitly shows the semantics actually desired. Not that
> >>>>"huge" a benefit as far as I can tell. Lisp programmers have gotten
> >>>>along fine without it for 40+ years...
> >>>
> >>>Uh yea. No lisp programmer has ever written a with-* function... ever.
> >>
> >>The context was Lisp programmers have gotten along fine without
> >>counting on the refcounting GC semantics that sjdevnull advocates
> >>Python stay with. GC is supposed to make it look like every object
> >>stays around forever, and any finalizer that causes an explicit
> >>internal state change in an extant object (like closing a file or
> >>socket) is not in the GC spirit to begin with.

> >
> >
> > I disagree, strongly. If you want "every object stays around forever"
> > semantics, you can just not free anything. GC is actually supposed to
> > free things that are unreachable at least when memory becomes tight,
> > and nearly every useful garbage collected language allows destructors
> > that could have effects visible to the rest of the program. Reference
> > counting allows more deterministic semantics that can eliminate
> > repeating scope information multiple times.
> >

> Clearly you guys are determined to disagree. It seemed obvious to me
> that Paul's reference to making it "look like every object stays around
> forever" doesn't exclude their being garbage-collected once the program
> no longer contains any reference to them.
>
> You simplify the problems involved with GC-triggered destructors to the
> point of triviality. There are exceedingly subtle and difficult issues
> here: read some of the posts to the python-dev list about such issues


No doubt that it's hard. On the other hand, current CPython captures
programmer-friendly behavior quite well. My main assertions are that:
1. Saying that GC is just freeing memory after it won't be referenced
anymore is disingenuous; it is _already_ common practice in Python (and
other languages) for destructors to close files, sockets, and otherwise
deallocate non-memory resources.
2. The ref-counting semantics are extremely valuable to the programmer.
Throwing them out without careful consideration is a bad
idea--ref-counting is _not_ simply one GC implementation among many, it
actually offers useful semantics and the cost of giving up those
semantics should be considered before throwing out refcounting.

I'm actually willing to be convinced on (2); I think that what
ref-counting offers is a massive improvement over nondeterministic GC,
and it seems that refcounting has historically been supportable, but if
there are real tangible benefits to python programmers from eliminating
it that outweigh the niceties of deterministic GC, then I'd be okay
with sacrificing it. It just seems like people are very cavalier about
giving up something that is a very nice feature in order to make other
implementations simpler.

(1) I think is here to stay, if you're going to tell programmers that
their destructors can't make program-visible changes (e.g. closing the
database connection when a dbconn is destroyed), that's a _huge_ change
from current practice that needs serious debate.

 
Reply With Quote
 
Paul Rubin
Guest
Posts: n/a
 
      09-06-2006
"(E-Mail Removed)" <(E-Mail Removed)> writes:
> Throwing them out without careful consideration is a bad
>idea--ref-counting is _not_ simply one GC implementation among many, it
>actually offers useful semantics and the cost of giving up those
>semantics should be considered before throwing out refcounting.


It's too late to consider anything before throwing out refcounting.
Refcounting has already been thrown out (in Jython, IronPython, and
maybe PyPy). It's just an implementation artifact of CPython and MANY
other language implementations have gotten along perfectly well
without it.

> (1) I think is here to stay, if you're going to tell programmers that
> their destructors can't make program-visible changes (e.g. closing the
> database connection when a dbconn is destroyed), that's a _huge_ change
> from current practice that needs serious debate.


We had that debate already (PEP 343). Yes, there is some sloppy
current practice by CPython users that relies on the GC to close the
db conn. That practice already fails in several other Python
implementations and with PEP 343, we now have a clean way to fix it.
I don't understand why you're so fixated on keeping the sloppy method
around. The benefit is marginal at best. If you want stack-like
deallocation of something, ask for it explicitly.
 
Reply With Quote
 
sjdevnull@yahoo.com
Guest
Posts: n/a
 
      09-06-2006
Paul Rubin wrote:
> "(E-Mail Removed)" <(E-Mail Removed)> writes:
> > (1) I think is here to stay, if you're going to tell programmers that
> > their destructors can't make program-visible changes (e.g. closing the
> > database connection when a dbconn is destroyed), that's a _huge_ change
> > from current practice that needs serious debate.

>
> We had that debate already (PEP 343). Yes, there is some sloppy
> current practice by CPython users that relies on the GC to close the
> db conn.


This point is unrelated to with or ref-counting. Even the standard
library will close file objects when they are GC'd. If this is not
acceptable, it's a major change; that's why I say (1) is here to stay.
But I think we're misunderstanding each other somehow on this point (I
don't think you're saying that the standard library is sloppily coded
in this regard), I just don't know how.

 
Reply With Quote
 
Paul Rubin
Guest
Posts: n/a
 
      09-06-2006
"(E-Mail Removed)" <(E-Mail Removed)> writes:
> > We had that debate already (PEP 343). Yes, there is some sloppy
> > current practice by CPython users that relies on the GC to close the
> > db conn.

>
> This point is unrelated to with or ref-counting. Even the standard
> library will close file objects when they are GC'd.


I don't think so. AFAIK, there is no such thing as "when they are
GC'd" in the language spec. There is, at best, "if they are GC'd",
not "when". You are not guaranteed that GC ever takes place. At
best, those destructors run when the application totally shuts down,
as the process is about to exit, but I'm not sure whether the spec
promises even that.

> If this is not acceptable, it's a major change;


It is guaranteeing GC running at any particular time that would be a
major change. There is no such guarantee right now; there is just an
implementation artifact in CPython that has led to careless habits
among some users.
 
Reply With Quote
 
sjdevnull@yahoo.com
Guest
Posts: n/a
 
      09-06-2006
Paul Rubin wrote:
> "(E-Mail Removed)" <(E-Mail Removed)> writes:
> > > We had that debate already (PEP 343). Yes, there is some sloppy
> > > current practice by CPython users that relies on the GC to close the
> > > db conn.

> >
> > This point is unrelated to with or ref-counting. Even the standard
> > library will close file objects when they are GC'd.

>
> I don't think so. AFAIK, there is no such thing as "when they are
> GC'd" in the language spec.


If they don't get GC'd, then "when they are GC'd" is never. The point
is that the standard library _does_ close files and take other
program-visible actions in __del__ methods; I'm unclear on if you think
that doing so is actually sloppy practice (as opposed to users of the
standard library relying on the GC running in some deterministic manner
or even at all, which you clearly do think is sloppy practice).

I originally thought that was what you meant when you said that "GC is
supposed to make it look like every object stays around forever, and
any finalizer that causes an explicit internal state change in an
extant object (like closing a file or socket) is not in the GC spirit
to begin with." but going back and reading it I'm not sure.

 
Reply With Quote
 
Paul Rubin
Guest
Posts: n/a
 
      09-06-2006
"(E-Mail Removed)" <(E-Mail Removed)> writes:
> If they don't get GC'd, then "when they are GC'd" is never. The point
> is that the standard library _does_ close files and take other
> program-visible actions in __del__ methods; I'm unclear on if you think
> that doing so is actually sloppy practice


I don't know if I'd say "sloppy"; it's certainly messy and can lead to
complex bugs, but there are pragmatic reasons for wanting to do it in
some situations and it can be used to good purpose if you're careful.
But I think people sometimes expect too much from it, as Steve Holden
indicated.

It should not be compared with C++ destructors, since C++ deallocation
is explicit. I'm not sure what other languages with actual GC support
anything like it. I implemented it in Lisp once but found it
disconcerting since some of the stuff removed by GC in the system I
was working on were no-longer-referenced graphics objects on the
screen, which would thereby wink out of existence at surprising times.
I concluded that the application should keep track of stuff like that
instead of tossing them into the air for the GC to clean up.

> I originally thought that was what you meant when you said that "GC is
> supposed to make it look like every object stays around forever, and
> any finalizer that causes an explicit internal state change in an
> extant object (like closing a file or socket) is not in the GC spirit
> to begin with." but going back and reading it I'm not sure.


GC is supposed to just release resources that are no longer referenced
(since you can't tell whether they're still around, you can act as if
they are always stay around). For files, maybe closing them is
ok--the file abstraction mimics synchronous i/o and in the idealized
version, closing them is a do-nothing and the closure is undetectable.
For sockets (which are supposed to send particular messages over the
wire when the socket shuts down) it's less appealing.
 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Re: threading in PyQt vs threading in standard library Steven Woody Python 0 01-09-2009 07:48 AM
threading in PyQt vs threading in standard library Steven Woody Python 0 01-09-2009 07:15 AM
Cooperative threading preemptive threading - a bit confused failure_to@yahoo.co.uk Java 9 12-29-2007 01:10 AM
threading support in python km Python 41 09-06-2006 06:39 PM
Re: threading support in python sjdevnull@yahoo.com Python 4 09-06-2006 07:46 AM



Advertisments