Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Python > PEP 288 ponderings

Reply
Thread Tools

PEP 288 ponderings

 
 
Steven Bethard
Guest
Posts: n/a
 
      01-02-2005
PEP 288 was mentioned in one of the lambda threads and so I ended up
reading it for the first time recently. I definitely don't like the
idea of a magical __self__ variable that isn't declared anywhere. It
also seemed to me like generator attributes don't really solve the
problem very cleanly. An example from the PEP[1]:

def mygen():
while True:
print __self__.data
yield None

g = mygen()
g.data = 1
g.next() # prints 1
g.data = 2
g.next() # prints 2

I looked in the archives but couldn't find a good discussion of why
setting an attribute on the generator is preferable to passing the
argument to next. Isn't this example basically equivalent to:

class mygen(object):
def next(self, data):
print data
return None

g = mygen()
g.next(1) # prints 1
g.next(2) # prints 2

Note that I didn't even define an __iter__ method since it's never used
in the example.

Another example from the PEP:

def filelike(packagename, appendOrOverwrite):
data = []
if appendOrOverwrite == 'w+':
data.extend(packages[packagename])
try:
while True:
data.append(__self__.dat)
yield None
except FlushStream:
packages[packagename] = data

ostream = filelike('mydest','w')
ostream.dat = firstdat; ostream.next()
ostream.dat = firstdat; ostream.next()
ostream.throw(FlushStream)

This could be rewritten as:

class filelike(object):
def __init__(self, packagename, appendOrOverwrite):
self.data = []
if appendOrOverwrite == 'w+':
self.data.extend(packages[packagename])
def next(self, dat):
self.data.append(dat)
return None
def close(self):
packages[packagename] = self.data

ostream = filelike('mydest','w')
ostream.next(firstdat)
ostream.next(firstdat)
ostream.close()

So, I guess I have two questions:

(1) What's the benefit of the generator versions of these functions over
the class-based versions?

(2) Since in all the examples there's a one-to-one correlation between
setting a generator attribute and calling the generator's next function,
aren't these generator attribute assignments basically just trying to
define the 'next' parameter list?

If this is true, I would have expected that a more useful idiom would
look something like:

def mygen():
while True:
data, = nextargs()
print data
yield None

g = mygen()
g.next(1) # prints 1
g.next(2) # prints 2

where the nextargs function retrieves the arguments of the most recent
call to the generator's next function.

With a little sys._getframe hack, you can basically get this behavior now:

py> class gen(object):
.... def __init__(self, gen):
.... self.gen = gen
.... def __iter__(self):
.... return self
.... def next(self, *args):
.... return self.gen.next()
.... @staticmethod
.... def nextargs():
.... return sys._getframe(2).f_locals['args']
....
py> def mygen():
.... while True:
.... data, = gen.nextargs()
.... print data
.... yield None
....
py> g = gen(mygen())
py> g.next(1)
1
py> g.next(2)
2

Of course, it's still a little magical, but I think I like it a little
better because you can see, looking only at 'mygen', when 'data' is
likely to change value...


Steve

[1] http://www.python.org/peps/pep-0288.html
 
Reply With Quote
 
 
 
 
Ian Bicking
Guest
Posts: n/a
 
      01-02-2005
Steven Bethard wrote:
> PEP 288 was mentioned in one of the lambda threads and so I ended up
> reading it for the first time recently. I definitely don't like the
> idea of a magical __self__ variable that isn't declared anywhere. It
> also seemed to me like generator attributes don't really solve the
> problem very cleanly. An example from the PEP[1]:
>
> def mygen():
> while True:
> print __self__.data
> yield None
>
> g = mygen()
> g.data = 1
> g.next() # prints 1
> g.data = 2
> g.next() # prints 2


I don't get why this isn't good enough:

def mygen(data):
while True:
print data[0]
yield None

data = [None]
g = mygen(data)
data[0] = 1
g.next()
data[0] = 1
g.next()

Using a one-element list is kind of annoying, because it isn't clear out
of context that it's just a way of creating shared state. But it's
okay, work right now, and provides the exact same functionality. The
exception part of PEP 288 still seems interesting.

--
Ian Bicking / http://www.velocityreviews.com/forums/(E-Mail Removed) / http://blog.ianbicking.org
 
Reply With Quote
 
 
 
 
Nick Coghlan
Guest
Posts: n/a
 
      01-02-2005
Ian Bicking wrote:
> Using a one-element list is kind of annoying, because it isn't clear out
> of context that it's just a way of creating shared state. But it's
> okay, work right now, and provides the exact same functionality.


Uh, isn't shared state what classes were invented for?

Py> class mygen(object):
.... def __init__(self, data):
.... self.data = data
.... def __iter__(self):
.... while 1:
.... print self.data
.... yield None
....
Py> g = mygen(0)
Py> giter = iter(g)
Py> giter.next()
0
Py> g.data = 1
Py> giter.next()
1

Cheers,
Nick.


--
Nick Coghlan | (E-Mail Removed) | Brisbane, Australia
---------------------------------------------------------------
http://boredomandlaziness.skystorm.net
 
Reply With Quote
 
Raymond Hettinger
Guest
Posts: n/a
 
      01-02-2005
[Steven Bethard]
> (1) What's the benefit of the generator versions of these functions over
> the class-based versions?


Generators are easier to write, are clearer, and run faster.

They automatically
* create a distinct generator-iterator object upon each invocation
* create the next() and idempotent __iter__() methods.
* raise StopIteration upon termination
* remain stopped if next() is called too many times
* save their own local variable state, avoiding the need for self.var references
* resume execution at the point of the last yield



> (2) Since in all the examples there's a one-to-one correlation between
> setting a generator attribute and calling the generator's next function,
> aren't these generator attribute assignments basically just trying to
> define the 'next' parameter list?


They are not the same. The generator needs some way to receive the values. The
function arguments cannot be used because they are needed to create the
generator-iterator. The yield statements likewise won't work because the first
yield is not encountered until well after the first next() call.

The given examples are minimal and are intended only to demonstrate the idea.



> I definitely don't like the
> idea of a magical __self__ variable that isn't declared anywhere.


It is no more magical than f.__name__ or f.__doc__ for functions. The concept
is almost identical to that for threading.local(). Also, the __self__ argument
is a non-issue because there are other alternate approaches such as providing a
function that retrieves the currently running generator. Which approach is
ultimately taken is a matter of aesthetics -- the PEP itself concentrates on the
core idea instead of debating syntax.

The real issue with PEP 288's idea for generator attributes is that the current
C implementation doesn't readily accommodate this change. Major surgery would
be required

The more important part of the PEP is the idea for generator exceptions. The
need arises in the context of flushing/closing resources upon generator
termination.



Raymond Hettinger


 
Reply With Quote
 
Steven Bethard
Guest
Posts: n/a
 
      01-02-2005
Raymond Hettinger wrote:
> [Steven Bethard]
>>(2) Since in all the examples there's a one-to-one correlation between
>>setting a generator attribute and calling the generator's next function,
>>aren't these generator attribute assignments basically just trying to
>>define the 'next' parameter list?

>
> They are not the same. The generator needs some way to receive the values. The
> function arguments cannot be used because they are needed to create the
> generator-iterator. The yield statements likewise won't work because the first
> yield is not encountered until well after the first next() call.


Yeah, I wasn't trying to claim that passing the arguments to .next() is
equivalent to generator attributes, only that the point at which new
values for the generator state variables are provided correspond with
calls to .next(). So if there was a means within a generator of getting
access to the arguments passed to .next(), generator attributes would be
unnecessary for the examples provided.

> The given examples are minimal and are intended only to demonstrate the idea.


Do you have an example where the generator state isn't updated in
lock-step with .next() calls? I'd be interested to look at an example
of this...

>>I definitely don't like the
>>idea of a magical __self__ variable that isn't declared anywhere.

>
> It is no more magical than f.__name__ or f.__doc__ for functions.


I'm not sure this is quite a fair parallel. The difference here is that
f.__name__ and f.__doc__ are accessed as attributes of the f object,
and the __name__ and __doc__ attributes are created as a result of
function creation. The proposed __self__ is (1) not an attribute that
becomes available, rather, a new binding local to the function, and (2)
not created as a result of generator object creation but created as a
result of calling .next() on the generator object.

> Also, the __self__ argument is a non-issue because there are other alternate
> approaches such as providing a function that retrieves the currently
> running generator.


Is there a discussion of some of these alternate suggested approaches
somewhere you could point me to?

> The more important part of the PEP is the idea for generator exceptions. The
> need arises in the context of flushing/closing resources upon generator
> termination.


I wonder if maybe it would be worth moving this part to a separate PEP.
It seems like it could probably stand on its own merit, and having it
in with the generator attributes PEP means it isn't likely to be
accepted separately.

Of course, I would probably declare a class and provide a .close()
method. =)

Steve
 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Why Does Hash Apparently Reorder Its Internal Representation AndOther Associated Ponderings thoran@thoran.com Ruby 44 08-23-2006 02:16 AM
WTS APX-SL-DSP-3 288 port Voice/Data DSP cards isptrader VOIP 0 08-15-2005 05:40 PM
WTS: APX-SL-DSP-3 288 port Voice/Data DSP cards isptrader VOIP 0 08-03-2005 04:20 PM
google desktop ponderings. Dave - Dave.net.nz NZ Computing 4 10-19-2004 07:12 PM
Callable generators (PEP 288: Generator Attributes, again) Francis Avila Python 9 11-20-2003 12:49 AM



Advertisments