Velocity Reviews > I don't understand generator.send()

# I don't understand generator.send()

Victor Eijkhout
Guest
Posts: n/a

 05-15-2011
#! /usr/bin/env python

def ints():
i=0
while True:
yield i
i += 1

gen = ints()
while True:
i = gen.next()
print i
if i==5:
r = gen.send(2)
print "return:",r
if i>10:
break

I thought the send call would push the value "2" at the front of the
queue. Instead it coughs up the 2, which seems senseless to me.

1/ How should I view the send call? I'm reading the manual and dont' get
it
2/ Is there a way to push something in the generator object? So that it
becomes the next yield expression? In my code I was hoping to get
0,1,2,3,4,5,2,6,7 as yield expressions.

Victor.

--
Victor Eijkhout -- eijkhout at tacc utexas edu

OKB (not okblacke)
Guest
Posts: n/a

 05-15-2011
Victor Eijkhout wrote:

> #! /usr/bin/env python
>
> def ints():
> i=0
> while True:
> yield i
> i += 1
>
> gen = ints()
> while True:
> i = gen.next()
> print i
> if i==5:
> r = gen.send(2)
> print "return:",r
> if i>10:
> break
>
> I thought the send call would push the value "2" at the front of
> the queue. Instead it coughs up the 2, which seems senseless to me.
>
> 1/ How should I view the send call? I'm reading the manual and
> dont' get it
> 2/ Is there a way to push something in the generator object? So
> that it becomes the next yield expression? In my code I was hoping
> to get 0,1,2,3,4,5,2,6,7 as yield expressions.

You can't usefully use send() unless the generator is set up to
make use of the sent values. You can't just push values into any old
generator. For it to do anything, you need to use assign the result of
the yield to something within your generator and make use of it. See
http://docs.python.org/whatsnew/2.5....rator-features
for an example.

--
--OKB (not okblacke)
Brendan Barnwell
"Do not follow where the path may lead. Go, instead, where there is
no path, and leave a trail."
--author unknown

Chris Rebert
Guest
Posts: n/a

 05-15-2011
On Sat, May 14, 2011 at 5:08 PM, Victor Eijkhout <(E-Mail Removed)> wrote:
> #! /usr/bin/env python
>
> def ints():
> Â* Â*i=0
> Â* Â*while True:
> Â* Â* Â* Â*yield i
> Â* Â* Â* Â*i += 1
>
> gen = ints()
> while True:
> Â* Â*i = gen.next()
> Â* Â*print i
> Â* Â*if i==5:
> Â* Â* Â* Â*r = gen.send(2)
> Â* Â* Â* Â*print "return:",r
> Â* Â*if i>10:
> Â* Â* Â* Â*break
>
> I thought the send call would push the value "2" at the front of the
> queue. Instead it coughs up the 2, which seems senseless to me.
>
> 1/ How should I view the send call? I'm reading the manual and dont' get
> it

`yield` is an expression. Within the generator, the result of that
expression is [, ignoring the complications of .throw() etc.,] the
argument to .send(). You're currently using `yield` only as a
statement, so it's no wonder you're not quite understanding .send(). I
think this example should clarify things somewhat:

>>> def example(start):

.... i = ord(start)
.... while True:
.... sent = (yield chr(i)) # Note use of yield as expression
.... print('was sent', sent)
.... i += 1
....
>>> g = example('a')
>>> g.send(3)

Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: can't send non-None value to a just-started generator
>>> # Ok, so we can't send something back to `yield`
>>> # until we hit the first `yield`.
>>> g.send(None) # Follow the error message's advice

'a'
>>> g.send(3) # Let's try again now.

was sent 3
'b'
>>> g.send(5)

was sent 5
'c'
>>> g.send(9)

was sent 9
'd'
>>>

Cheers,
Chris
--
http://rebertia.com

Chris Angelico
Guest
Posts: n/a

 05-15-2011
On Sun, May 15, 2011 at 10:08 AM, Victor Eijkhout <(E-Mail Removed)> wrote:
> * * * *yield i

> * * * *r = gen.send(2)

When you send() something to a generator, it becomes the return value
of the yield expression. See the example here:
http://docs.python.org/whatsnew/2.5....rator-features

For what you're doing, there's a little complexity. If I understand,
you want send() to be like an ungetc call... you could do that like
this:

def ints():
i=0
while True:
sent=(yield i)
if sent is not None:
yield None # This becomes the return value from gen.send()
yield sent # This is the next value yielded
i += 1

This lets you insert at most one value per iteration. Supporting more
than one insertion is more complicated, but changing the loop
structure entirely may help:

def ints():
i=0
queue=[]
while True:
if queue: # see other thread, this IS legal and pythonic and
quite sensible
sent=(yield queue.pop(0))
else:
sent=(yield i)
i+=1
if sent is not None:
yield None # This is the return value from gen.send()
queue.append(sent)

With this generator, you maintain a queue of sent values (if you want
it to be a LIFO stack rather than a FIFO queue, just change the pop(0)
to just pop()), and if the queue's empty, it produces sequential
integers. (Incidentally, the sent values don't have to be integers. I
leave it to you to decide whether that's any use or not.)

Hope that helps!

Chris Angelico

Ian Kelly
Guest
Posts: n/a

 05-15-2011
On Sat, May 14, 2011 at 6:08 PM, Victor Eijkhout <(E-Mail Removed)> wrote:
> I thought the send call would push the value "2" at the front of the
> queue. Instead it coughs up the 2, which seems senseless to me.
>
> 1/ How should I view the send call? I'm reading the manual and dont' get
> it

There is no queue unless you create one inside the generator. The
generator by itself behaves more like a coroutine.

> 2/ Is there a way to push something in the generator object? So that it
> becomes the next yield expression? In my code I was hoping to get
> 0,1,2,3,4,5,2,6,7 as yield expressions.

This will do what you're asking for:

def ints():
i=0
while True:
next_yield = (yield i)
while next_yield is not None:
next_yield = (yield next_yield)
i += 1

However, I don't think this is what you want. The send call returns a
yield expression, which will then be the value that you just passed
in, which seems a bit silly. Probably you want something more like
this:

def ints():
i=0
while True:
next_yield = (yield i)
while next_yield is not None:
yield None
next_yield = (yield next_yield)
i += 1

Then the send() call will return None, and the next next() call will
return the value you passed in. Note though that this is too simple
to work correctly if you call send() more than once before calling
next() again.

In general, I think it is a bad idea to mix calling next() and send()
on the same generator. It makes the generator logic too complicated,
and I think it's better just to create a stateful iterator class
instead, where send() and next() are two entirely separate methods.

Cheers,
Ian

Ian Kelly
Guest
Posts: n/a

 05-15-2011
On Sat, May 14, 2011 at 6:47 PM, Chris Angelico <(E-Mail Removed)> wrote:
> def ints():
> * *i=0
> * *queue=[]
> * *while True:
> * * * *if queue: *# see other thread, this IS legal and pythonic and
> quite sensible
> * * * * * *sent=(yield queue.pop(0))
> * * * *else:
> * * * * * *sent=(yield i)
> * * * * * *i+=1
> * * * *if sent is not None:
> * * * * * *yield None *# This is the return value from gen.send()
> * * * * * *queue.append(sent)
>
> With this generator, you maintain a queue of sent values (if you want
> it to be a LIFO stack rather than a FIFO queue, just change the pop(0)
> to just pop()), and if the queue's empty, it produces sequential
> integers. (Incidentally, the sent values don't have to be integers. I
> leave it to you to decide whether that's any use or not.)

Actually, this won't work, because the value of the "yield None" gets
ignored. Thus if you try to call send() twice in a row, the generator
the treats second send() as if it were a next(), and it is not
possible to have more than one item in the queue.

Chris Angelico
Guest
Posts: n/a

 05-15-2011
On Sun, May 15, 2011 at 11:05 AM, Ian Kelly <(E-Mail Removed)> wrote:
> Actually, this won't work, because the value of the "yield None" gets
> ignored. *Thus if you try to call send() twice in a row, the generator
> the treats second send() as if it were a next(), and it is not
> possible to have more than one item in the queue.

You're right. It needs a while loop instead of the if (and some slight
reordering):

def ints():
i=0
queue=[]
while True:
if queue: # see other thread, this IS legal and pythonic and
quite sensible
sent=(yield queue.pop(0))
else:
sent=(yield i)
i+=1
while sent is not None:
queue.append(sent)
sent=(yield None) # This is the return value from gen.send()

That should work.

Chris Angelico

Victor Eijkhout
Guest
Posts: n/a

 05-15-2011
Chris Angelico <(E-Mail Removed)> wrote:

> For what you're doing, there's a little complexity. If I understand,
> you want send() to be like an ungetc call... you could do that like
> this:
>
>
> def ints():
> i=0
> while True:
> sent=(yield i)
> if sent is not None:
> yield None # This becomes the return value from gen.send()
> yield sent # This is the next value yielded
> i += 1

I think this will serve my purposes.

Thanks everyone for broadening my understanding of generators.

Victor.
--
Victor Eijkhout -- eijkhout at tacc utexas edu

Ian Kelly
Guest
Posts: n/a

 05-15-2011
On Sat, May 14, 2011 at 7:17 PM, Chris Angelico <(E-Mail Removed)> wrote:
> You're right. It needs a while loop instead of the if (and some slight
> reordering):
>
> def ints():
> * i=0
> * queue=[]
> * while True:
> * * * if queue: *# see other thread, this IS legal and pythonic and
> quite sensible
> * * * * * sent=(yield queue.pop(0))
> * * * else:
> * * * * * sent=(yield i)
> * * * * * i+=1
> * * * while sent is not None:
> * * * * * queue.append(sent)
> * * * * * sent=(yield None) *# This is the return value from gen.send()
>
> That should work.

Yeah, that should do it. But this is so much easier to get right and
to understand:

import itertools

class Ints(object):

def __init__(self):
self.ints = itertools.count()
self.queue = []

def __iter__(self):
return self

def next(self):
if self.queue:
return self.queue.pop(0)
else:
return self.ints.next()

def insert(self, x):
self.queue.append(x)