Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Python > Why are "broken iterators" broken?

Reply
Thread Tools

Why are "broken iterators" broken?

 
 
Steven D'Aprano
Guest
Posts: n/a
 
      09-21-2008
According to the Python docs, once an iterator raises StopIteration, it
should continue to raise StopIteration forever. Iterators that fail to
behave in this fashion are deemed to be "broken":

http://docs.python.org/lib/typeiter.html

I don't understand the reasoning behind this. As I understand it, an
iterator is something like a stream. There's no constraint that once a
stream is empty it must remain empty forever.

Can somebody explain why "broken iterators" are broken?


--
Steven
 
Reply With Quote
 
 
 
 
Fredrik Lundh
Guest
Posts: n/a
 
      09-21-2008
Steven D'Aprano wrote:

> According to the Python docs, once an iterator raises StopIteration, it
> should continue to raise StopIteration forever. Iterators that fail to
> behave in this fashion are deemed to be "broken":
>
> http://docs.python.org/lib/typeiter.html
>
> I don't understand the reasoning behind this. As I understand it, an
> iterator is something like a stream. There's no constraint that once a
> stream is empty it must remain empty forever.


it's a design guideline, not an absolute rule.

but I disagree that an iterator is "something like a stream". it's
rather "something like a pointer or an index", that is, an object that
helps you iterate over all members in a collection.

</F>

 
Reply With Quote
 
 
 
 
Roy Smith
Guest
Posts: n/a
 
      09-21-2008
In article <(E-Mail Removed)>,
Fredrik Lundh <(E-Mail Removed)> wrote:

> Steven D'Aprano wrote:
>
> > According to the Python docs, once an iterator raises StopIteration, it
> > should continue to raise StopIteration forever. Iterators that fail to
> > behave in this fashion are deemed to be "broken":
> >
> > http://docs.python.org/lib/typeiter.html
> >
> > I don't understand the reasoning behind this. As I understand it, an
> > iterator is something like a stream. There's no constraint that once a
> > stream is empty it must remain empty forever.

>
> it's a design guideline, not an absolute rule.
>
> but I disagree that an iterator is "something like a stream". it's
> rather "something like a pointer or an index", that is, an object that
> helps you iterate over all members in a collection.
>
> </F>


There are plausible examples of collections which grow while you're
iterating over them. I'm thinking specifically of a queue in a
multi-threaded application. One thread pushes work onto the back of the
queue while another pops from the front. The queue could certainly go
empty at times. But, maybe a Python iterator is just the wrong way to
model such behavior.
 
Reply With Quote
 
Fredrik Lundh
Guest
Posts: n/a
 
      09-21-2008
Roy Smith wrote:

> There are plausible examples of collections which grow while you're
> iterating over them. I'm thinking specifically of a queue in a
> multi-threaded application. One thread pushes work onto the back of the
> queue while another pops from the front. The queue could certainly go
> empty at times. But, maybe a Python iterator is just the wrong way to
> model such behavior.


you probably want the consumer thread to block when it catches up with
the producer, rather than exit.

(that's the default behaviour of Python's Queue object, btw)

</F>

 
Reply With Quote
 
Terry Reedy
Guest
Posts: n/a
 
      09-21-2008
Steven D'Aprano wrote:
> According to the Python docs, once an iterator raises StopIteration, it
> should continue to raise StopIteration forever. Iterators that fail to
> behave in this fashion are deemed to be "broken":
>
> http://docs.python.org/lib/typeiter.html
>
> I don't understand the reasoning behind this. As I understand it, an
> iterator is something like a stream. There's no constraint that once a
> stream is empty it must remain empty forever.


It is quite possible that a stream reader will return '' on one call and
then something non-empty the next. An iterator that reads a stream
and yields chunks of whatever size should either block until it gets
sufficient data or yield nulls as long as the stream is open and not
raise StopIteration until the steam is closed and it has yielded the
last chunk of data.

> Can somebody explain why "broken iterators" are broken?


There is an important different between a store that is closed until the
next day and one that closed - out of business. Similarly, there is a
difference between an item being out-of-stock until the next delivery
and out-of-stock and discontinued permanently, or between a road closed
for repairs versus removal for something else. Using the same sign or
signal for temporary and permanent conditions is confusing and therefore
'broken'.

Terry Jan Reedy

 
Reply With Quote
 
Miles
Guest
Posts: n/a
 
      09-22-2008
On Sun, Sep 21, 2008 at 11:13 AM, Steven D'Aprano
<(E-Mail Removed)> wrote:
> According to the Python docs, once an iterator raises StopIteration, it
> should continue to raise StopIteration forever. Iterators that fail to
> behave in this fashion are deemed to be "broken":
>
> http://docs.python.org/lib/typeiter.html
>
> I don't understand the reasoning behind this. As I understand it, an
> iterator is something like a stream. There's no constraint that once a
> stream is empty it must remain empty forever.
>
> Can somebody explain why "broken iterators" are broken?


It's not a terribly onerous restriction. If you're iterating with a
for-loop, you can make the iterable return a new iterator object when
the old one is exhausted, and if the intent is for the next()-method
to be called directly, you don't have to conform to the iterator
protocol.

Strictly speaking, file objects are broken iterators:

Python 2.5.1 (r251:54863, Jan 17 2008, 19:35:16)
>>> f = open('foo')
>>> it = iter(f)
>>> it.next()

'hi\n'
>>> it.next()

'bye\n'
>>> it.next()

Traceback (most recent call last):
File "<stdin>", line 1, in <module>
StopIteration
>>> f.seek(0)
>>> it.next()

'hi\n'

-Miles
 
Reply With Quote
 
Craig Allen
Guest
Posts: n/a
 
      09-23-2008
I'm interested what others think of this because at first I couldn't
get it... I have an object which can iterate over its parts... and at
first I thought, what? I'm supposed to create a new object every time
the user needs to iterate the contents?

In the end I interpreted that statement as if "unless __iter__()" is
called again, in which case it makes sense that an iterator should
stay "finished" until it's told to start to iterate again.

Then even the stream analogy holds, you don't expect a stream to say
EOF then start giving you bytes (from beyond the end, I guess)...
instead, such a stream would more likely block if it got to the end of
available data.

I'm probably being stupid in this... perhaps I'm the only one that at
first interpreted the phrase as being regardless of a fresh call to
__iter__()... and that the OP was worried about some other
implication. But if I was wrong to think it's ok for one and the same
iterator to reset when __iter__() is called again, then I seriously
don't understand.

what does "forever" mean in that dictum?
 
Reply With Quote
 
greg
Guest
Posts: n/a
 
      09-27-2008
Craig Allen wrote:

> In the end I interpreted that statement as if "unless __iter__()" is
> called again


It seems you were confusing iterators and iterables. The
iterator is the object that is returned by calling
__iter__() on an iterable, and yes, you are expected to
get a new one each time you want to start iterating
again.

--
Greg
 
Reply With Quote
 
Lie
Guest
Posts: n/a
 
      09-28-2008
On Sep 21, 10:13*pm, Steven D'Aprano <st...@REMOVE-THIS-
cybersource.com.au> wrote:
> According to the Python docs, once an iterator raises StopIteration, it
> should continue to raise StopIteration forever. Iterators that fail to
> behave in this fashion are deemed to be "broken":
>
> http://docs.python.org/lib/typeiter.html
>
> I don't understand the reasoning behind this. As I understand it, an
> iterator is something like a stream. There's no constraint that once a
> stream is empty it must remain empty forever.


I think empty != StopIteration. StopIteration (IMHO) shouldn't be
raised when the stream is empty, instead a sentinel value specifying
that "there is no data yet, but if you wait there might be" should be
returned (possibly None or empty string). When you raise
StopIteration, it is a signal that I don't have any more data and
there is no use in waiting.

> Can somebody explain why "broken iterators" are broken?
>
> --
> Steven


 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
why why why why why Mr. SweatyFinger ASP .Net 4 12-21-2006 01:15 PM
findcontrol("PlaceHolderPrice") why why why why why why why why why why why Mr. SweatyFinger ASP .Net 2 12-02-2006 03:46 PM
Cisco 2611 and Cisco 1721 : Why , why , why ????? sam@nospam.org Cisco 10 05-01-2005 08:49 AM
Why, why, why??? =?Utf-8?B?VGltOjouLg==?= ASP .Net 6 01-27-2005 03:35 PM
Why Why Why You HAVE NO IDEA MCSE 31 04-24-2004 06:40 PM



Advertisments