Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Python > Re: proposal: another file iterator

Reply
Thread Tools

Re: proposal: another file iterator

 
 
Jean-Paul Calderone
Guest
Posts: n/a
 
      01-16-2006
On 15 Jan 2006 16:44:24 -0800, Paul Rubin <"http://phr.cx"@nospam.invalid> wrote:
>I find pretty often that I want to loop through characters in a file:
>
> while True:
> c = f.read(1)
> if not c: break
> ...
>
>or sometimes of some other blocksize instead of 1. It would sure
>be easier to say something like:
>
> for c in f.iterbytes(): ...
>
>or
>
> for c in f.iterbytes(blocksize): ...
>
>this isn't anything terribly advanced but just seems like a matter of
>having the built-in types keep up with language features. The current
>built-in iterator (for line in file: ...) is useful for text files but
>can potentially read strings of unbounded size, so it's inadvisable for
>arbitrary files.
>
>Does anyone else like this idea?


It's a pretty useful thing to do, but the edge-cases are somewhat complex. When I just want the dumb version, I tend to write this:

for chunk in iter(lambda: f.read(blocksize), ''):
...

Which is only very slightly longer than your version. I would like it even more if iter() had been written with the impending doom of lambda in mind, so that this would work:

for chunk in iter('', f.read, blocksize):
...

But it's a bit late now. Anyhow, here are some questions about your iterbytes():

* Would it guarantee the chunks returned were read using a single read? If blocksize were a multiple of the filesystem block size, would it guarantee reads on block-boundaries (where possible)?

* How would it handle EOF? Would it stop iterating immediately after the first short read or would it wait for an empty return?

* What would the buffering behavior be? Could one interleave calls to .next() on whatever iterbytes() returns with calls to .read() on the file?

Jean-Paul
 
Reply With Quote
 
 
 
 
Paul Rubin
Guest
Posts: n/a
 
      01-16-2006
Jean-Paul Calderone <(E-Mail Removed)> writes:
> Which is only very slightly longer than your version. I would like
> it even more if iter() had been written with the impending doom of
> lambda in mind, so that this would work:
>
> for chunk in iter('', f.read, blocksize):
> ...
>
> But it's a bit late now.


Well, iter could be extended to take *args and **kwargs parameters, so
you'd say

for chunk in iter(f.read, '', (blocksize,)): ...

That leaves the params in a clumsy order for backwards compatibility,
but it's not unbearable.

> Anyhow, here are some questions about your iterbytes():
>
> * Would it guarantee the chunks returned were read using a single
> read? If blocksize were a multiple of the filesystem block size,
> would it guarantee reads on block-boundaries (where possible)?


I expect that the iterator's .next() would just get the result
of f.read(blocksize), which makes no such guarantees.

> * How would it handle EOF? Would it stop iterating immediately
>after the first short read or would it wait for an empty return?


Wait for empty return.

> * What would the buffering behavior be? Could one interleave
> calls to .next() on whatever iterbytes() returns with calls to
> .read() on the file?


I don't see why not. Iterbytes would just call read() and yield the
result. You could even have multiple iterators going at once.
 
Reply With Quote
 
 
 
 
Bengt Richter
Guest
Posts: n/a
 
      01-16-2006
On 15 Jan 2006 18:58:53 -0800, Paul Rubin <http://(E-Mail Removed)> wrote:

>Jean-Paul Calderone <(E-Mail Removed)> writes:
>> Which is only very slightly longer than your version. I would like
>> it even more if iter() had been written with the impending doom of
>> lambda in mind, so that this would work:
>>
>> for chunk in iter('', f.read, blocksize):
>> ...
>>
>> But it's a bit late now.

>
>Well, iter could be extended to take *args and **kwargs parameters, so
>you'd say
>
> for chunk in iter(f.read, '', (blocksize,)): ...
>

Whatever "Which" refers to got snipped or may be in a post that hasn't
become visible for me yet, but I assume Jean-Paul was referring to lambda use
as in e.g. (untested):

for chunk in iter(lambda frd=open('yerf', 'rb').read:frd(blocksize), ''): ...

Does it not do what you want?

Regards,
Bengt Richter
 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
List iterator assignment fails, assert iterator not dereferencable David Bilsby C++ 5 10-09-2007 02:05 PM
What makes an iterator an iterator? Steven D'Aprano Python 28 04-20-2007 03:34 AM
Difference between Java iterator and iterator in Gang of Four Hendrik Maryns Java 18 12-22-2005 05:14 AM
How to convert from std::list<T*>::iterator to std::list<const T*>::iterator? PengYu.UT@gmail.com C++ 6 10-30-2005 03:31 AM
Iterator doubts, Decision on Iterator usage greg C++ 6 07-17-2003 01:26 PM



Advertisments