Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Python > Re: Writev

Reply
Thread Tools

Re: Writev

 
 
Adam DePrince
Guest
Posts: n/a
 
      12-21-2004
On Sun, 2004-12-19 at 23:43, Jp Calderone wrote:
> On Sun, 19 Dec 2004 23:12:27 -0500, Adam DePrince <(E-Mail Removed)> wrote:
> > [snip]
> >
> > Of course, to take advantage of this requires that writev be exposed. I
> > have an implementation of writev. This implementation is reasonably
> > smart, it "unrolls" only so as many iteration.next calls as necessary to
> > fill the pointer list for the next invocation of writev. Of course, on
> > many systems (Linux, BSD) writev's limit is 1024 items, so to
> > accommodate users who don't want to peel off and hold in memory 1024
> > iteration.next()'s, you can set an optional parameter to a smaller
> > value.
> >
> > I'm not sure where to take this as a "next step." It seems too small a
> > change for a PEP. Any ideas?
> >
> > You can download the patch from
> > http://deprince.net/software/writev/index.html
> >

>
> writev() in the standard library would be nice. I have an
> implementation too I wonder how many other people do as
> well.
>
> Regarding the implementation, just one part confuses me.
> What's the idea behind this check at the beginning of the
> iterator unrolling loop?
>
> + if ( !PyObject_CheckReadBuffer( buf ) )
> + continue;
>
> Is it me, or is that silently ignoring bad input?
>
> Other miscellaneous feedback:
>
> The use of alloca() isn't common in CPython. PyMem_Malloc
> is probably what you want to be calling. At the very least,
> alloca()'s return value should be checked to be sure it is
> non-NULL. If you switch to PyMem_Malloc, you'll have to be
> sure to free the memory, of course.
>
> The support of iterators is a cool idea, but I'm not sure
> it is actually useful. Consider the case where not all bytes
> are written. How is an application supposed to handle writing
> the rest of the bytes, if they have been consumed from an
> iterator? Hopefully I'm missing the obvious answer, because
> supporting arbitrary iterators would be cool, but it seems like
> only sequences can really be usefully supported.
>


Fixed this. The return value is a tuple of bytes written and a tuple of
stuff that didn't get shipped. And yes, the first element is replaced
with a string that represents the fraction of that element that didn't
get sent. Here is simple sample code that illustrates how this works.

data = [ .... lots of little strings in this list ... ]
data = iter( data )
import itertools
import writev

while 1:
(count, leftovers ) = writev.writev( fd, data )
if leftovers == None: break
data = itertools.chain( iter( leftovers ), data )
... Of course, our loop would in a real example be controlled by a
select. In this naive example we would want to sleep ...

Adam DePrince

 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Calling writev and readv from perl A. Farber Perl Misc 6 08-21-2008 12:58 AM
writev() on a SOCK_STREAM socket kid joe C Programming 5 06-20-2008 09:04 AM
Writev Adam DePrince Python 6 12-20-2004 11:51 PM
Re: Writev Adam DePrince Python 0 12-20-2004 05:44 AM



Advertisments