Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Python > Re: select.poll.poll() never blocks

Reply
Thread Tools

Re: select.poll.poll() never blocks

 
 
Jean-Paul Calderone
Guest
Posts: n/a
 
      02-12-2009
On Wed, 11 Feb 2009 18:44:53 -0800 (PST), birdsong <(E-Mail Removed)> wrote:
>I'm pretty sure I've exhausted all searches and read all the forums
>Google will turn up related to this issue.
>
>I touch an empty file in a sh shell, fire up the python shell, open
>the file for reading(tried all buffering options), register it with a
>poll object for select.POLLIN and call poll(), but the poll never
>blocks and always returns for the FD, EVENT combination I ask for, but
>the file has not anything written to it.


Filesystem files are always reported as readable and writeable by select,
poll, epoll, etc.

If you want to do non-blocking filesystem I/O, your choices are to use a
native thread (Python's threading module, or another third-party module
which wraps the platform thread API) or to use POSIX AIO (or the mildly
incompatible Linux variant. Of course, AIO has tons of caveats and is
generally in a miserable state, so you probably shouldn't use it. That
leaves you with threads.

Jean-Paul
 
Reply With Quote
 
 
 
 
birdsong
Guest
Posts: n/a
 
      02-12-2009
So I guess I didn't have a complete understanding of poll, I thought
it returned file descriptors that had registered an event that the
user asked to watch for. In my case, select.POLLIN
Constant Meaning
POLLIN There is data to read

....there ins't any data to be read. The file is readable, yes, but is
that all that the syscall has to offer?


On Feb 11, 6:49*pm, Jean-Paul Calderone <(E-Mail Removed)> wrote:
> On Wed, 11 Feb 2009 18:44:53 -0800 (PST), birdsong <(E-Mail Removed)> wrote:
> >I'm pretty sure I've exhausted all searches and read all the forums
> >Google will turn up related to this issue.

>
> >I touch an empty file in a sh shell, fire up the python shell, open
> >the file for reading(tried all buffering options), register it with a
> >poll object for select.POLLIN and call poll(), but the poll never
> >blocks and always returns for the FD, EVENT combination I ask for, but
> >the file has not anything written to it.

>
> Filesystem files are always reported as readable and writeable by select,
> poll, epoll, etc.
>
> If you want to do non-blocking filesystem I/O, your choices are to use a
> native thread (Python's threading module, or another third-party module
> which wraps the platform thread API) or to use POSIX AIO (or the mildly
> incompatible Linux variant. *Of course, AIO has tons of caveats and is
> generally in a miserable state, so you probably shouldn't use it. *That
> leaves you with threads.
>
> Jean-Paul


 
Reply With Quote
 
 
 
 
Rhodri James
Guest
Posts: n/a
 
      02-12-2009
On Thu, 12 Feb 2009 03:01:16 -0000, birdsong <(E-Mail Removed)>
wrote:

> So I guess I didn't have a complete understanding of poll, I thought
> it returned file descriptors that had registered an event that the
> user asked to watch for. In my case, select.POLLIN
> Constant Meaning
> POLLIN There is data to read
>
> ...there ins't any data to be read. The file is readable, yes, but is
> that all that the syscall has to offer?


To quote the web page referenced in help(select):

"It cannot be used on regular files to determine whether a file has grown
since it was last read."

poll() and friends are wrappers over the socket library "poll" and
"select" functions, which primarily exist to facilitate asynchronous
socket communications. They can take ordinary file descriptors as
well as sockets because there's no good reason not to, but a regular
file will always be ready to read as long as you haven't read past
the end of file.

You're trying to use a regular file as if it was a pipe. My first
obvious question is do you have to do it this way? Do you have
any control over the program writing to file? Can you cause the
output to go to stdout and pipe it into your script's stdin
instead? That would make your life vastly easier.

If you can't, I'm not sure what your best strategy is. I'd
be tempted to use "tail -f filetocheck | yourscript.py" and
palm the job off on an already-written tool. If you want to
do it in Python, the only thing that springs to mind is
periodically checking the size of the file and reading more
when that changes. You'll need to be very careful to keep
what size you think the file is in sync with how much you've
read!

--
Rhodri James *-* Wildebeeste Herder to the Masses
 
Reply With Quote
 
birdsong
Guest
Posts: n/a
 
      02-12-2009
On Feb 11, 7:47*pm, "Rhodri James" <(E-Mail Removed)>
wrote:
> On Thu, 12 Feb 2009 03:01:16 -0000, birdsong <(E-Mail Removed)> *
> wrote:
>
> > So I guess I didn't have a complete understanding of poll, I thought
> > it returned file descriptors that had registered an event that the
> > user asked to watch for. *In my case, select.POLLIN
> > Constant * Meaning
> > POLLIN * * There is data to read

>
> > ...there ins't any data to be read. *The file is readable, yes, but is
> > that all that the syscall has to offer?

>
> To quote the web page referenced in help(select):
>
> "It cannot be used on regular files to determine whether a file has grown *
> since it was last read."
>
> poll() and friends are wrappers over the socket library "poll" and
> "select" functions, which primarily exist to facilitate asynchronous
> socket communications. *They can take ordinary file descriptors as
> well as sockets because there's no good reason not to, but a regular
> file will always be ready to read as long as you haven't read past
> the end of file.
>
> You're trying to use a regular file as if it was a pipe. *My first
> obvious question is do you have to do it this way? *Do you have
> any control over the program writing to file? *Can you cause the
> output to go to stdout and pipe it into your script's stdin
> instead? *That would make your life vastly easier.

agreed, this was my original idea, but i figured implementing a tail -
f via poll was alot faster to write and implement. i'm glad at least
know why it wasn't working as expected.

i've already written a tail -f similar to the one found in the link
from a later poster, but i was hoping to use poll to cut down on
unecessary seeks -i'm trying to avoid actions that would drive up
iowait on the boxes this would run on.

thanks for the clarification on the poll family of syscalls.

>
> If you can't, I'm not sure what your best strategy is. *I'd
> be tempted to use "tail -f filetocheck | yourscript.py" and
> palm the job off on an already-written tool. *If you want to
> do it in Python, the only thing that springs to mind is
> periodically checking the size of the file and reading more
> when that changes. *You'll need to be very careful to keep
> what size you think the file is in sync with how much you've
> read!
>
> --
> Rhodri James *-* Wildebeeste Herder to the Masses


 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
You've never seen it before and you'll never see it again. Fred A Stover Computer Support 7 12-26-2007 03:33 AM
XML Schema never-never occurence of declared elements / attributes Soren Kuula XML 1 12-01-2005 01:27 PM
procs/blocks - blocks with procs, blocks with blocks? matt Ruby 1 08-06-2004 01:33 AM
string routines go to never never land on unix Kevin C Programming 4 10-17-2003 06:07 PM



Advertisments