Re: read stdout/stderr without blocking
I had just mixed Pexpect and subProcess code until I'd got something that worked, you can actually explain my code better a I can myself. I find it quite cumbersome to read stdout/strerr separately, and to be able to write to stdin in reaction to either of them, but at least on Linux you can get it to work. My NON_BLOCKing command might be unnecesary, I'll try without it.
The OP seemed interested on how to do this on Windows, but I've yet to see an answer on that one I think.
Thank you for the reply.
|>>>Donn Cave <firstname.lastname@example.org> 09/15/05 6:19 pm >>>
|In article <ebGdnZ2dnZ1wQWfKnZ2dnTtdu96dnZ2dRVn-zZ2dnZ0@powergate.ca>,
|Peter Hansen <email@example.com> wrote:
|>Jacek Popławski wrote:
|>>Grant Edwards wrote:
|>>>On 2005-09-12, Jacek Pop?awski <firstname.lastname@example.org> wrote:
|>>>>> ready = select.select(tocheck, , , 0.25) ##continues
|>>>>> for file in ready:
|>>>>> text = os.read(file, 1024)
|>>>>How do you know here, that you should read 1024 characters?
|>>>>What will happen when output is shorter?
|>>>It will return however much data is available.
|>>My tests showed, that it will block.
|>Not if you use non-blocking sockets, as I believe you are expected to
|>when using select().
|On the contrary, you need non-blocking sockets only if
|you don't use select. select waits until a read [write]
|would not block - it's like "if dict.has_key(x):" instead of
|"try: val = dict[x] ; except KeyError:". I suppose you
|knew that, but have read some obscure line of reasoning
|that makes non-blocking out to be necessary anyway.
|Who knows, but it certainly isn't in this case.
|I don't recall the beginning of this thread, so I'm not sure
|if this is the usual wretched exercise of trying to make this
|work on both UNIX and Windows, but there are strong signs
|of the usual confusion over os.read (a.k.a. posix.read), and
|file object read. Let's hopefully forget about Windows for
|The above program looks fine to me, but it will not work
|reliably if file object read() is substituted for os.read().
|In this case, C library buffering will read more than 1024
|bytes if it can, and then that data will not be visible to
|select(), so there's no guarantee it will return in a timely
|manner even though the next read() would return right
|away. Reading one byte at a time won't resolve this problem,
|obviously it will only make it worse. The only reason to
|read one byte at a time is for data-terminated read semantics,
|specifically readline(), in an unbuffered file. That's what
|happens -- at the system call level, where it's expensive --
|when you turn off stdio buffering and then call readline().
|In the C vs. Python example, read() is os.read(), and file
|object read() is fread(); so of course, C read() works
|where file object read() doesn't.
|Use select, and os.read (and UNIX) and you can avoid blocking
|on a pipe. That's essential if as I am reading it there are supposed
|to be two separate pipes from the same process, since if one is
|allowed to fill up, that process will block, causing a deadlock if
|the reading process blocks on the other pipe.
|Hope I'm not missing anything here. I just follow this group
|to answer this question over and over, so after a while it
|gets sort of automatic.
| Donn Cave, email@example.com
|All times are GMT. The time now is 09:06 PM.|
Powered by vBulletin®. Copyright ©2000 - 2014, vBulletin Solutions, Inc.
SEO by vBSEO ©2010, Crawlability, Inc.