«

Hi all,

I have been looking into non-blocking read (readline) operations on
PIPES on windows XP and there seems to be no way of doing this. Ive
read that you could use a Thread to read from the pipe, but if you
still use readline() wouldnt the Thread block too?

What i need to do is, create a process using subprocess.Popen, where
the subprocess outputs information on one line (but the info
continuesly changes and its always on the same line) and read this
information without blocking, so i can retrieve other data from the
line i read in then put this in a GUI interface.


readline() blocks until the newline character is read, but when i use
read(X) where X is a number of bytes then it doesnt block(expected
functionality) but i dont know how many bytes the line will be and its
not constant so i cant use this too.

Any ideas of solving this problem?


Cheers

placid wrote:
> Hi all,
>
> I have been looking into non-blocking read (readline) operations on
> PIPES on windows XP and there seems to be no way of doing this. Ive
> read that you could use a Thread to read from the pipe, but if you
> still use readline() wouldnt the Thread block too?

Yes it will, but that's ok. In this case that's what it's for. While
the thread waits for the readline(), the rest of your program continues
to carry on.

>
> What i need to do is, create a process using subprocess.Popen, where
> the subprocess outputs information on one line (but the info
> continuesly changes and its always on the same line) and read this
> information without blocking, so i can retrieve other data from the
> line i read in then put this in a GUI interface.
>
>
> readline() blocks until the newline character is read, but when i use
> read(X) where X is a number of bytes then it doesnt block(expected
> functionality) but i dont know how many bytes the line will be and its
> not constant so i cant use this too.
>
> Any ideas of solving this problem?
>
>
> Cheers

On 27 Jul 2006 22:26:25 -0700, "placid" <> declaimed the
following in comp.lang.python:

>
> readline() blocks until the newline character is read, but when i use
> read(X) where X is a number of bytes then it doesnt block(expected
> functionality) but i dont know how many bytes the line will be and its
> not constant so i cant use this too.
>
> Any ideas of solving this problem?
>
Use a thread that reads one character at a time; when it sees
whatever signals "end of line" (it sounds like you're reading a progress
bar implemented via <cr>overwrite). Combine the characters into a
string, return the string to the main program via a queue.

If there is no such "end of line" character, but there IS a
noticeable delay between "writes", a more complex method might suffice
-- in which one thread does the byte reads, setting a time value on each
read; a related thread then does a sleep() loop, checking the "last read
time" against the pause length -- if close enough to the pause duration,
combine and return...

Alternatively, take a good old style terminal keyboard (a VT100
Tempest-rated model should be ideal), and use it to beat Bill Gates over
the head until he agrees to push a high-priority upgrade to the command
line I/O system... or makes files work with select() (so you can combine
the time-out with the byte read)

--
Wulfraed Dennis Lee Bieber KD6MOG

HTTP://wlfraed.home.netcom.com/
(Bestiaria Support Staff: web-)
HTTP://www.bestiaria.com/

"placid" <> wrote in message
news: oups.com

> Hi all,
>
> I have been looking into non-blocking read (readline) operations on
> PIPES on windows XP and there seems to be no way of doing this. Ive
> read that you could use a Thread to read from the pipe, but if you
> still use readline() wouldnt the Thread block too?

http://aspn.activestate.com/ASPN/Coo.../Recipe/440554

> What i need to do is, create a process using subprocess.Popen, where
> the subprocess outputs information on one line (but the info
> continuesly changes and its always on the same line) and read this
> information without blocking, so i can retrieve other data from the
> line i read in then put this in a GUI interface.
>
>
> readline() blocks until the newline character is read, but when i use
> read(X) where X is a number of bytes then it doesnt block(expected
> functionality) but i dont know how many bytes the line will be and its
> not constant so i cant use this too.
>
> Any ideas of solving this problem?
>
>
> Cheers

I realized something very similar to what you described in

http://sourceforge.net/projects/bestgui

- the subprocess2.py module realizes the non blocking I/O
- the outputparser.py module processes the output from the controlled
process and updates the progress-bar, the status-bar and the log
messages in the GUI. Incomplete lines are stored in a buffer and
processed at the next read.

ciao

--
Antonio Valentino



--
Posted via Mailgate.ORG Server - http://www.Mailgate.ORG

Dennis Lee Bieber wrote:
> On 27 Jul 2006 22:26:25 -0700, "placid" <> declaimed the
> following in comp.lang.python:
>
> >
> > readline() blocks until the newline character is read, but when i use
> > read(X) where X is a number of bytes then it doesnt block(expected
> > functionality) but i dont know how many bytes the line will be and its
> > not constant so i cant use this too.
> >
> > Any ideas of solving this problem?
> >
> Use a thread that reads one character at a time; when it sees
> whatever signals "end of line" (it sounds like you're reading a progress
> bar implemented via <cr>overwrite). Combine the characters into a
> string, return the string to the main program via a queue.
>

Yes it is a progress bar implemented via <cr> overwrite. I will try
this.

> If there is no such "end of line" character, but there IS a
> noticeable delay between "writes", a more complex method might suffice
> -- in which one thread does the byte reads, setting a time value on each
> read; a related thread then does a sleep() loop, checking the "last read
> time" against the pause length -- if close enough to the pause duration,
> combine and return...

i dont think there is a noticeable delay between "writes".


> Alternatively, take a good old style terminal keyboard (a VT100
> Tempest-rated model should be ideal), and use it to beat Bill Gates over
> the head until he agrees to push a high-priority upgrade to the command
> line I/O system... or makes files work with select() (so you can combine
> the time-out with the byte read)

Tsk Tsk

On 30 Jul 2006 16:22:34 -0700, "placid" <> declaimed the
following in comp.lang.python:

>
> Tsk Tsk

Have you ever seen a Tempest VT-100? Lead shielding on the monitor
FACE... Turn the brightness all the way up and it still looked dim.
Fiber optic to the computer. And a shield keyboard about three inches
thick and weighing 5 lbs.

All to keep "them" from using radio equipment to pick up key strokes
or the display scan line radiation (use an external vertical/horizontal
sweep generator to an oscilloscope, and a high-gain amplifier to detect
the brightness variation of the CRT).
--
Wulfraed Dennis Lee Bieber KD6MOG

HTTP://wlfraed.home.netcom.com/
(Bestiaria Support Staff: web-)
HTTP://www.bestiaria.com/

Dennis Lee Bieber wrote:
> On 30 Jul 2006 16:22:34 -0700, "placid" <> declaimed the
> following in comp.lang.python:
>
> >
> > Tsk Tsk
>
> Have you ever seen a Tempest VT-100? Lead shielding on the monitor
> FACE... Turn the brightness all the way up and it still looked dim.
> Fiber optic to the computer. And a shield keyboard about three inches
> thick and weighing 5 lbs.
>
> All to keep "them" from using radio equipment to pick up key strokes
> or the display scan line radiation (use an external vertical/horizontal
> sweep generator to an oscilloscope, and a high-gain amplifier to detect
> the brightness variation of the CRT).


Nope, i have not seen one.

placid wrote:
> What i need to do is, create a process using subprocess.Popen, where
> the subprocess outputs information on one line (but the info
> continuesly changes and its always on the same line) and read this
> information without blocking, so i can retrieve other data from the
> line i read in then put this in a GUI interface.

> readline() blocks until the newline character is read, but when i use
> read(X) where X is a number of bytes then it doesnt block(expected
> functionality) but i dont know how many bytes the line will be and its
> not constant so i cant use this too.

I wrote something for this the other day. The class has a getline()
method, which either returns a line or raises an exception instead of
blocking. It also raises an exception instead of returning EOF.

My use case had me reading from multiple processes at once; since
select() doesn't work on files in win32, I had to get a little cheesy.
I've appended the function that implements that use case, for
reference.

The central idea is to use PeekNamedPipe to figure out what's in the
pipe. You can then read that data without fear of blocking. I wrote it
quickly, therefore the code is a little embarassing, but... hopefully
useful all the same.

.. class NoLineError(Exception): pass
.. class NoMoreLineError(Exception): pass
.. class liner(object):
.. """Helper class for multi_readlines."""
.. def __init__(self, f):
.. self.fd = f.fileno()
.. self.osf = msvcrt.get_osfhandle(self.fd)
.. self.buf = ''
..
.. def getline(self):
.. """Returns a line of text, or raises NoLineError, or
NoMoreLineError."""
.. try:
.. data, avail, _ = win32pipe.PeekNamedPipe(self.osf, 0)
.. except pywintypes.error:
.. # Pipe closed: give up what we have, then that's it
.. if self.buf:
.. ret, self.buf = self.buf, None
.. return ret
.. else:
.. raise NoMoreLineError
.. if avail:
.. self.buf += os.read(self.fd, avail)
..
.. idx = self.buf.find('\n')
.. if idx >= 0:
.. ret, self.buf = self.buf[:idx+1], self.buf[idx+1:]
.. return ret
.. else:
.. raise NoLineError
..
..
.. def multi_readlines(fs):
.. """Read lines from |fs|, a list of file objects.
.. The lines come out in arbitrary order, depending on which files
.. have output available first."""
.. if type(fs) not in (list, tuple):
.. raise Exception("argument must be a list.")
.. objs = [liner(f) for f in fs]
.. for i,obj in enumerate(objs): obj._index = i
.. while objs:
.. for i,obj in enumerate(objs):
.. try:
.. yield (obj._index, obj.getline())
.. except NoLineError:
.. pass
.. except NoMoreLineError:
.. del objs[i]
.. break # Because we mutated the array

Hi !

Sorry, but I want to share my experiences. I hope this help to you.

I think that specialized MSWindows based services too complicated. They have
to many bug possibilites.
So I trying with normal, "in python accessable" pipes. I see that with
flush(), and some of the bintotext tricks I can use the
subprocess/masterprocess communication.
Ok, this is not asynchronous. I can send some job to sp(s), and I can
receive the report from sp(s).
But with threading I can create non-blocking communication.

You can see in the example: the PipeBPPThr define a pipe based process-pool
thread.
This can communicate with a subprocess, can send/receive jobs, etc.

If you collect these threads, and write a process pool object, you can
handle all of the communications with one object.

I hope to these examples can help to you.

If not, you can try with wm_copydata messages in Windows.
http://msdn.microsoft.com/library/de...ngdatacopy.asp
This way of data exchanging is based on the message handling/sending.

dd