Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Python > Python Subprocess module

Reply
Thread Tools

Python Subprocess module

 
 
Dave Sampson
Guest
Posts: n/a
 
      07-13-2007
hey folks,

A simple question hopefully. despite all my searching I have not found a
satisfactory response.

The goal. Interact with a command line program. Simple enough, but the
key is INTERACT.

I tried the shell and comand approaches but that initiates, it does not
allow interaction with the programs.

So then I went with Popen and such... which then led to the subprocess
module.

I can create the object and read a few lines of output. but if I go too
far then the program hangs. the number of lines will differ depandening
on many function including the format of an input file so I can;t
hardcode how many lines to read.

I want to read all of STDOUT without failing because I went out of range.

next I want to read the final line of the output because it tells me
what is required for the next line of input.

I am supposed to be able to entre 'y' and return for the program to
continue if I agree with what I see in the stdout.

A problem exists though that I have tried
'y'
'y\n'
'y\r'

and nothing seems to get the program going again for I still cant; read
past the same point in the standard output. then I have to kill and
start over.

So the next approach included looking at Pexpect, which got realy
confusing realy fast and despite running fedora core and python 2.4.4 I
would like my application to be cross platform and there is no Pexpect
for Windows That I can see.

I have checked out many mailing lists and posts and tutorials but they
all admit to not be samples of complex interactivity. I guess I am
having that special case.

ASPN Python cookbok provided some code for a new Popen Class that allows
for interaction but I don't think I need to go that root....

I am essentialy trying to build and Python wrapper for a coomand line
program that i want to build a GUI around.

Any links or suggestions would be great. Unfortunaetly I don't
understand that abstract concepts in the manual reference. I am fresh in
the python world and find examples better than technical abstracts.

Cheers
 
Reply With Quote
 
 
 
 
Dan Stromberg - Datallegro
Guest
Posts: n/a
 
      07-13-2007

You could start up a pty and do your own Pexpect-like thing, using
read/readline/write.

However, this may not be portable to windows. Maybe if you get the python
that comes with cygwin, or build your own python under the cygwin
environment - that might be more likely to get you pty's under windows.

On Fri, 13 Jul 2007 17:15:49 -0400, Dave Sampson wrote:

> hey folks,
>
> A simple question hopefully. despite all my searching I have not found a
> satisfactory response.
>
> The goal. Interact with a command line program. Simple enough, but the
> key is INTERACT.
>
> I tried the shell and comand approaches but that initiates, it does not
> allow interaction with the programs.
>
> So then I went with Popen and such... which then led to the subprocess
> module.
>
> I can create the object and read a few lines of output. but if I go too
> far then the program hangs. the number of lines will differ depandening
> on many function including the format of an input file so I can;t
> hardcode how many lines to read.
>
> I want to read all of STDOUT without failing because I went out of range.
>
> next I want to read the final line of the output because it tells me
> what is required for the next line of input.
>
> I am supposed to be able to entre 'y' and return for the program to
> continue if I agree with what I see in the stdout.
>
> A problem exists though that I have tried
> 'y'
> 'y\n'
> 'y\r'
>
> and nothing seems to get the program going again for I still cant; read
> past the same point in the standard output. then I have to kill and
> start over.
>
> So the next approach included looking at Pexpect, which got realy
> confusing realy fast and despite running fedora core and python 2.4.4 I
> would like my application to be cross platform and there is no Pexpect
> for Windows That I can see.
>
> I have checked out many mailing lists and posts and tutorials but they
> all admit to not be samples of complex interactivity. I guess I am
> having that special case.
>
> ASPN Python cookbok provided some code for a new Popen Class that allows
> for interaction but I don't think I need to go that root....
>
> I am essentialy trying to build and Python wrapper for a coomand line
> program that i want to build a GUI around.
>
> Any links or suggestions would be great. Unfortunaetly I don't
> understand that abstract concepts in the manual reference. I am fresh in
> the python world and find examples better than technical abstracts.
>
> Cheers


 
Reply With Quote
 
 
 
 
Eric_Dexter@msn.com
Guest
Posts: n/a
 
      07-14-2007
On Jul 13, 4:15 pm, Dave Sampson <(E-Mail Removed)> wrote:
> hey folks,
>
> A simple question hopefully. despite all my searching I have not found a
> satisfactory response.
>
> The goal. Interact with a command line program. Simple enough, but the
> key is INTERACT.
>
> I tried the shell and comand approaches but that initiates, it does not
> allow interaction with the programs.
>
> So then I went with Popen and such... which then led to the subprocess
> module.
>
> I can create the object and read a few lines of output. but if I go too
> far then the program hangs. the number of lines will differ depandening
> on many function including the format of an input file so I can;t
> hardcode how many lines to read.
>
> I want to read all of STDOUT without failing because I went out of range.
>
> next I want to read the final line of the output because it tells me
> what is required for the next line of input.
>
> I am supposed to be able to entre 'y' and return for the program to
> continue if I agree with what I see in the stdout.
>
> A problem exists though that I have tried
> 'y'
> 'y\n'
> 'y\r'
>
> and nothing seems to get the program going again for I still cant; read
> past the same point in the standard output. then I have to kill and
> start over.
>
> So the next approach included looking at Pexpect, which got realy
> confusing realy fast and despite running fedora core and python 2.4.4 I
> would like my application to be cross platform and there is no Pexpect
> for Windows That I can see.
>
> I have checked out many mailing lists and posts and tutorials but they
> all admit to not be samples of complex interactivity. I guess I am
> having that special case.
>
> ASPN Python cookbok provided some code for a new Popen Class that allows
> for interaction but I don't think I need to go that root....
>
> I am essentialy trying to build and Python wrapper for a coomand line
> program that i want to build a GUI around.
>
> Any links or suggestions would be great. Unfortunaetly I don't
> understand that abstract concepts in the manual reference. I am fresh in
> the python world and find examples better than technical abstracts.
>
> Cheers


There is something called http://www.rutherfurd.net/python/sendkeys/
send keys for the p.c. you can use a try: except: concept and on the
windows platform there is also a way to use autoit in python using
pywin. At one time (2001) there was a module that allowed you to use
gvim but it has dissapeared unless it is listed somewhere else (gvim
doesn't seem to work with the sendkey method and it is a shame it has
alot of cool text functions including command line support)

http://sourceforge.net/projects/dex-tracker/

 
Reply With Quote
 
Noah
Guest
Posts: n/a
 
      07-15-2007
On Jul 13, 2:15 pm, Dave Sampson <(E-Mail Removed)> wrote:
> hey folks,
>
> A simple question hopefully. despite all my searching I have not found a
> satisfactory response.
>
> The goal. Interact with a command line program. Simple enough, but the
> key is INTERACT.


You need Pexpect.

> So then I went with Popen and such... which then led to the subprocess
> module. I can create the object and read a few lines of output. but if I go too
> far then the program hangs. the number of lines will differ depandening
> on many function including the format of an input file so I can;t
> hardcode how many lines to read.


There is nothing you can do about this when using a pipe.
This is because stdio will change the type of buffering when sending
stdout and stderr to a pipe. From the client side there is no way you
can change the buffer mode (you CAN change the mode, but it only
changes
it on YOUR side of the pipe -- not your child application's side).

> and nothing seems to get the program going again for I still cant; read
> past the same point in the standard output. then I have to kill and
> start over.


Your child application is waiting for you, but you can't see what it
wrote
becuase it has not filled or flushed the output buffer. When a program
write to stdout and it is connected to a TTY it will automatically
flush
the output buffer when it writes a newline (or, rather, the clib will
do this).
But when writing to a pipe it will automatically use block buffering
and
will only flush the buffer when it is full.

It's not very complicated, but it can be very confusing at first.
It is also surprising that this behavior cannot be changed by the
parent application.

> So the next approach included looking atPexpect, which got realy
> confusing realy fast and despite running fedora core and python 2.4.4 I
> would like my application to be cross platform and there is noPexpect
> for Windows That I can see.


You are going to need something like Pexpect that uses pseudo-ttys
(pty).
Pexpect works under Cygwin on Windows, but not under the native
Windows Python.

Email me if you have questions about Pexpect and I'll try to help you.

Yours,
Noah

 
Reply With Quote
 
Josiah Carlson
Guest
Posts: n/a
 
      07-16-2007
Dave Sampson wrote:
> I am supposed to be able to entre 'y' and return for the program to
> continue if I agree with what I see in the stdout.
>
> A problem exists though that I have tried
> 'y'
> 'y\n'
> 'y\r'


Depending on the platform, you may need to send '\r\n'.


> ASPN Python cookbok provided some code for a new Popen Class that allows
> for interaction but I don't think I need to go that root....


You probably mean:
http://aspn.activestate.com/ASPN/Coo.../Recipe/440554

Why is that not sufficient? It will allow interaction with a called
subprocess, in a cross-platform manner, etc.


- Josiah
 
Reply With Quote
 
Dave Sampson
Guest
Posts: n/a
 
      07-21-2007
On Jul 13, 7:50 pm, Dan Stromberg - Datallegro
<(E-Mail Removed)> wrote:
> You could start up a pty and do your own Pexpect-like thing, using
> read/readline/write.
>
> However, this may not be portable to windows. Maybe if you get the python
> that comes with cygwin, or build your own python under the cygwin



thanks for the suggestion. I will follow up on Noah's pexpect
sugestion bellow.

thanks for the input, cross platform is key with low overhead.

 
Reply With Quote
 
Dave Sampson
Guest
Posts: n/a
 
      07-21-2007
send keys for the p.c. you can use a try: except: concept and on the
> windows platform there is also a way to use autoit in python using
> pywin.


thanks for the uggestion about sendkeys. I might be able to build that
into the wrapper.

So you're sugesting:

NIX = Pexpect

WINDOZE = sendkeys

Correct?


 
Reply With Quote
 
Dave Sampson
Guest
Posts: n/a
 
      07-21-2007
Thanks for your response. It;s good to see you're promtoing Pexpect. I
was actualy wokring on a project at work that could have used Pexpect
to automate the creation of maps by tieing togethr a bunch of
processes through SSH and Pexpect looked like the money shot.

however it was enough of a battle to bring Python into the game since
it is government that bringing CYGWIN was a battle not worth
fighting. Which is unfortunate because I think Pexpect was exactly
what we needed in that case
>
> You need Pexpect.
>


> There is nothing you can do about this when using a pipe.
> This is because stdio will change the type of buffering when sending
> stdout and stderr to a pipe. From the client side there is no way you
> can change the buffer mode (you CAN change the mode, but it only
> changes
> it on YOUR side of the pipe -- not your child application's side).



I obviously lack skills and understanding in many aspects of
programming. After all I'm a geographer / recreologist not a comp-sci
professional.

I understand the CONCEPT of pipes but don't know how to use or
implement.. the bigest things I ever do is something like

dmesg | grep wlan0

Where | is my pipe... but that is the extent of my knowledge and
skill.

Pexpect may very well be the option I need, and having a project that
supports the module is great. I am curous what is stopping Pexpect
from becoming cross platform.

Above, a user suggested implementing send keys in windows. Why not
include that as an try/except in Pexpect instead of my wrapper? That
way it can benifit many more people?

Cheers


 
Reply With Quote
 
Dave Sampson
Guest
Posts: n/a
 
      07-21-2007

> You probably mean:http://aspn.activestate.com/ASPN/Coo.../Recipe/440554
>
> Why is that not sufficient? It will allow interaction with a called
> subprocess, in a cross-platform manner, etc.
>
> - Josiah


Sorry maybe 'not sufficient' was the wrong excuse. Maybe the right
thing to say is that when I look at it I don't know if I'm supposed to
copy paste in my own module, or paste into a seprate module, or use it
in a pre-existing module. Does the liscence for the content through
ASPN even allow reuse of code. or Am I supposed to modify it?

If I use, how is it to be used?. Wher eare the exmpales and
documentation.

By sufficient I mean I don't have sufficient info from the site to
implement it. And I don't have sufficient know how to implement it
base don the comments in the source code.

IN SHORT... IT'S SCARY....

So if you can answer any of those questions I am open.




 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
how to import subprocess into my 'subprocess.py' file hiral Python 2 05-05-2010 12:56 PM
Descriptor leak in python 2.4 subprocess module Michel Lespinasse Python 0 08-28-2008 09:09 AM
Re: [Python 2.4/2.5] subprocess module is sorely deficient? Harishankar Python 2 04-23-2008 02:38 PM
[Python 2.4/2.5] subprocess module is sorely deficient? Harishankar Python 18 04-23-2008 09:31 AM
[Subprocess/Windows] subprocess module under Windows 98 Andreas Jung Python 2 11-02-2005 05:41 PM



Advertisments