Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Python > A tale of two execs

Reply
Thread Tools

A tale of two execs

 
 
aha
Guest
Posts: n/a
 
      02-23-2009
Hello All,
I am working on a project where I need to support versions of Python
as old as 2.3. Previously, we distributed Python with our product, but
this seemed a bit silly so we are no longer doing this. The problem
that I am faced with is that we have Python scripts that use the
subprocess module, and subprocess is not available in Python 2.3.
Below is the strategy I am thinking about using, however if, you have
better ideas please let me know.

def runner(cmd, stdin, stdout, ...):
try:
import subprocess
sbm = 1
except:
sbm = 0

# Now do something
if sbm:
process = subporcess(...)
else:
import popen2
process = popen2.Popen4(...)

Has anyone else run into a situation similar to this one?
 
Reply With Quote
 
 
 
 
bieffe62@gmail.com
Guest
Posts: n/a
 
      02-23-2009
On Feb 23, 5:53*pm, aha <(E-Mail Removed)> wrote:
> Hello All,
> * I am working on a project where I need to support versions of Python
> as old as 2.3. Previously, we distributed Python with our product, but
> this seemed a bit silly so we are no longer doing this. *The problem
> that I am faced with is that we have Python scripts that use the
> subprocess module, and subprocess is not available in Python 2.3.
> Below is the strategy I am thinking about using, however if, you have
> better ideas please let me know.
>
> def runner(cmd, stdin, stdout, ...):
> * try:
> * * import subprocess
> * * sbm = 1
> * except:
> * * sbm = 0
>
> * # Now do something
> * if sbm:
> * * process = subporcess(...)
> * else:
> * * import popen2
> * * process = popen2.Popen4(...)
>
> Has anyone else run into a situation similar to this one?


IIRC, subprocess is a pure python module. If so, tou could try if
subprocess compiles under 2.3.
If so, or if it has easily fixable problems, you could rename it as
someting like 'backported_subprocess.py'
and include it in your app, which then should do:

try:
import subprocess
except ImportError:
import backported_subprocess as subprocess

This should minimize the change at your code. Of course if the usage
of subprocess is restricted
to few functions, your approach to provide two alternative
implementations of those function
is also workable. However, check if/when the module popen2, which is
deprecated, is planned for
remioval, otherwise you will have the same problem later.

HTH

Ciao
------
FB
 
Reply With Quote
 
 
 
 
Matt Nordhoff
Guest
Posts: n/a
 
      02-23-2009
aha wrote:
> Hello All,
> I am working on a project where I need to support versions of Python
> as old as 2.3. Previously, we distributed Python with our product, but
> this seemed a bit silly so we are no longer doing this. The problem
> that I am faced with is that we have Python scripts that use the
> subprocess module, and subprocess is not available in Python 2.3.
> Below is the strategy I am thinking about using, however if, you have
> better ideas please let me know.
>
> def runner(cmd, stdin, stdout, ...):
> try:
> import subprocess
> sbm = 1
> except:
> sbm = 0
>
> # Now do something
> if sbm:
> process = subporcess(...)
> else:
> import popen2
> process = popen2.Popen4(...)
>
> Has anyone else run into a situation similar to this one?


FWIW, the Mercurial project went the other way: They wrote "popen2",
"popen3" and "Popen3" functions that were wrappers around subprocess:

<http://selenic.com/repo/hg-stable/file/dd970a311ea8/mercurial/util.py#l52>
--
 
Reply With Quote
 
bieffe62@gmail.com
Guest
Posts: n/a
 
      02-23-2009
On Feb 23, 6:06*pm, (E-Mail Removed) wrote:
> On Feb 23, 5:53*pm, aha <(E-Mail Removed)> wrote:
>
>
>
>
>
> > Hello All,
> > * I am working on a project where I need to support versions of Python
> > as old as 2.3. Previously, we distributed Python with our product, but
> > this seemed a bit silly so we are no longer doing this. *The problem
> > that I am faced with is that we have Python scripts that use the
> > subprocess module, and subprocess is not available in Python 2.3.
> > Below is the strategy I am thinking about using, however if, you have
> > better ideas please let me know.

>
> > def runner(cmd, stdin, stdout, ...):
> > * try:
> > * * import subprocess
> > * * sbm = 1
> > * except:
> > * * sbm = 0

>
> > * # Now do something
> > * if sbm:
> > * * process = subporcess(...)
> > * else:
> > * * import popen2
> > * * process = popen2.Popen4(...)

>
> > Has anyone else run into a situation similar to this one?

>
> IIRC, *subprocess is a pure python module. If so, tou could try if
> subprocess compiles under 2.3.


....

I checked, and, for windows platform subprocess.py uses the modules
mvscrt and _subprocess, which I ham unable to
locate on my windows XP python 2.6 installation. This make the whole
thing harder, even impossible if _subprocess has
been created especially for subprocess.py.

For non-windows platform, subprocess.py seem to use only fairly well-
established module, so there is a chance to backport it.

Ciao again
----------
FB
 
Reply With Quote
 
aha
Guest
Posts: n/a
 
      02-23-2009
Hello All,
So below is my naive attempt at the wrapper, it is only half baked
since I am no master at Interprocess communication... I know that it
is lacking a lot of things comment greatly appreciated:

#!/usr/bin/env python

import os, sys, re, exceptions
try:
import subprocess
except ImportError:
import popen2
subprocess = None

PIPE = 10
SOUT = 11

class Runner:
def __init__(self):
self.stdout = None
self.stdin = None
self.stderr = None
self.pid = None
self.process = None

def runCmdLine(self, args, executable=None, stdin=None,
stdout=None, stderr=None, preexec_fn=None,
close_fds=False, shell=False,cwd=None, env=None):
"""
args -- a string, or a sequence of program arguments. The
argument to execute is
normally the first item in the args sequence or the string if
a string is given,
but can be explicitly set by using the executable argument.

executable -- string to be executed

stdin, stdout, stderr -- value of standard input, standard
output, and standard
error file handles. None implies no redirection. Valid values
are PIPE, an existing
file descriptor (a positive integer), an existing file object,
and None.

shell -- specifies whether command should be executed through
shell

cwd -- specifies whether, childs current directory will be
changed to cwd

env -- A map that specifies the environment variables in child
process
that will be overwritten

NOTE: Depending on the modules available, some arguments may
not be used.
"""
if subprocess:
# Use subprocess to exec functions
if stdin == PIPE:
sin = subprocess.PIPE
else:
# NOTE: User may input invalid values that will cause
exceptions
sin = stdin
if stdout == PIPE:
sout = subprocess.PIPE
else:
sout = stdout
if stderr == SOUT:
serr = subprocess.STDOUT
else:
serr = stderr

self.process = subprocess.Popen(args, stdin=sin,
stdout=sout, stderr=serr,

preexec_fn=preexec_fn,close_fds=close_fds, shell=shell,
env=env, cwd=cwd)
self.pid = self.process.pid
# Add attributes stdin, stdout, and stderr
self.stdin = self.process.stdin
self.stdout = self.process.stdout
self.stderr = self.process.stderr
else:
# Use popen2 to exec functions
# Determine which form of popen2 to use
if stderr == SOUT or stderr == None:
self.process = popen2.Popen4(args)
self.stdin = self.process.tochild
self.stdout = self.process.fromchild
self.stderr = None
else:
self.process = popen2.Popen3(args)
self.stdin = self.process.tochild
self.stdout = self.process.fromchild
self.stderr = self.process.childerr

self.pid = self.process.pid

def wait(self,):
"""
Waits for and returns the status code of the child process.
"""
return self.process.wait()

def poll(self,):
"""
Returns -1 if the child process hasn't completed yet, or it's
return code
otherwise
"""
return self.process.poll()

Aquil

On Feb 23, 2:10*pm, Christian Heimes <(E-Mail Removed)> wrote:
> aha wrote
>
> > def runner(cmd, stdin, stdout, ...):
> > * try:
> > * * import subprocess
> > * * sbm = 1
> > * except:
> > * * sbm = 0

>
> > * # Now do something
> > * if sbm:
> > * * process = subporcess(...)
> > * else:
> > * * import popen2
> > * * process = popen2.Popen4(...)

>
> > Has anyone else run into a situation similar to this one?

>
> The canonical way for your try/except clause is:
>
> try:
> * * import subprocess
> except ImportError:
> * * subprocess = None
>
> ...
>
> if subprocess is not None:
> * * ...
> else:
> * * ...
>
> Christian


 
Reply With Quote
 
aha
Guest
Posts: n/a
 
      02-23-2009
I've decided to change the runCmdLine method to Popen.
On Feb 23, 4:54*pm, aha <(E-Mail Removed)> wrote:
> Hello All,
> * So below is my naive attempt at the wrapper, it is only half baked
> since I am no master at Interprocess communication... I know that it
> is lacking a lot of things comment greatly appreciated:
>
> #!/usr/bin/env python
>
> import os, sys, re, exceptions
> try:
> * * import subprocess
> except ImportError:
> * * import popen2
> * * subprocess = None
>
> PIPE = 10
> SOUT = 11
>
> class Runner:
> * * def __init__(self):
> * * * * self.stdout = None
> * * * * self.stdin = None
> * * * * self.stderr = None
> * * * * self.pid = None
> * * * * self.process = None
>
> * * def runCmdLine(self, args, executable=None, stdin=None,
> stdout=None, stderr=None, preexec_fn=None,
> * * * * * * * * * *close_fds=False, shell=False,cwd=None, env=None):
> * * * * """
> * * * * args -- a string, or a sequence of program arguments. The
> argument to execute is
> * * * * normally the first item in the args sequence or the string if
> a string is given,
> * * * * but can be explicitly set by using the executable argument.
>
> * * * * executable -- string to be executed
>
> * * * * stdin, stdout, stderr -- value of standard input, standard
> output, and standard
> * * * * error file handles. *None implies no redirection. Valid values
> are PIPE, an existing
> * * * * file descriptor (a positive integer), an existing file object,
> and None.
>
> * * * * shell -- specifies whether command should be executed through
> shell
>
> * * * * cwd -- specifies whether, childs current directory will be
> changed to cwd
>
> * * * * env -- A map that specifies the environment variables in child
> process
> * * * * that will be overwritten
>
> * * * * NOTE: Depending on the modules available, some arguments may
> not be used.
> * * * * """
> * * * * if subprocess:
> * * * * * * # Use subprocess to exec functions
> * * * * * * if stdin == PIPE:
> * * * * * * * * sin = subprocess.PIPE
> * * * * * * else:
> * * * * * * * * # NOTE: User may input invalid values that will cause
> exceptions
> * * * * * * * * sin = stdin
> * * * * * * if stdout == PIPE:
> * * * * * * * * sout = subprocess.PIPE
> * * * * * * else:
> * * * * * * * * sout = stdout
> * * * * * * if stderr == SOUT:
> * * * * * * * * serr = subprocess.STDOUT
> * * * * * * else:
> * * * * * * * * serr = stderr
>
> * * * * * * self.process = subprocess.Popen(args, stdin=sin,
> stdout=sout, stderr=serr,
>
> preexec_fn=preexec_fn,close_fds=close_fds, shell=shell,
> * * * * * * * * * * * * * * * * * * * * * * env=env, cwd=cwd)
> * * * * * * self.pid = self.process.pid
> * * * * * *# Add attributes stdin, stdout, and stderr
> * * * * * * self.stdin = self.process.stdin
> * * * * * * self.stdout = self.process.stdout
> * * * * * * self.stderr = self.process.stderr
> * * * * else:
> * * * * * * # Use popen2 to exec functions
> * * * * * * # Determine which form of popen2 to use
> * * * * * * if stderr == SOUT or stderr == None:
> * * * * * * * * self.process = *popen2.Popen4(args)
> * * * * * * * * self.stdin = self.process.tochild
> * * * * * * * * self.stdout = self.process.fromchild
> * * * * * * * * self.stderr = None
> * * * * * * else:
> * * * * * * * * self.process = *popen2.Popen3(args)
> * * * * * * * * self.stdin = self.process.tochild
> * * * * * * * * self.stdout = self.process.fromchild
> * * * * * * * * self.stderr = self.process.childerr
>
> * * * * * * self.pid = self.process.pid
>
> * * def wait(self,):
> * * * * """
> * * * * Waits for and returns the status code of the child process.
> * * * * """
> * * * * return self.process.wait()
>
> * * def poll(self,):
> * * * * """
> * * * * Returns -1 if the child process hasn't completed yet, or it's
> return code
> * * * * otherwise
> * * * * """
> * * * * return self.process.poll()
>
> Aquil
>
> On Feb 23, 2:10*pm, Christian Heimes <(E-Mail Removed)> wrote:
>
> > aha wrote

>
> > > def runner(cmd, stdin, stdout, ...):
> > > * try:
> > > * * import subprocess
> > > * * sbm = 1
> > > * except:
> > > * * sbm = 0

>
> > > * # Now do something
> > > * if sbm:
> > > * * process = subporcess(...)
> > > * else:
> > > * * import popen2
> > > * * process = popen2.Popen4(...)

>
> > > Has anyone else run into a situation similar to this one?

>
> > The canonical way for your try/except clause is:

>
> > try:
> > * * import subprocess
> > except ImportError:
> > * * subprocess = None

>
> > ...

>
> > if subprocess is not None:
> > * * ...
> > else:
> > * * ...

>
> > Christian

>
>


 
Reply With Quote
 
Steve Holden
Guest
Posts: n/a
 
      02-23-2009
aha wrote:
> I've decided to change the runCmdLine method to Popen.
> [...]


So just exactly why was it necessary to follow this remark with about
three hundred lines of quoted text? And can't you put your replies at
the bottom rather than the top, please? And trim the stuff that isn't
required. It's simple courtesy to your readers.

regards
Steve
--
Steve Holden +1 571 484 6266 +1 800 494 3119
Holden Web LLC http://www.holdenweb.com/

 
Reply With Quote
 
aha
Guest
Posts: n/a
 
      02-24-2009
Hello All,
It occurred to me that I might just want to try copying the
subprocess.py installed in a /usr/lib/python24 installation to the
directory where I place the scripts that I need to launch my
application...I know this was mentioned earlier. Anyway, my
application worked under python 2.3 and later python versions as
well. Here is what I did:

I renamed subprocess.py to subprocess23.py and then used the following
code:

try:
import subprocess
except ImportError:
import subprocess23 as subprocess

I chose this route because I took a deeper look at the code in the
subprocess module and I also realized that I would not be able to use
the popen2, because I would not be able to do things like os.setpgrp
for preexec_fn, with popen2. In the end, if I wanted that type of
functionality I would have to use os.dup, fork, exec, which meant
reinventing the wheel. I overcame the issue of dealing with msvcrt
and _subprocess under windows by requiring python24 or greater under
windows.

Regards,
Aquil
 
Reply With Quote
 
Carl Banks
Guest
Posts: n/a
 
      02-24-2009
On Feb 23, 9:13*pm, aha <(E-Mail Removed)> wrote:
> Hello All,
> * It occurred to me that I might just want to try copying the
> subprocess.py installed in a /usr/lib/python24 installation to the
> directory where I place the scripts that I need to launch my
> application...I know this was mentioned earlier. *Anyway, my
> application worked under python 2.3 and later python versions as
> well. *Here is what I did:
>
> I renamed subprocess.py to subprocess23.py and then used the following
> code:
>
> try:
> * import subprocess
> except ImportError:
> * import subprocess23 as subprocess
>
> I chose this route because I took a deeper look at the code in the
> subprocess module and I also realized that I would not be able to use
> the popen2, because I would not be able to do things like os.setpgrp
> for preexec_fn, with popen2. *In the end, if I wanted that type of
> functionality I would have to use os.dup, fork, exec, which meant
> reinventing the wheel. *I overcame the issue of dealing with msvcrt
> and _subprocess under windows by requiring python24 or greater under
> windows.



Distributing the interpreter doesn't look quite as silly as it did
this morning now, does it?


Carl Banks
 
Reply With Quote
 
Aahz
Guest
Posts: n/a
 
      03-12-2009
[slowly catching up...]

In article <(E-Mail Removed)>,
Carl Banks <(E-Mail Removed)> wrote:
>
>Distributing the interpreter doesn't look quite as silly as it did
>this morning now, does it?


+1 QOTW, three weeks late
--
Aahz ((E-Mail Removed)) <*> http://www.pythoncraft.com/

"All problems in computer science can be solved by another level of
indirection." --Butler Lampson
 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Re: Olympus says preparing legal action against execs John Turco Digital Photography 2 12-22-2011 08:36 AM
Re: Olympus says preparing legal action against execs RichA Digital Photography 4 11-21-2011 02:03 AM
Sigma execs contract advance Mad Cow disease RichA Digital Photography 14 06-12-2011 07:47 PM
Telecom NZ execs visit BT Mutlley NZ Computing 4 06-15-2006 01:28 AM
A Tale of Two PIXes Dustin Cisco 3 11-17-2005 05:27 PM



Advertisments