Velocity Reviews

Velocity Reviews (http://www.velocityreviews.com/forums/index.php)
-   Python (http://www.velocityreviews.com/forums/f43-python.html)
-   -   A tale of two execs (http://www.velocityreviews.com/forums/t671990-a-tale-of-two-execs.html)

aha 02-23-2009 04:53 PM

A tale of two execs
 
Hello All,
I am working on a project where I need to support versions of Python
as old as 2.3. Previously, we distributed Python with our product, but
this seemed a bit silly so we are no longer doing this. The problem
that I am faced with is that we have Python scripts that use the
subprocess module, and subprocess is not available in Python 2.3.
Below is the strategy I am thinking about using, however if, you have
better ideas please let me know.

def runner(cmd, stdin, stdout, ...):
try:
import subprocess
sbm = 1
except:
sbm = 0

# Now do something
if sbm:
process = subporcess(...)
else:
import popen2
process = popen2.Popen4(...)

Has anyone else run into a situation similar to this one?

bieffe62@gmail.com 02-23-2009 05:06 PM

Re: A tale of two execs
 
On Feb 23, 5:53*pm, aha <aquil.abdul...@gmail.com> wrote:
> Hello All,
> * I am working on a project where I need to support versions of Python
> as old as 2.3. Previously, we distributed Python with our product, but
> this seemed a bit silly so we are no longer doing this. *The problem
> that I am faced with is that we have Python scripts that use the
> subprocess module, and subprocess is not available in Python 2.3.
> Below is the strategy I am thinking about using, however if, you have
> better ideas please let me know.
>
> def runner(cmd, stdin, stdout, ...):
> * try:
> * * import subprocess
> * * sbm = 1
> * except:
> * * sbm = 0
>
> * # Now do something
> * if sbm:
> * * process = subporcess(...)
> * else:
> * * import popen2
> * * process = popen2.Popen4(...)
>
> Has anyone else run into a situation similar to this one?


IIRC, subprocess is a pure python module. If so, tou could try if
subprocess compiles under 2.3.
If so, or if it has easily fixable problems, you could rename it as
someting like 'backported_subprocess.py'
and include it in your app, which then should do:

try:
import subprocess
except ImportError:
import backported_subprocess as subprocess

This should minimize the change at your code. Of course if the usage
of subprocess is restricted
to few functions, your approach to provide two alternative
implementations of those function
is also workable. However, check if/when the module popen2, which is
deprecated, is planned for
remioval, otherwise you will have the same problem later.

HTH

Ciao
------
FB

Matt Nordhoff 02-23-2009 05:07 PM

Re: A tale of two execs
 
aha wrote:
> Hello All,
> I am working on a project where I need to support versions of Python
> as old as 2.3. Previously, we distributed Python with our product, but
> this seemed a bit silly so we are no longer doing this. The problem
> that I am faced with is that we have Python scripts that use the
> subprocess module, and subprocess is not available in Python 2.3.
> Below is the strategy I am thinking about using, however if, you have
> better ideas please let me know.
>
> def runner(cmd, stdin, stdout, ...):
> try:
> import subprocess
> sbm = 1
> except:
> sbm = 0
>
> # Now do something
> if sbm:
> process = subporcess(...)
> else:
> import popen2
> process = popen2.Popen4(...)
>
> Has anyone else run into a situation similar to this one?


FWIW, the Mercurial project went the other way: They wrote "popen2",
"popen3" and "Popen3" functions that were wrappers around subprocess:

<http://selenic.com/repo/hg-stable/file/dd970a311ea8/mercurial/util.py#l52>
--

bieffe62@gmail.com 02-23-2009 05:28 PM

Re: A tale of two execs
 
On Feb 23, 6:06*pm, bieff...@gmail.com wrote:
> On Feb 23, 5:53*pm, aha <aquil.abdul...@gmail.com> wrote:
>
>
>
>
>
> > Hello All,
> > * I am working on a project where I need to support versions of Python
> > as old as 2.3. Previously, we distributed Python with our product, but
> > this seemed a bit silly so we are no longer doing this. *The problem
> > that I am faced with is that we have Python scripts that use the
> > subprocess module, and subprocess is not available in Python 2.3.
> > Below is the strategy I am thinking about using, however if, you have
> > better ideas please let me know.

>
> > def runner(cmd, stdin, stdout, ...):
> > * try:
> > * * import subprocess
> > * * sbm = 1
> > * except:
> > * * sbm = 0

>
> > * # Now do something
> > * if sbm:
> > * * process = subporcess(...)
> > * else:
> > * * import popen2
> > * * process = popen2.Popen4(...)

>
> > Has anyone else run into a situation similar to this one?

>
> IIRC, *subprocess is a pure python module. If so, tou could try if
> subprocess compiles under 2.3.


....

I checked, and, for windows platform subprocess.py uses the modules
mvscrt and _subprocess, which I ham unable to
locate on my windows XP python 2.6 installation. This make the whole
thing harder, even impossible if _subprocess has
been created especially for subprocess.py.

For non-windows platform, subprocess.py seem to use only fairly well-
established module, so there is a chance to backport it.

Ciao again
----------
FB

aha 02-23-2009 09:54 PM

Re: A tale of two execs
 
Hello All,
So below is my naive attempt at the wrapper, it is only half baked
since I am no master at Interprocess communication... I know that it
is lacking a lot of things comment greatly appreciated:

#!/usr/bin/env python

import os, sys, re, exceptions
try:
import subprocess
except ImportError:
import popen2
subprocess = None

PIPE = 10
SOUT = 11

class Runner:
def __init__(self):
self.stdout = None
self.stdin = None
self.stderr = None
self.pid = None
self.process = None

def runCmdLine(self, args, executable=None, stdin=None,
stdout=None, stderr=None, preexec_fn=None,
close_fds=False, shell=False,cwd=None, env=None):
"""
args -- a string, or a sequence of program arguments. The
argument to execute is
normally the first item in the args sequence or the string if
a string is given,
but can be explicitly set by using the executable argument.

executable -- string to be executed

stdin, stdout, stderr -- value of standard input, standard
output, and standard
error file handles. None implies no redirection. Valid values
are PIPE, an existing
file descriptor (a positive integer), an existing file object,
and None.

shell -- specifies whether command should be executed through
shell

cwd -- specifies whether, childs current directory will be
changed to cwd

env -- A map that specifies the environment variables in child
process
that will be overwritten

NOTE: Depending on the modules available, some arguments may
not be used.
"""
if subprocess:
# Use subprocess to exec functions
if stdin == PIPE:
sin = subprocess.PIPE
else:
# NOTE: User may input invalid values that will cause
exceptions
sin = stdin
if stdout == PIPE:
sout = subprocess.PIPE
else:
sout = stdout
if stderr == SOUT:
serr = subprocess.STDOUT
else:
serr = stderr

self.process = subprocess.Popen(args, stdin=sin,
stdout=sout, stderr=serr,

preexec_fn=preexec_fn,close_fds=close_fds, shell=shell,
env=env, cwd=cwd)
self.pid = self.process.pid
# Add attributes stdin, stdout, and stderr
self.stdin = self.process.stdin
self.stdout = self.process.stdout
self.stderr = self.process.stderr
else:
# Use popen2 to exec functions
# Determine which form of popen2 to use
if stderr == SOUT or stderr == None:
self.process = popen2.Popen4(args)
self.stdin = self.process.tochild
self.stdout = self.process.fromchild
self.stderr = None
else:
self.process = popen2.Popen3(args)
self.stdin = self.process.tochild
self.stdout = self.process.fromchild
self.stderr = self.process.childerr

self.pid = self.process.pid

def wait(self,):
"""
Waits for and returns the status code of the child process.
"""
return self.process.wait()

def poll(self,):
"""
Returns -1 if the child process hasn't completed yet, or it's
return code
otherwise
"""
return self.process.poll()

Aquil

On Feb 23, 2:10*pm, Christian Heimes <li...@cheimes.de> wrote:
> aha wrote
>
> > def runner(cmd, stdin, stdout, ...):
> > * try:
> > * * import subprocess
> > * * sbm = 1
> > * except:
> > * * sbm = 0

>
> > * # Now do something
> > * if sbm:
> > * * process = subporcess(...)
> > * else:
> > * * import popen2
> > * * process = popen2.Popen4(...)

>
> > Has anyone else run into a situation similar to this one?

>
> The canonical way for your try/except clause is:
>
> try:
> * * import subprocess
> except ImportError:
> * * subprocess = None
>
> ...
>
> if subprocess is not None:
> * * ...
> else:
> * * ...
>
> Christian



aha 02-23-2009 10:20 PM

Re: A tale of two execs
 
I've decided to change the runCmdLine method to Popen.
On Feb 23, 4:54*pm, aha <aquil.abdul...@gmail.com> wrote:
> Hello All,
> * So below is my naive attempt at the wrapper, it is only half baked
> since I am no master at Interprocess communication... I know that it
> is lacking a lot of things comment greatly appreciated:
>
> #!/usr/bin/env python
>
> import os, sys, re, exceptions
> try:
> * * import subprocess
> except ImportError:
> * * import popen2
> * * subprocess = None
>
> PIPE = 10
> SOUT = 11
>
> class Runner:
> * * def __init__(self):
> * * * * self.stdout = None
> * * * * self.stdin = None
> * * * * self.stderr = None
> * * * * self.pid = None
> * * * * self.process = None
>
> * * def runCmdLine(self, args, executable=None, stdin=None,
> stdout=None, stderr=None, preexec_fn=None,
> * * * * * * * * * *close_fds=False, shell=False,cwd=None, env=None):
> * * * * """
> * * * * args -- a string, or a sequence of program arguments. The
> argument to execute is
> * * * * normally the first item in the args sequence or the string if
> a string is given,
> * * * * but can be explicitly set by using the executable argument.
>
> * * * * executable -- string to be executed
>
> * * * * stdin, stdout, stderr -- value of standard input, standard
> output, and standard
> * * * * error file handles. *None implies no redirection. Valid values
> are PIPE, an existing
> * * * * file descriptor (a positive integer), an existing file object,
> and None.
>
> * * * * shell -- specifies whether command should be executed through
> shell
>
> * * * * cwd -- specifies whether, childs current directory will be
> changed to cwd
>
> * * * * env -- A map that specifies the environment variables in child
> process
> * * * * that will be overwritten
>
> * * * * NOTE: Depending on the modules available, some arguments may
> not be used.
> * * * * """
> * * * * if subprocess:
> * * * * * * # Use subprocess to exec functions
> * * * * * * if stdin == PIPE:
> * * * * * * * * sin = subprocess.PIPE
> * * * * * * else:
> * * * * * * * * # NOTE: User may input invalid values that will cause
> exceptions
> * * * * * * * * sin = stdin
> * * * * * * if stdout == PIPE:
> * * * * * * * * sout = subprocess.PIPE
> * * * * * * else:
> * * * * * * * * sout = stdout
> * * * * * * if stderr == SOUT:
> * * * * * * * * serr = subprocess.STDOUT
> * * * * * * else:
> * * * * * * * * serr = stderr
>
> * * * * * * self.process = subprocess.Popen(args, stdin=sin,
> stdout=sout, stderr=serr,
>
> preexec_fn=preexec_fn,close_fds=close_fds, shell=shell,
> * * * * * * * * * * * * * * * * * * * * * * env=env, cwd=cwd)
> * * * * * * self.pid = self.process.pid
> * * * * * *# Add attributes stdin, stdout, and stderr
> * * * * * * self.stdin = self.process.stdin
> * * * * * * self.stdout = self.process.stdout
> * * * * * * self.stderr = self.process.stderr
> * * * * else:
> * * * * * * # Use popen2 to exec functions
> * * * * * * # Determine which form of popen2 to use
> * * * * * * if stderr == SOUT or stderr == None:
> * * * * * * * * self.process = *popen2.Popen4(args)
> * * * * * * * * self.stdin = self.process.tochild
> * * * * * * * * self.stdout = self.process.fromchild
> * * * * * * * * self.stderr = None
> * * * * * * else:
> * * * * * * * * self.process = *popen2.Popen3(args)
> * * * * * * * * self.stdin = self.process.tochild
> * * * * * * * * self.stdout = self.process.fromchild
> * * * * * * * * self.stderr = self.process.childerr
>
> * * * * * * self.pid = self.process.pid
>
> * * def wait(self,):
> * * * * """
> * * * * Waits for and returns the status code of the child process.
> * * * * """
> * * * * return self.process.wait()
>
> * * def poll(self,):
> * * * * """
> * * * * Returns -1 if the child process hasn't completed yet, or it's
> return code
> * * * * otherwise
> * * * * """
> * * * * return self.process.poll()
>
> Aquil
>
> On Feb 23, 2:10*pm, Christian Heimes <li...@cheimes.de> wrote:
>
> > aha wrote

>
> > > def runner(cmd, stdin, stdout, ...):
> > > * try:
> > > * * import subprocess
> > > * * sbm = 1
> > > * except:
> > > * * sbm = 0

>
> > > * # Now do something
> > > * if sbm:
> > > * * process = subporcess(...)
> > > * else:
> > > * * import popen2
> > > * * process = popen2.Popen4(...)

>
> > > Has anyone else run into a situation similar to this one?

>
> > The canonical way for your try/except clause is:

>
> > try:
> > * * import subprocess
> > except ImportError:
> > * * subprocess = None

>
> > ...

>
> > if subprocess is not None:
> > * * ...
> > else:
> > * * ...

>
> > Christian

>
>



Steve Holden 02-23-2009 11:00 PM

Re: A tale of two execs
 
aha wrote:
> I've decided to change the runCmdLine method to Popen.
> [...]


So just exactly why was it necessary to follow this remark with about
three hundred lines of quoted text? And can't you put your replies at
the bottom rather than the top, please? And trim the stuff that isn't
required. It's simple courtesy to your readers.

regards
Steve
--
Steve Holden +1 571 484 6266 +1 800 494 3119
Holden Web LLC http://www.holdenweb.com/


aha 02-24-2009 05:13 AM

Re: A tale of two execs
 
Hello All,
It occurred to me that I might just want to try copying the
subprocess.py installed in a /usr/lib/python24 installation to the
directory where I place the scripts that I need to launch my
application...I know this was mentioned earlier. Anyway, my
application worked under python 2.3 and later python versions as
well. Here is what I did:

I renamed subprocess.py to subprocess23.py and then used the following
code:

try:
import subprocess
except ImportError:
import subprocess23 as subprocess

I chose this route because I took a deeper look at the code in the
subprocess module and I also realized that I would not be able to use
the popen2, because I would not be able to do things like os.setpgrp
for preexec_fn, with popen2. In the end, if I wanted that type of
functionality I would have to use os.dup, fork, exec, which meant
reinventing the wheel. I overcame the issue of dealing with msvcrt
and _subprocess under windows by requiring python24 or greater under
windows.

Regards,
Aquil

Carl Banks 02-24-2009 10:04 AM

Re: A tale of two execs
 
On Feb 23, 9:13*pm, aha <aquil.abdul...@gmail.com> wrote:
> Hello All,
> * It occurred to me that I might just want to try copying the
> subprocess.py installed in a /usr/lib/python24 installation to the
> directory where I place the scripts that I need to launch my
> application...I know this was mentioned earlier. *Anyway, my
> application worked under python 2.3 and later python versions as
> well. *Here is what I did:
>
> I renamed subprocess.py to subprocess23.py and then used the following
> code:
>
> try:
> * import subprocess
> except ImportError:
> * import subprocess23 as subprocess
>
> I chose this route because I took a deeper look at the code in the
> subprocess module and I also realized that I would not be able to use
> the popen2, because I would not be able to do things like os.setpgrp
> for preexec_fn, with popen2. *In the end, if I wanted that type of
> functionality I would have to use os.dup, fork, exec, which meant
> reinventing the wheel. *I overcame the issue of dealing with msvcrt
> and _subprocess under windows by requiring python24 or greater under
> windows.



Distributing the interpreter doesn't look quite as silly as it did
this morning now, does it?


Carl Banks

Aahz 03-12-2009 09:59 PM

Re: A tale of two execs
 
[slowly catching up...]

In article <9095ae87-c11d-4439-aeed-05533c610e54@m24g2000vbp.googlegroups.com>,
Carl Banks <pavlovevidence@gmail.com> wrote:
>
>Distributing the interpreter doesn't look quite as silly as it did
>this morning now, does it?


+1 QOTW, three weeks late
--
Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/

"All problems in computer science can be solved by another level of
indirection." --Butler Lampson


All times are GMT. The time now is 04:45 AM.

Powered by vBulletin®. Copyright ©2000 - 2014, vBulletin Solutions, Inc.
SEO by vBSEO ©2010, Crawlability, Inc.