Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Python > socket and subprocess problem

Reply
Thread Tools

socket and subprocess problem

 
 
goatold@gmail.com
Guest
Posts: n/a
 
      12-16-2008
Hi all,

Here is my problem, see if any one else met this before
In my python code I use subprocess.Popen to run and external program
who will listen to a TCP port. And I also create a socket to connect
to the TCP port that the external program is listening.
I will get 'Connection refused, errno=111' when I try to socket.connect
().
But if I run my subprocess.Popen code and socket code in two separate
python process. Socket will connect just fine. OS is RHEL5 x86_64,
python version is 2.6.1 Pseudo code as below


Class a:
def run()
subprocess.Popen(..)
Class b:
def run():
sock = socket.socket()
sock.connect(..)
#################################
test.py
# socket connect will fail here
a.run()
b.run()
###################################
test1.py
if __name__ = '__main__':
a.run()

test2.py
# socket will connect fine
if __name__ = '__main__':
b.run
 
Reply With Quote
 
 
 
 
Bryan Olson
Guest
Posts: n/a
 
      12-16-2008
http://www.velocityreviews.com/forums/(E-Mail Removed) wrote:
> In my python code I use subprocess.Popen to run and external program
> who will listen to a TCP port. And I also create a socket to connect
> to the TCP port that the external program is listening.
> I will get 'Connection refused, errno=111' when I try to socket.connect
> ().


Looks like a race. The first process tries to connect before the
external program gets through socket.listen(). If you think about it,
it's not that surprising.

> But if I run my subprocess.Popen code and socket code in two separate
> python process. Socket will connect just fine.


It's still a race condition even if the side you want to win almost
always does.


--
--Bryan
 
Reply With Quote
 
 
 
 
Roy Smith
Guest
Posts: n/a
 
      12-16-2008
In article
<(E-Mail Removed)>,
(E-Mail Removed) wrote:

> In my python code I use subprocess.Popen to run and external program
> who will listen to a TCP port. And I also create a socket to connect
> to the TCP port that the external program is listening.
> I will get 'Connection refused, errno=111' when I try to
> socket.connect().


> Class a:
> def run()
> subprocess.Popen(..)
> Class b:
> def run():
> sock = socket.socket()
> sock.connect(..)
> #################################
> test.py
> # socket connect will fail here
> a.run()
> b.run()
> ###################################
> test1.py
> if __name__ = '__main__':
> a.run()
>
> test2.py
> # socket will connect fine
> if __name__ = '__main__':
> b.run


Sounds like a timing problem. I assume that the process started by a.run()
creates a socket and does a bind/listen/accept sequence on it. The problem
is, there's nothing in your code which guarantees that this happens before
b.run() executes the connect() call.

The cheesy way to test this is to sleep for a second somewhere between
a.run() and b.run(). See if that helps.

If it doesn't, then it's possible the process started by a.run() isn't
doing what it's supposed to do. Try running test1.py, and while it's
running, run netstat to see if you've got something listening on the port
you expect.
 
Reply With Quote
 
goatold@gmail.com
Guest
Posts: n/a
 
      12-16-2008
Guys thanks to point it out.
Yes, it's a race problem. I tried sleep long enough, then I can
connect to the socket. I should add code to try to connect to the
socket for a given time out.

Roy Smith wrote:
> In article
> <(E-Mail Removed)>,
> (E-Mail Removed) wrote:
>
> > In my python code I use subprocess.Popen to run and external program
> > who will listen to a TCP port. And I also create a socket to connect
> > to the TCP port that the external program is listening.
> > I will get 'Connection refused, errno=111' when I try to
> > socket.connect().

>
> > Class a:
> > def run()
> > subprocess.Popen(..)
> > Class b:
> > def run():
> > sock = socket.socket()
> > sock.connect(..)
> > #################################
> > test.py
> > # socket connect will fail here
> > a.run()
> > b.run()
> > ###################################
> > test1.py
> > if __name__ = '__main__':
> > a.run()
> >
> > test2.py
> > # socket will connect fine
> > if __name__ = '__main__':
> > b.run

>
> Sounds like a timing problem. I assume that the process started by a.run()
> creates a socket and does a bind/listen/accept sequence on it. The problem
> is, there's nothing in your code which guarantees that this happens before
> b.run() executes the connect() call.
>
> The cheesy way to test this is to sleep for a second somewhere between
> a.run() and b.run(). See if that helps.
>
> If it doesn't, then it's possible the process started by a.run() isn't
> doing what it's supposed to do. Try running test1.py, and while it's
> running, run netstat to see if you've got something listening on the port
> you expect.

 
Reply With Quote
 
James Mills
Guest
Posts: n/a
 
      12-16-2008
On Tue, Dec 16, 2008 at 3:30 PM, <(E-Mail Removed)> wrote:
> Guys thanks to point it out.
> Yes, it's a race problem. I tried sleep long enough, then I can
> connect to the socket. I should add code to try to connect to the
> socket for a given time out.


This is where event-driven approaches
become really useful

subprocess process:

#1. When my subprocess process has successfully
started notify the parent.
#2. When my subprocess process has successfully
created a listening socket, notify the parent.

parent process:

#1. When our subprocess process has
successfully started a listening socket
initiate a connection.

I could implement a prototype of this
if the OP is interested.

--JamesMills
 
Reply With Quote
 
Bryan Olson
Guest
Posts: n/a
 
      12-16-2008
(E-Mail Removed) wrote:
> Guys thanks to point it out.
> Yes, it's a race problem. I tried sleep long enough, then I can
> connect to the socket. I should add code to try to connect to the
> socket for a given time out.


As Roy noted, that's "the cheesy way". Are the kind of programmers who
accept cheesy solutions? Of course not.

The right solution is for the child process to tell the parent when the
port is ready for connections. There are a variety of ways to transfer
that message; the most straightforward is for the child process to write
something to its standard output.

Parent process:

Launch child.
read() child's output, waiting for the ready message.
connect() to the port the child established.

Child process:

Create the socket; bind(); listen().
Write the ready message to stdout.
accept().

There's a subtle yet important point here, that frequently gets lost:

Roy Smith wrote:
>> Sounds like a timing problem. I assume that the process started by a.run()
>> creates a socket and does a bind/listen/accept sequence on it. The problem
>> is, there's nothing in your code which guarantees that this happens before
>> b.run() executes the connect() call.


Actually, it's just bind() and listen() that have to happen before the
client can connect(); accept() will not succeed until after a client
connects. In our example the child process is the server, and the
server's readiness alert should go after his call to listen() returns,
but before he calls accept().


--
--Bryan
 
Reply With Quote
 
Bryan Olson
Guest
Posts: n/a
 
      12-18-2008
James Mills wrote:
> subprocess process:
>
> #1. When my subprocess process has successfully
> started notify the parent.
> #2. When my subprocess process has successfully
> created a listening socket, notify the parent.
>
> parent process:
>
> #1. When our subprocess process has
> successfully started a listening socket
> initiate a connection.


I'd swear James copied my response, except his came first. Even the
formatting came out similar. I hadn't seen his response when I wrote
mine, and wouldn't have bothered posing the same thing again.


--
--Bryan




 
Reply With Quote
 
James Mills
Guest
Posts: n/a
 
      12-18-2008
On Thu, Dec 18, 2008 at 8:00 PM, Bryan Olson <(E-Mail Removed)> wrote:
> I'd swear James copied my response, except his came first. Even the
> formatting came out similar. I hadn't seen his response when I wrote mine,
> and wouldn't have bothered posing the same thing again.


Great minds think alike huh
You should check out my circuits (1) library!

cheers
James

1: http://trac.softcircuit.net.au/circuits/
 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Re: socket.unbind or socket.unlisten? - socket.error: (48, 'Addressalready in use') Steve Holden Python 1 02-03-2009 06:20 AM
Re: socket.unbind or socket.unlisten? - socket.error: (48, 'Addressalready in use') Steve Holden Python 0 02-01-2009 12:45 PM
Re: socket.unbind or socket.unlisten? - socket.error: (48, 'Addressalready in use') Laszlo Nagy Python 0 02-01-2009 07:37 AM
socket.unbind or socket.unlisten? - socket.error: (48, 'Addressalready in use') Laszlo Nagy Python 1 01-27-2009 05:05 PM
Re: socket.unbind or socket.unlisten? - socket.error: (48,'Address already in use') Jean-Paul Calderone Python 0 01-27-2009 01:41 PM



Advertisments