Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Python > Socket being garbage collected too early

Reply
Thread Tools

Socket being garbage collected too early

 
 
Scott Robinson
Guest
Posts: n/a
 
      12-16-2004
I have been having trouble with the garbage collector and sockets.
Unfortunately, google keeps telling me that the problem is the garbage
collector ignoring dead (closed?) sockets instead of removing live
ones. My problem is


x.sock=socket.socket(socket.AF_INET,socket.SOCK_ST REAM)
do_stuff(x.sock)


def do_stuff(sock):
sock_list.append(sock)


once do_stuff finishes, x.sock disappears, and I can only believe it
is being garbage collected. I'd like to hear the standard means for
avoiding this issue (gc appears to have only the interface to declare
something garbage, not to declare something not garbage).

Scott Robinson

 
Reply With Quote
 
 
 
 
Steve Holden
Guest
Posts: n/a
 
      12-17-2004
Scott Robinson wrote:

> I have been having trouble with the garbage collector and sockets.
> Unfortunately, google keeps telling me that the problem is the garbage
> collector ignoring dead (closed?) sockets instead of removing live
> ones. My problem is
>
>
> x.sock=socket.socket(socket.AF_INET,socket.SOCK_ST REAM)
> do_stuff(x.sock)
>
>
> def do_stuff(sock):
> sock_list.append(sock)
>
>
> once do_stuff finishes, x.sock disappears, and I can only believe it
> is being garbage collected. I'd like to hear the standard means for
> avoiding this issue (gc appears to have only the interface to declare
> something garbage, not to declare something not garbage).
>
> Scott Robinson
>

Unfortunately, assuming it's being garbage collected might turn out to
be incorrect. What evidence do you have that the socket "disappears"? Do
you get a segmentation ault, or what?

If the socket simply fails to work that would be a different case
altogether, but it seems to me that we need a bit more evodence that the
anecdotal stuff you've provided so far.

Quite apart from anything else, by the way, the code you posted appears
to use a global sock_list. A reference b y that list would in any case
stop the socket from being garbage collected (quite apart from the fact
that the socket module itself will do so as long as the socket is open).

So, could we see an error message, or some other evidence of what is
going on. For example, after the call to do_stuff(), what do you see if you

print sock_list

for example. I think your initial hypothesis is insufficient.

regards
Steve
--
Steve Holden http://www.holdenweb.com/
Python Web Programming http://pydish.holdenweb.com/
Holden Web LLC +1 703 861 4237 +1 800 494 3119
 
Reply With Quote
 
 
 
 
Mike Meyer
Guest
Posts: n/a
 
      12-17-2004
Scott Robinson <(E-Mail Removed)> writes:

> I have been having trouble with the garbage collector and sockets.
> Unfortunately, google keeps telling me that the problem is the garbage
> collector ignoring dead (closed?) sockets instead of removing live
> ones. My problem is
>
>
> x.sock=socket.socket(socket.AF_INET,socket.SOCK_ST REAM)
> do_stuff(x.sock)
>
>
> def do_stuff(sock):
> sock_list.append(sock)
>
>
> once do_stuff finishes, x.sock disappears, and I can only believe it
> is being garbage collected. I'd like to hear the standard means for
> avoiding this issue (gc appears to have only the interface to declare
> something garbage, not to declare something not garbage).


The code as shown doesn't work:

>> import socket
>> def do_stuff(sock):

.. sock_list.append(sock)
..
>> sock = socket.socket(socket.AF_INET,socket.SOCK_STREAM)
>> do_stuff(sock)

Traceback (most recent call last):
File "<stdin>", line 1, in ?
File "<stdin>", line 2, in do_stuff
NameError: global name 'sock_list' is not defined
>>


If you add "sock_list = []" just before the def of do_stuff, the code
will work, and your sockets won't get garbage collected.

<mike
--
Mike Meyer <(E-Mail Removed)> http://www.mired.org/home/mwm/
Independent WWW/Perforce/FreeBSD/Unix consultant, email for more information.
 
Reply With Quote
 
David Bolen
Guest
Posts: n/a
 
      12-17-2004
Scott Robinson <(E-Mail Removed)> writes:

> I have been having trouble with the garbage collector and sockets.


Are you actually getting errors or is this just theoretical?

> Unfortunately, google keeps telling me that the problem is the garbage
> collector ignoring dead (closed?) sockets instead of removing live
> ones. My problem is
>
>
> x.sock=socket.socket(socket.AF_INET,socket.SOCK_ST REAM)
> do_stuff(x.sock)
>
>
> def do_stuff(sock):
> sock_list.append(sock)
>
> once do_stuff finishes, x.sock disappears, and I can only believe it
> is being garbage collected.


Can you clarify this? What do you mean by "x.sock" disappears? Are
you getting a NameError later when trying to use "x.sock"?

x.sock is just a name binding, so it is not really involved in garbage
collection (GC applies to the objects to which names are bound).

In this case, you need to include much more in the way of code (a
fully running, but smallest possible, snippet of code would be best),
since the above can be interpreted many ways. At the least, it's very
important to include information about the namespace within which
those two code snippets run if anyone is likely to be able to give you
a good answer. Also, being very precise about the error condition you
are experiencing (including actual error messages, tracebacks, etc...)
is crucial.

Is 'x' referencing a local or global object, and does that socket code
occur within a method, a function, or what? Also, in do_stuff, where
is sock_list defined? Is it local, global?

If, as written, sock_list is a local name to do_stuff, then that
binding is going to disappear when do_stuff completes, thus, the list
to which it is bound will be destroyed, including all references to
objects that the list may contain. So at that point, when you return
from do_stuff, the only reference to the socket object will be in
x.sock. But if 'x' is also local to the function/method where the
call to do_stuff is, the name binding will be removed when the
function/method returns, at which point there will be no references to
the socket object, and yes, it will be destroyed.

But if sock_list is global, and continues to exist when do_stuff
completes, then the reference it contains to the socket will keep the
socket object alive even if you remove the x.sock binding.

-- David

 
Reply With Quote
 
Scott Robinson
Guest
Posts: n/a
 
      12-18-2004
On 16 Dec 2004 20:38:29 -0500, David Bolen <(E-Mail Removed)> wrote:

>Scott Robinson <(E-Mail Removed)> writes:
>
>> I have been having trouble with the garbage collector and sockets.

>
>Are you actually getting errors or is this just theoretical?
>
>> Unfortunately, google keeps telling me that the problem is the garbage
>> collector ignoring dead (closed?) sockets instead of removing live
>> ones. My problem is
>>
>>
>> x.sock=socket.socket(socket.AF_INET,socket.SOCK_ST REAM)
>> do_stuff(x.sock)
>>
>>
>> def do_stuff(sock):
>> sock_list.append(sock)
>>
>> once do_stuff finishes, x.sock disappears, and I can only believe it
>> is being garbage collected.

>
>Can you clarify this? What do you mean by "x.sock" disappears? Are
>you getting a NameError later when trying to use "x.sock"?
>
>x.sock is just a name binding, so it is not really involved in garbage
>collection (GC applies to the objects to which names are bound).
>
>In this case, you need to include much more in the way of code (a
>fully running, but smallest possible, snippet of code would be best),
>since the above can be interpreted many ways. At the least, it's very
>important to include information about the namespace within which
>those two code snippets run if anyone is likely to be able to give you
>a good answer. Also, being very precise about the error condition you
>are experiencing (including actual error messages, tracebacks, etc...)
>is crucial.
>
>Is 'x' referencing a local or global object, and does that socket code
>occur within a method, a function, or what? Also, in do_stuff, where
>is sock_list defined? Is it local, global?
>
>If, as written, sock_list is a local name to do_stuff, then that
>binding is going to disappear when do_stuff completes, thus, the list
>to which it is bound will be destroyed, including all references to
>objects that the list may contain. So at that point, when you return
>from do_stuff, the only reference to the socket object will be in
>x.sock. But if 'x' is also local to the function/method where the
>call to do_stuff is, the name binding will be removed when the
>function/method returns, at which point there will be no references to
>the socket object, and yes, it will be destroyed.
>
>But if sock_list is global, and continues to exist when do_stuff
>completes, then the reference it contains to the socket will keep the
>socket object alive even if you remove the x.sock binding.
>
>-- David

(so much for Python being executable psuedocode).

It looks like I was completely wrong. The problem I ran into was not
checking into baseHTTPserver and looking for self.close_connection=1.

I am pretty sure this happened to me before, and I rewrote the code to
avoid sockets being closed after being referenced to a live object,
but I can't reproduce it.

Anyway, here is my debugged test rig. It works. Ignore it.


Scott
[test program follows]
import socket, thread, time

class sock_holder:
pass

HOST = '127.0.0.1'
PORT = 2004

def echo_server(port):
print "started at",port
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.bind((HOST, port))
s.listen(1)
conn, addr = s.accept()
print 'Connected by', addr
while 1:
data = conn.recv(1024)
if not data: break
conn.send(data)
conn.close()

def wait_and_speak(a):
time.sleep(5)
try:
a.sock.send("this is message 2")
except:
print "error on message 2"
try:
data = a.sock.recv(1024)
print data
except:
print "error recieving message 2"
a.sock.close()

def other_thread(a):
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
thread.start_new(echo_server,(PORT,))
time.sleep(1)
s.connect((HOST,PORT ))
s.send('Hello, port 1')
data = s.recv(1024)
print data
a.sock=s

thread.start_new(wait_and_speak,(a,))

a=sock_holder()
thread.start_new(other_thread,(a,))
time.sleep(10)


 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
C extension: VALUEs being collected too early eden li Ruby 2 06-14-2009 05:30 PM
When is a thread garbage collected? Kent Johnson Python 1 03-16-2005 10:28 PM
When are static members garbage collected? =?Utf-8?B?Sg==?= ASP .Net 1 02-25-2004 11:58 PM
Can objects with active threads be garbage collected? Mark McKay Java 5 10-03-2003 03:33 AM
JInternalFrame not garbage collected Cheng Thao Java 0 08-06-2003 08:21 PM



Advertisments