Socket error: 10053 software caused connection abort

Hello there

I'm getting an error on Windows XP when I send big data chunks over a socket
connection. The exception I get is: "socket.error: (10053, 'Software caused
connection abort')"
This happens in the socket.recv function.

What I found out is that such errors can occur if there's not enough buffer
for receiving data in the underlying hardware.
On Linux it works fine. I guess, that the socket-code on windows could be
buggy and tries to read from a buffer whithout checking the length.

The server.py script does not alway receive all the data that was sent by
the client.

I'd be glad if someone could test theses scripts to see if this error also
occurs on other coputers and systems.

Or is there anything wrong with the code?


Regards

James

================================================== ====
server.py
================================================== ====
import SocketServer

class MyHandler(SocketServer.StreamRequestHandler):
def get_all(self):
# Get data as long as there's something on the line
data = ''
block_size = 1024
while(1):
chunk = self.connection.recv(block_size)
if not chunk:
break

data += chunk

if (len(chunk) < block_size):
# We got all the data
break

return data

def handle(self):
data = self.get_all()

data_len = len(data)

if (data_len == 0):
return

print "Received: %d" % data_len

total_sent = 0
while(total_sent < data_len):
sent = self.connection.send(data)
if not sent:
break

data = data[sent:]
total_sent += sent

try:
s = SocketServer.TCPServer(('', 12345), MyHandler)
s.serve_forever()
except Error, msg:
print msg


================================================== ====
client.py
================================================== ====
import sys
import socket
from time import sleep

port = 12345
host = 'localhost'
loops = 100
data_len = 12000
send_data = 'x' * data_len
to_send = len(send_data)

for _ in range(loops):
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect(("localhost", port))
total_sent = 0
while(total_sent < to_send):
sent = s.send(send_data)
if not sent:
break;
print "Sent: %d" % sent
total_sent += sent

print "Total-sent: %d" % total_sent

# Get data as long as there's something on the line
data = ''

block_size = 1024

while(1):
got = s.recv(block_size)

if not got:
print "Connection broken"
break

data += got

if (len(got) < block_size):
# We got all the data
break

data_len = len(data)
s.close()
print 'Received bytes: %d' % len(data)

Jean-Pierre Bergamin

Jean-Pierre Bergamin wrote:

> I'd be glad if someone could test theses scripts to see if this error
> also occurs on other coputers and systems.
>
> Or is there anything wrong with the code?

Could anyone reproduce this error?


Regards

James

Jean-Pierre Bergamin

Jean-Pierre Bergamin wrote:

> Jean-Pierre Bergamin wrote:
>
>>I'd be glad if someone could test theses scripts to see if this error
>>also occurs on other coputers and systems.
>>
>>Or is there anything wrong with the code?
>
>
> Could anyone reproduce this error?

Yes I could observe this error some time ago.
It is not related to Python but it is rather a Windows Sockets error.
Searching the google I have found this description of the error message:
http://support.ipswitch.com/kb/WSK-19980702-EM02.htm
Try to use a sniffer like Ethereal and see what packets
are sent and received from the network.

--
Stach Tlen: stachobywatelpl, GG: 1811474
Jabber: stach at jabber atman pl

Krzysztof Stachlewski

Jean-Pierre Bergamin wrote:
> Hello there
>
> I'm getting an error on Windows XP when I send big data chunks over a socket
> connection. The exception I get is: "socket.error: (10053, 'Software caused
> connection abort')"
> This happens in the socket.recv function.

I have the same error here, and it is NOT because of a bug in Windows' TCP
stack. (even though the problem does not occur on Linux).

The problem is in your code.

You are receiving chunks of data like this:

> # Get data as long as there's something on the line
> data = ''
> block_size = 1024
> while(1):
> chunk = self.connection.recv(block_size)
> if not chunk:
> break
>
> data += chunk
>
> if (len(chunk) < block_size):
> # We got all the data
> break
>
> return data

But this code is flawed: recv() CAN return less than block_size,
*even* if there is more data to be read on the socket.

So, you should not check if the length of the received chunk
is less than the length argument passed to recv().

Instead, you should count the total number of bytes you
received and check that with the expected total amount.

Also, string appending is slow. Better append to a list and when
the loop is complete, return ''.join(alist).

I fixed your client and server receive loops in the mentioned way,
and they work fine even on windows

However there is one more thing, when your block_size (the argument
to recv) is "too large" there are certain OSes that show problems
(VMS, Windows XP). MemoryErrors and such. I have been using:

chunk=sock.recv(min(60000,block_size))

quite successfully.

--Irmen de Jong

Irmen de Jong

Hi Irmen

>> I'm getting an error on Windows XP when I send big data chunks over
>> a socket connection. The exception I get is: "socket.error: (10053,
>> 'Software caused connection abort')"
>> This happens in the socket.recv function.
>
> I have the same error here, and it is NOT because of a bug in
> Windows' TCP stack. (even though the problem does not occur on Linux).
>
> The problem is in your code.
>
> You are receiving chunks of data like this:

> [....]

> But this code is flawed: recv() CAN return less than block_size,
> *even* if there is more data to be read on the socket.
>
> So, you should not check if the length of the received chunk
> is less than the length argument passed to recv().
>
> Instead, you should count the total number of bytes you
> received and check that with the expected total amount.

But how can I know the amout of data I will receive? There's no way to do
that, since it's totally random how large the sent data will be.

Could you show me how you did it?


Thanks

James

Jean-Pierre Bergamin

Jean-Pierre Bergamin wrote:
> But how can I know the amout of data I will receive? There's no way to do
> that, since it's totally random how large the sent data will be.

Ah! You have to adapt your program. The usual pattern here is to encode
the length of the data that will be sent, send that size first (in a
fixed amount of bytes for instance) and after that, send the data.
The receiving side first grabs the encoded integer from the socket,
and then starts to recv() the given amount of bytes.

You can use the struct module to encode the length of the data stream
in a fixed amount of bytes (say, 4).
Send those 4 bytes first.
Then send your message.
The recipient first reads exactly 4 bytes, uses the struct module to
decode that back into an integer, then reads the specified length.

If you only use the socket connection for a single request/response,
you can also shutdown() the socket after sending is complete. The
recv() will notice this.

HTH,
--Irmen de Jong.

PS:
I once developed a low level socket protocol, that is used by Pyro.
It essentially does the same as described above (but more complex).
But since I now have Pyro, I usually don't have to think and bother
about the low level stuff anymore

Irmen de Jong

At some point, "Jean-Pierre Bergamin" <> wrote:

> Hi Irmen
>
>> But this code is flawed: recv() CAN return less than block_size,
>> *even* if there is more data to be read on the socket.
>>
>> So, you should not check if the length of the received chunk
>> is less than the length argument passed to recv().
>>
>> Instead, you should count the total number of bytes you
>> received and check that with the expected total amount.
>
> But how can I know the amout of data I will receive? There's no way to do
> that, since it's totally random how large the sent data will be.

You don't know, unless you send the size (like HTTP with the
Content-length: header), or have some way of knowing when
you're done (like SMTP or IMAP, which use CR-LF to signify the end of
the line).

--
|>|\/|<
/--------------------------------------------------------------------------\
|David M. Cooke
|cookedm(at)physics(dot)mcmaster(dot)ca

David M. Cooke