Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Python > socket problem

Reply
Thread Tools

socket problem

 
 
Gordon Wetzstein
Guest
Posts: n/a
 
      07-11-2003
Hello everyone,

I have a problem with python sockets.
I broadcast a lot of pickled objects with socket. sendto(...), that works.
Ireceive them on the other site with socket.recvfrom(16384)
The pickled objects are smaller (like 10000 bytes) than the max bytes.

The problem appears if I send too many pickled objects very quickly one
after another,
then I only receive a part of the sent objects.
I print them before I send them, they are there but they never reach the
destination.
When I do a time.sleep(0.1) (but not lesser than 0.1) after socket.send()
all is good but it's too slow!

Does anyone know what the problem is? And maybe a solution!

thanks gordon

--
+++ GMX - Mail, Messaging & more http://www.gmx.net +++

Jetzt ein- oder umsteigen und USB-Speicheruhr als Prämie sichern!


 
Reply With Quote
 
 
 
 
Alan Kennedy
Guest
Posts: n/a
 
      07-12-2003
Gordon Wetzstein wrote:

> I have a problem with python sockets.
> I broadcast a lot of pickled objects with socket. sendto(...), that
> works.
> Ireceive them on the other site with socket.recvfrom(16384)
> The pickled objects are smaller (like 10000 bytes) than the max bytes.
>
> The problem appears if I send too many pickled objects very quickly one
> after another,
> then I only receive a part of the sent objects.


As Jp Calderone pointed out, UDP is an unreliable protocol. Therefore,
it *will* drop packets occasionally. Worse, it doesn't guarantee
ordering of delivery of packets, meaning that your pickles could
arrive all jumbled up and unusable.

You have three main choices to get around this problem.

1. Use TCP, which guarantees delivery to the destination address once
and only once, guarantees that packets will be received in the order
in which they were sent. However, you can't broadcast with TCP: it's a
point to point protocol.

2. Implement your own guaranteed delivery protocol over UDP. This is
usually done by assigning packets a sequence number as they are
transmitted from the publisher. All consumers then keep track of the
sequence numbers they receive, and re-request any packets they missed.
Implementing your own such protocol can get quite tricky, depending on
how complex your requirements.

3. Use a pre-made, highly efficient python library which is custom
designed for the job: spread. Spread is designed to solve all of the
problems of reliable broadcasting over UDP, and gives you a wide range
of options for how to balance efficiency versus reliability. And more
importantly, it's industrial-strength robust, stable and platform
independent. Also, it's configurable to work with a wide range of
network topologies. More info from

http://www.python.org/other/spread/

If I were you, I'd seriously consider spending an hour getting up and
running with Spread: could be the most productive hour you'll spend in
this area.

HTH,

--
alan kennedy
-----------------------------------------------------
check http headers here: http://xhaus.com/headers
email alan: http://xhaus.com/mailto/alan
 
Reply With Quote
 
 
 
 
Gordon Wetzstein
Guest
Posts: n/a
 
      07-12-2003

> > I have a problem with python sockets.
> > I broadcast a lot of pickled objects with socket. sendto(...), that
> > works.
> > Ireceive them on the other site with socket.recvfrom(16384)
> > The pickled objects are smaller (like 10000 bytes) than the max bytes.
> >
> > The problem appears if I send too many pickled objects very quickly one
> > after another,
> > then I only receive a part of the sent objects.

>
> As Jp Calderone pointed out, UDP is an unreliable protocol. Therefore,
> it *will* drop packets occasionally. Worse, it doesn't guarantee
> ordering of delivery of packets, meaning that your pickles could
> arrive all jumbled up and unusable.
>
> You have three main choices to get around this problem.
>
> 1. Use TCP, which guarantees delivery to the destination address once
> and only once, guarantees that packets will be received in the order
> in which they were sent. However, you can't broadcast with TCP: it's a
> point to point protocol.
>
> 2. Implement your own guaranteed delivery protocol over UDP. This is
> usually done by assigning packets a sequence number as they are
> transmitted from the publisher. All consumers then keep track of the
> sequence numbers they receive, and re-request any packets they missed.
> Implementing your own such protocol can get quite tricky, depending on
> how complex your requirements.
>
> 3. Use a pre-made, highly efficient python library which is custom
> designed for the job: spread. Spread is designed to solve all of the
> problems of reliable broadcasting over UDP, and gives you a wide range
> of options for how to balance efficiency versus reliability. And more
> importantly, it's industrial-strength robust, stable and platform
> independent. Also, it's configurable to work with a wide range of
> network topologies. More info from
>
> http://www.python.org/other/spread/
>
> If I were you, I'd seriously consider spending an hour getting up and
> running with Spread: could be the most productive hour you'll spend in
> this area.
>
> HTH,
>
> --
> alan kennedy


thanks, i will try it.
gordon

--
+++ GMX - Mail, Messaging & more http://www.gmx.net +++

Jetzt ein- oder umsteigen und USB-Speicheruhr als Prämie sichern!


 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Re: socket.unbind or socket.unlisten? - socket.error: (48, 'Addressalready in use') Steve Holden Python 1 02-03-2009 06:20 AM
Re: socket.unbind or socket.unlisten? - socket.error: (48, 'Addressalready in use') Steve Holden Python 0 02-01-2009 12:45 PM
Re: socket.unbind or socket.unlisten? - socket.error: (48, 'Addressalready in use') Laszlo Nagy Python 0 02-01-2009 07:37 AM
socket.unbind or socket.unlisten? - socket.error: (48, 'Addressalready in use') Laszlo Nagy Python 1 01-27-2009 05:05 PM
Re: socket.unbind or socket.unlisten? - socket.error: (48,'Address already in use') Jean-Paul Calderone Python 0 01-27-2009 01:41 PM



Advertisments