«

I am attempting to send a jpeg image file created on an embedded
device over a wifi socket to a Python client running on a Linux pc
(Ubuntu). All works well, except I don't know, on the pc client side,
what the file size is? The following is a snippet:

[code]
f = open("frame.jpg",mode = 'wb')
while True:
data = self.s.recv(MAXPACKETLEN)
if len(data) == 0:
break
recvd += len(data)
f.write(data)
f.close()
[end]

It appears to be locking up in 'data=self.s.recv(MAXPACKETLEN)' on
the final packet, which will always be less than MAXPACKETLEN.

I guess my question is, how do I detect end of data on the client side?

twgray wrote:
> I am attempting to send a jpeg image file created on an embedded
> device over a wifi socket to a Python client running on a Linux pc
> (Ubuntu). All works well, except I don't know, on the pc client side,
> what the file size is?

You don't. Sockets are just endless streams of bytes. You will have to design some form
of 'wire protocol' that includes the length of the message that is to be read.
For instance a minimalistic protocol could be the following:
Send 4 bytes that contain the length (an int) then the data itself. The client reads 4
bytes, decodes it into the integer that tells it the length, and then reads the correct
amount of bytes from the socket.


--irmen

On Sat, Jul 18, 2009 at 4:43 PM, Irmen de Jong<> wrote:
> twgray wrote:
>>
>> I am attempting to send a jpeg image file created on an embedded
>> device over a wifi socket to a Python client running on a Linux pc
>> (Ubuntu). *All works well, except I don't know, on the pc client side,
>> what the file size is?
>
> You don't. Sockets are just endless streams of bytes. You will have to
> design some form of 'wire protocol' that includes the length of the message
> that is to be read.
> For instance a minimalistic protocol could be the following:
> Send 4 bytes that contain the length (an int) then the data itself. The
> client reads 4 bytes, decodes it into the integer that tells it the length,
> and then reads the correct amount of bytes from the socket.

Exactly, sending the length first is the only way to know ahead of
time. Alternatively, if you know what the end of the data looks like,
you can look for that 'flag' as well, and stop trying to recv() after
that.

Some things that may be useful, though, are socket.settimeout() and
socket.setblocking(). More information is availible in the docs:
http://docs.python.org/library/socket.html.

You need to be careful with this, though, since network latency may
cause problems. Using these methods will keep your program from
sitting in recv() forever, though.

\t
--
http://tycho.ws

On Jul 18, 4:43*pm, Irmen de Jong <irmen.NOS...@xs4all.nl> wrote:
> twgray wrote:
> > I am attempting to send a jpeg image file created on an embedded
> > device over a wifi socket to a Python client running on a Linux pc
> > (Ubuntu). *All works well, except I don't know, on the pc client side,
> > what the file size is? *
>
> You don't. Sockets are just endless streams of bytes. You will have to design some form
> of 'wire protocol' that includes the length of the message that is to be read.
> For instance a minimalistic protocol could be the following:
> Send 4 bytes that contain the length (an int) then the data itself. The client reads 4
> bytes, decodes it into the integer that tells it the length, and then reads the correct
> amount of bytes from the socket.
>
> --irmen

Thanks for the reply. But, now I have a newbie Python question. If I
send a 4 byte address from the embedded device, how do I convert that,
in Python, to a 4 byte, or long, number?

twgray wrote:
> On Jul 18, 4:43 pm, Irmen de Jong <irmen.NOS...@xs4all.nl> wrote:
>> twgray wrote:
>>> I am attempting to send a jpeg image file created on an embedded
>>> device over a wifi socket to a Python client running on a Linux pc
>>> (Ubuntu). All works well, except I don't know, on the pc client side,
>>> what the file size is?
>> You don't. Sockets are just endless streams of bytes. You will have to design some form
>> of 'wire protocol' that includes the length of the message that is to be read.
>> For instance a minimalistic protocol could be the following:
>> Send 4 bytes that contain the length (an int) then the data itself. The client reads 4
>> bytes, decodes it into the integer that tells it the length, and then reads the correct
>> amount of bytes from the socket.
>>
> Thanks for the reply. But, now I have a newbie Python question. If I
> send a 4 byte address from the embedded device, how do I convert that,
> in Python, to a 4 byte, or long, number?

If you send the length as 4 bytes then you'll have to decide whether
it's big-endian or little-endian. An alternative is to send the length
as characters, terminated by, say, '\n' or chr(0).

On Jul 19, 8:04*am, twgray <twgray2...@gmail.com> wrote:

> send a 4 byte address from the embedded device, how do I convert that,
> in Python, to a 4 byte, or long, number?

struct.unpack() is your friend. Presuming the embedded device is
little-endian, you do:

the_int = struct.unpack('<I', four_bytes)[0]

See http://docs.python.org/library/struct.html

On Sat, 18 Jul 2009 14:33:48 -0700, twgray wrote:

> It appears to be locking up in 'data=self.s.recv(MAXPACKETLEN)' on
> the final packet, which will always be less than MAXPACKETLEN.
>
> I guess my question is, how do I detect end of data on the client side?

recv() should return zero when the sender closes its end of the connection.

Is the sender actually closing its end? If you are unsure, use a packet
sniffer such as tcpdump to look for a packet with the FIN flag.

If you need to keep the connection open for further transfers, you need to
incorporate some mechanism for identifying the end of the data into the
protocol. As others have suggested, prefixing the data by its length is
one option. Another is to use an end-of-data marker, but then you need a
mechanism to "escape" the marker if it occurs in the data. A length prefix
is probably simpler to implement, but has the disadvantage that you can't
start sending the data until you know how long it is going to be.

On Jul 19, 7:43*am, Irmen de Jong <irmen.NOS...@xs4all.nl> wrote:
> twgray wrote:
> > I am attempting to send a jpeg image file created on an embedded
> > device over a wifi socket to a Python client running on a Linux pc
> > (Ubuntu). *All works well, except I don't know, on the pc client side,
> > what the file size is? *
>
> You don't. Sockets are just endless streams of bytes. You will have to design some form
> of 'wire protocol' that includes the length of the message that is to be read.

Apologies in advance for my ignorance -- the last time I dipped my toe
in that kind of water, protocols like zmodem and Kermit were all the
rage -- but I would have thought there would have been an off-the-
shelf library for peer-to-peer file transfer over a socket
interface ... not so?

Nobody wrote:
> On Sat, 18 Jul 2009 14:33:48 -0700, twgray wrote:
>
>> It appears to be locking up in 'data=self.s.recv(MAXPACKETLEN)' on
>> the final packet, which will always be less than MAXPACKETLEN.
>>
>> I guess my question is, how do I detect end of data on the client side?
>
> recv() should return zero when the sender closes its end of the connection.
>
> Is the sender actually closing its end? If you are unsure, use a packet
> sniffer such as tcpdump to look for a packet with the FIN flag.
>
> If you need to keep the connection open for further transfers, you need to
> incorporate some mechanism for identifying the end of the data into the
> protocol. As others have suggested, prefixing the data by its length is
> one option. Another is to use an end-of-data marker, but then you need a
> mechanism to "escape" the marker if it occurs in the data. A length prefix
> is probably simpler to implement, but has the disadvantage that you can't
> start sending the data until you know how long it is going to be.
>
You could send it in chunks, ending with a chunk length of zero.

On Jul 18, 7:33*pm, MRAB <pyt...@mrabarnett.plus.com> wrote:
> Nobody wrote:
> > On Sat, 18 Jul 2009 14:33:48 -0700, twgray wrote:
>
> >> It appears to be locking up in *'data=self.s.recv(MAXPACKETLEN)' on
> >> the final packet, which will always be less than MAXPACKETLEN.
>
> >> I guess my question is, how do I detect end of data on the client side?
>
> > recv() should return zero when the sender closes its end of the connection.
>
> > Is the sender actually closing its end? If you are unsure, use a packet
> > sniffer such as tcpdump to look for a packet with the FIN flag.
>
> > If you need to keep the connection open for further transfers, you need to
> > incorporate some mechanism for identifying the end of the data into the
> > protocol. As others have suggested, prefixing the data by its length is
> > one option. Another is to use an end-of-data marker, but then you need a
> > mechanism to "escape" the marker if it occurs in the data. A length prefix
> > is probably simpler to implement, but has the disadvantage that you can't
> > start sending the data until you know how long it is going to be.
>
> You could send it in chunks, ending with a chunk length of zero.

Thanks for the help!