«

I am trying to unpack values from sensor data I am retrieving through a serial cable, but I get errors while using struct.unpack, how can I use struct.unpack to unload the data in a readable format?

I checked the python documentation for struct and I can seen to find any argument for this.

I have data = struct.unpack('char',data) but I still get errors

On 25/08/2012 19:34, 9bizy wrote:
> I am trying to unpack values from sensor data I am retrieving through a serial cable, but I get errors while using struct.unpack, how can I use struct.unpack to unload the data in a readable format?
>
> I checked the python documentation for struct and I can seen to find any argument for this.
>
> I have data = struct.unpack('char',data) but I still get errors
>

We have two options here. Either

a) People reading your original request go on a mind reading course or
similar in an attempt to find out what the errors are, though I'm
confused as to how you get errors from one line of code.

Or

b) Provide the smallest sample of code that allows the problem to be
reproduced together with the complete traceback so we can see exactly
what happened.

--
Cheers.

Mark Lawrence.

On 25/08/2012 19:34, 9bizy wrote:
> I am trying to unpack values from sensor data I am retrieving through
> a serial cable, but I get errors while using struct.unpack, how can I
> use struct.unpack to unload the data in a readable format?
>
> I checked the python documentation for struct and I can seen to find
> any argument for this.
>
> I have data = struct.unpack('char',data) but I still get errors
>
The format strings are described here for Python 3:

http://docs.python.org/3.2/library/struct.html

and here for Python 2:

http://docs.python.org/2.7/library/struct.html

On Sat, 25 Aug 2012 11:34:39 -0700 (PDT), 9bizy
<a.m.akingbulu-> declaimed the following in
gmane.comp.python.general:

> I am trying to unpack values from sensor data I am retrieving through a serial cable, but I get errors while using struct.unpack, how can I use struct.unpack to unload the data in a readable format?
>
> I checked the python documentation for struct and I can seen to find any argument for this.
>
Pardon "... I can seen to find..."?

> I have data = struct.unpack('char',data) but I still get errors

No surprise... You've asked for a single-character, a short integer,
and two unknown field types.

What about

"""
s char[] string
"""

and

"""
For the "s" format character, the count is interpreted as the size of
the string, not a repeat count like for the other format characters; for
example, '10s' means a single 10-byte string, while '10c' means 10
characters. For packing, the string is truncated or padded with null
bytes as appropriate to make it fit. For unpacking, the resulting string
always has exactly the specified number of bytes. As a special case,
'0s' means a single, empty string (while '0c' means 0 characters).
"""

(both from the 2.5 help file).

The struct module relies upon the user knowing the format of the data.
If your problem is that you have some null-terminated string data in a
variable width field, you will have to locate the position of the null
FIRST, and specify the appropriate "count" for the s format.
--
Wulfraed Dennis Lee Bieber AF6VN
HTTP://wlfraed.home.netcom.com/

On 26.08.2012 01:31, Dennis Lee Bieber wrote:
> The struct module relies upon the user knowing the format of the data.
> If your problem is that you have some null-terminated string data in a
> variable width field, you will have to locate the position of the null
> FIRST, and specify the appropriate "count" for the s format.

This gave me the idea of an Enhancement of the Struct class with an
additional format character (perhaps 'n') which corresponds to a
null-terminated string:

-----
# -*- coding: utf-8 -*-

import struct

class Nstr(object):
def __init__(self, ncount):
self.ncount = ncount

def unpack(self, s):
s = s.split('\0')
return s[:self.ncount], '\0'.join(s[self.ncount:])

def pack(self, *s):
if len(s)!=self.ncount:
raise ValueError
for st in s:
if '\0' in st:
raise ValueError
return '\0'.join(s)+'\0'

def __repr__(self):
return 'Nstr('+repr(self.ncount)+')'

class NStruct(object):
def __init__(self, format):
self.format = format
if format[0] in '!=<>@':
self.endianness = format[0]
format = format[1:]
else:
self.endianness = ''
self.chunks = []
while len(format)>0:
j = format.find('n')
if j > -1:
k = j-1
while format[k].isdigit():
k-=1
chunkformat, ncount, format = format[:k+1],\
format[k+1:j], format[j+1:]
ncount = 1 if len(ncount)==0 else int(ncount)
else:
chunkformat, ncount, format = format, '', ''
ncount = 0
stru = struct.Struct(self.endianness+chunkformat)
l = len(stru.unpack("0"*stru.size))
self.chunks.append((stru, l))
if ncount > 0:
self.chunks.append((Nstr(ncount), ncount))

def unpack(self, data):
res = []
for sth, n in self.chunks:
if isinstance(sth, struct.Struct):
chunk, data = data[:sth.size], data[sth.size:]
res.extend(sth.unpack(chunk))
elif isinstance(sth, Nstr):
chunk, data = sth.unpack(data)
res.extend(chunk)
return res

def pack(self, *data):
res = []
for sth, n in self.chunks:
chunk, data = data[:n], data[n:]
res.append(sth.pack(*chunk))
return ''.join(res)

def __repr__(self):
return 'NStruct('+repr(self.format)+')'

if __name__=="__main__":
a = NStruct('h b 2n 2h')
print repr(a)
d = 'asdblah blah\0haha\0asdf'
r = a.unpack(d)
assert r == [29537, 100, 'blah blah', 'haha', 29537, 26212]
print repr(d), repr(r)
dd = a.pack(*r)
print r, repr(dd)
assert dd == d

-----

beware of bugs in the above code, i haven't testet it much yet.

Alex

This is what I have to reproduce the challenge I am having below:


import csv
import struct


data = []

for Node in csv.reader(file('s_data.xls')):
data.append(list((file('s_data.xls'))))


data = struct.unpack('!B4HH', data)
print "s_data.csv: ", data

I tries so many format for the struct.unpack but I got this errors:

Traceback (most recent call last):

data = struct.unpack('!B4HH', data)
struct.error: unpack requires a string argument of length 11

On Saturday, 25 August 2012 19:34:39 UTC+1, 9bizy wrote:
> I am trying to unpack values from sensor data I am retrieving through a serial cable, but I get errors while using struct.unpack, how can I use struct.unpack to unload the data in a readable format?
>
>
>
> I checked the python documentation for struct and I can seen to find any argument for this.
>
>
>
> I have data = struct.unpack('char',data) but I still get errors

On Saturday, 25 August 2012 20:16:54 UTC+1, MRAB wrote:
> On 25/08/2012 19:34, 9bizy wrote:
>
> > I am trying to unpack values from sensor data I am retrieving through
>
> > a serial cable, but I get errors while using struct.unpack, how can I
>
> > use struct.unpack to unload the data in a readable format?
>
> >
>
> > I checked the python documentation for struct and I can seen to find
>
> > any argument for this.
>
> >
>
> > I have data = struct.unpack('char',data) but I still get errors
>
> >
>
> The format strings are described here for Python 3:
>
>
>
> http://docs.python.org/3.2/library/struct.html
>
>
>
> and here for Python 2:
>
>
>
> http://docs.python.org/2.7/library/struct.html

I used this documents but they do not explain or provide an example on how to use struct.unpack for sensor data from an external source or even data from a excel sheet.

On Saturday, 25 August 2012 20:16:54 UTC+1, MRAB wrote:
> On 25/08/2012 19:34, 9bizy wrote:
>
> > I am trying to unpack values from sensor data I am retrieving through
>
> > a serial cable, but I get errors while using struct.unpack, how can I
>
> > use struct.unpack to unload the data in a readable format?
>
> >
>
> > I checked the python documentation for struct and I can seen to find
>
> > any argument for this.
>
> >
>
> > I have data = struct.unpack('char',data) but I still get errors
>
> >
>
> The format strings are described here for Python 3:
>
>
>
> http://docs.python.org/3.2/library/struct.html
>
>
>
> and here for Python 2:
>
>
>
> http://docs.python.org/2.7/library/struct.html

I used this documents but they do not explain or provide an example on how to use struct.unpack for sensor data from an external source or even data from a excel sheet.

On 28/08/2012 23:34, 9bizy wrote:
> This is what I have to reproduce the challenge I am having below:
>
>
> import csv
> import struct
>
>
> data = []
>
> for Node in csv.reader(file('s_data.xls')):

That tries to read the file as CSV, but, judging from the extension,
it's in Excel's format. You don't even use what is read, i.e. Node.

> data.append(list((file('s_data.xls'))))
>
That opens the file again and 'list' causes it to read the file as
though it were a series of lines in a text file, which, as I've said,
it looks like it isn't. The list of 'lines' is appended to the list
'data', so that's a list of lists.
>
> data = struct.unpack('!B4HH', data)
> print "s_data.csv: ", data
>
> I tries so many format for the struct.unpack but I got this errors:
>
> Traceback (most recent call last):
>
> data = struct.unpack('!B4HH', data)
> struct.error: unpack requires a string argument of length 11
>
[snip]
It's complaining because it's expecting a string argument but you're
giving it a list instead.

On 28/08/2012 23:35, 9bizy wrote:
> On Saturday, 25 August 2012 20:16:54 UTC+1, MRAB wrote:
>> On 25/08/2012 19:34, 9bizy wrote:
>>
>> > I am trying to unpack values from sensor data I am retrieving through
>>
>> > a serial cable, but I get errors while using struct.unpack, how can I
>>
>> > use struct.unpack to unload the data in a readable format?
>>
>> >
>>
>> > I checked the python documentation for struct and I can seen to find
>>
>> > any argument for this.
>>
>> >
>>
>> > I have data = struct.unpack('char',data) but I still get errors
>>
>> >
>>
>> The format strings are described here for Python 3:
>>
>>
>>
>> http://docs.python.org/3.2/library/struct.html
>>
>>
>>
>> and here for Python 2:
>>
>>
>>
>> http://docs.python.org/2.7/library/struct.html
>
> I used this documents but they do not explain or provide an example on how to use struct.unpack for sensor data from an external source or even data from a excel sheet.
>
If you want to read from an Excel file you should be using the 'xlrd'
module. You can find it here: http://www.python-excel.org/