Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > C Programming > packed structs

Reply
Thread Tools

packed structs

 
 
Nick Keighley
Guest
Posts: n/a
 
      09-23-2012
On Sep 22, 11:33*pm, Ian Collins <(E-Mail Removed)> wrote:
> On 09/22/12 08:53 PM, JohnF wrote:
>
>
>
>
>
>
>
>
>
> > Nick Keighley<(E-Mail Removed)> *wrote:
> >> Eric Sosman<(E-Mail Removed)> *wrote:

>
> >> serializing structs

>
> >>> One way is the function-per-struct-type approach, and
> >>> although it may be "b..o..r..i..n..g" it has advantages
> >>> that should not be dismissed lightly. ?Consider that there
> >>> are (most likely) only a handful of structs and hence only
> >>> a handful of functions; writing them won't take enough
> >>> time to b..o..r..e anyone except an ADHD sufferer.

>
> >> I've dealt with cases where there were considerably more than a
> >> "handful"- communication protocol.

>
> >> Since I have a low b..o..r..e..d..o..m threshold I resorted to code
> >> generation. The protocol was defined by tables in a PDF document (ug).
> >> Copy-paste turned them into text and perl turned them into something
> >> easily processed. Most of the code was generated by people who didn't
> >> appear to mind writing tons of tedious boring repetitive code. There
> >> was a bit-banging library that did most of the heavy lifting.

>
> > What do you think of my my smemf()-type of solution in other post?
> > Not necessarily that particular solution, but the point being that
> > this has got to be a pretty frequently occurring problem, and the
> > only available solutions, like yours above, seem to be pretty
> > awfully ugly. But it ain't rocket science -- there ought to be
> > a way to deal with these situations elegantly. smemf() solves it
> > with>>zero<< *structs.

>
> having worked on numerous on the wire protocols, I've encountered this
> problem many times. *The most practical solution for all but the most
> trivial cases is to code generate the formatting code. *The source for
> the code generator can either be C (or C++ if inheritance helps) code or
> some other easy to parse format. *This is one case where I prefer XML
> (often in the form of an OpenOffice document) as the "other easy to
> parse format".


there's ASN.1 too...


 
Reply With Quote
 
 
 
 
Stephen Sprunk
Guest
Posts: n/a
 
      09-23-2012
On 22-Sep-12 01:37, JohnF wrote:
> I'd actually finished (what I think is) a more elegant solution,
> that I called smemf() that's like memcpy() but under format
> control, including additonal format specifiers for hex, for bits,
> and for other stuff. The code actually works fine, but still
> uncompleted is 723 lines (though that includes >>many<< comments),
> which is somewhat of a tail-wagging-dog situation which I also
> want to avoid.


Your smemf() looks interesting, but I'm curious why you went with that
syntax (which, despite claiming to be similar to printf, doesn't seem to
end up looking much like it) rather than leveraging the syntax of an
existing system for the same purpose, eg. Perl's pack/unpack.

S

--
Stephen Sprunk "God does not play dice." --Albert Einstein
CCIE #3723 "God is an inveterate gambler, and He throws the
K5SSS dice at every possible opportunity." --Stephen Hawking
 
Reply With Quote
 
 
 
 
JohnF
Guest
Posts: n/a
 
      09-23-2012
Stephen Sprunk <(E-Mail Removed)> wrote:
> On 22-Sep-12 01:37, JohnF wrote:
>> I'd actually finished (what I think is) a more elegant solution,
>> that I called smemf() that's like memcpy() but under format
>> control, including additonal format specifiers for hex, for bits,
>> and for other stuff. The code actually works fine, but still
>> uncompleted is 723 lines (though that includes >>many<< comments),
>> which is somewhat of a tail-wagging-dog situation which I also
>> want to avoid.

>
> Your smemf() looks interesting, but I'm curious why you went with that
> syntax (which, despite claiming to be similar to printf, doesn't seem to
> end up looking much like it) rather than leveraging the syntax of an
> existing system for the same purpose, eg. Perl's pack/unpack.
> S


You didn't read all the followups -- I wasn't aware of perl's
(or python's) pack/unpack, but they were brought to my attention
(thanks again, guys). Then I indeed said I'd be reading up more
carefully about them all, and trying to re-spec smemf() (and maybe
rename it -- pack or spackf???) based on the all that stuff,
as the best C variant I can come up with.
By the way, compared with those pack/unpack formats, I think
smemf()'s format string syntax is a lot more C/sprintf-like
(could you be more specific?). That's, of course, the trick:
access all functionality from a format string syntax that's
immediately intuitively sensible to both (a)people already
familiar with perl and/or python pack/unpack and maybe C/sprintf,
as well as (b)people only familiar with C/sprintf. Since I'm a
b-type myself, and wasn't even aware of the a-types until
brought to my attention here, the C/sprintf look-alike was
my sole original goal (which I'd thought was pretty successful,
modulo the minimal unavoidable syntax differences due to
fundamental functional requirements differences).
--
John Forkosh ( mailto: http://www.velocityreviews.com/forums/(E-Mail Removed) where j=john and f=forkosh )
 
Reply With Quote
 
Ben Bacarisse
Guest
Posts: n/a
 
      09-24-2012
JohnF <(E-Mail Removed)> writes:
<snip>
> By the way, compared with those pack/unpack formats, I think
> smemf()'s format string syntax is a lot more C/sprintf-like
> (could you be more specific?). That's, of course, the trick:
> access all functionality from a format string syntax that's
> immediately intuitively sensible to both (a)people already
> familiar with perl and/or python pack/unpack and maybe C/sprintf,
> as well as (b)people only familiar with C/sprintf. Since I'm a
> b-type myself, and wasn't even aware of the a-types until
> brought to my attention here, the C/sprintf look-alike was
> my sole original goal (which I'd thought was pretty successful,
> modulo the minimal unavoidable syntax differences due to
> fundamental functional requirements differences).


The main departure from sprintf is that literal characters are not
copied to the destination. That's going to look very odd at first
glance. The main departure from pack/unpack is the lack of support for
alternative byte orderings. As a result, I'm not sure it's all that
close to either familiar "model".

--
Ben.
 
Reply With Quote
 
JohnF
Guest
Posts: n/a
 
      09-24-2012
Ben Bacarisse <(E-Mail Removed)> wrote:
> JohnF <(E-Mail Removed)> writes:
> <snip>
>> By the way, compared with those pack/unpack formats, I think
>> smemf()'s format string syntax is a lot more C/sprintf-like
>> (could you be more specific?). That's, of course, the trick:
>> access all functionality from a format string syntax that's
>> immediately intuitively sensible to both (a)people already
>> familiar with perl and/or python pack/unpack and maybe C/sprintf,
>> as well as (b)people only familiar with C/sprintf. Since I'm a
>> b-type myself, and wasn't even aware of the a-types until
>> brought to my attention here, the C/sprintf look-alike was
>> my sole original goal (which I'd thought was pretty successful,
>> modulo the minimal unavoidable syntax differences due to
>> fundamental functional requirements differences).

>
> The main departure from sprintf is that literal characters are not
> copied to the destination. That's going to look very odd at first
> glance.


Impossible to avoid (I think): what are you supposed to do with
smemf(mem,"deaf"); ?
Is that 4 ascii chars or two hex bytes? Somehow, the
user has specify what he wants. My solution:
smemf(mem,"deaf %s"); or smemf(mem,"deaf %x");
where a format specification preceded by a literal
is applied to that literal rather than eating the next arg.
If you've got a better idea, please follow up... consider
that both a request and a challenge. I tried and failed
to think of any better idea. But I'd be grateful for one.
I can (probably) code it, but I can't think of it.

> The main departure from pack/unpack is the lack of support for
> alternative byte orderings. As a result, I'm not sure it's all that
> close to either familiar "model".


It does have little/big-endian support as an extra %d flag.
In the middle of the state machine parsing the format string...
/* --- endian option flags (for %d) --- */
case 'l': case 'L': endian = (-1); break; /* little-endian */
case 'b': case 'B': endian = (+1); break; /* or big-endian */
case 'e': case 'E': endian = ENDIAN; break; /*or whatever machine uses*/
I just didn't document it very well in the comments yet.
Sorry about that. I have a back-and-forth technique between
writing up the functional specs as comments in enough detail
to get going, then writing some code to see how it works,
then returning to the comments to fix them up or add some more
or whatever, etc, etc.
It was primarily smemf()'s general idea I wanted to get across
by posting that comment block. Obviously, any shortcomings
in the specific functional details can be corrected and coded.
Of course, when I originally posted those comments, I wasn't
yet aware of python/perl pack and unpack. In that case, I could
have gotten across the general idea just by mentioning those.
--
John Forkosh ( mailto: (E-Mail Removed) where j=john and f=forkosh )
 
Reply With Quote
 
Ben Bacarisse
Guest
Posts: n/a
 
      09-24-2012
JohnF <(E-Mail Removed)> writes:

> Ben Bacarisse <(E-Mail Removed)> wrote:
>> JohnF <(E-Mail Removed)> writes:
>> <snip>
>>> By the way, compared with those pack/unpack formats, I think
>>> smemf()'s format string syntax is a lot more C/sprintf-like
>>> (could you be more specific?). That's, of course, the trick:
>>> access all functionality from a format string syntax that's
>>> immediately intuitively sensible to both (a)people already
>>> familiar with perl and/or python pack/unpack and maybe C/sprintf,
>>> as well as (b)people only familiar with C/sprintf. Since I'm a
>>> b-type myself, and wasn't even aware of the a-types until
>>> brought to my attention here, the C/sprintf look-alike was
>>> my sole original goal (which I'd thought was pretty successful,
>>> modulo the minimal unavoidable syntax differences due to
>>> fundamental functional requirements differences).

>>
>> The main departure from sprintf is that literal characters are not
>> copied to the destination. That's going to look very odd at first
>> glance.

>
> Impossible to avoid (I think): what are you supposed to do with
> smemf(mem,"deaf"); ?
> Is that 4 ascii chars or two hex bytes?


My point was that its 4 chars (the encoding is whatever your C
implementation decides, it need not be ASCII) in sprintf and not doing
that causes a difference. You said you'd aimed for (or achieved)
"minimal unavoidable syntax differences".

> Somehow, the
> user has specify what he wants. My solution:
> smemf(mem,"deaf %s"); or smemf(mem,"deaf %x");
> where a format specification preceded by a literal
> is applied to that literal rather than eating the next arg.


In sprintf et. al. the number of required arguments is equal to the
number of format specifiers and everything else is literal bytes. I am
not saying this is wrong, I am saying that it does not match my notion
of "minimal unavoidable syntax differences" from sprintf.

<snip>
--
Ben.
 
Reply With Quote
 
JohnF
Guest
Posts: n/a
 
      09-24-2012
Ben Bacarisse <(E-Mail Removed)> wrote:
> JohnF <(E-Mail Removed)> writes:
>> Ben Bacarisse <(E-Mail Removed)> wrote:
>>> JohnF <(E-Mail Removed)> writes:
>>> <snip>
>>>> By the way, compared with those pack/unpack formats, I think
>>>> smemf()'s format string syntax is a lot more C/sprintf-like
>>>> (could you be more specific?). That's, of course, the trick:
>>>> access all functionality from a format string syntax that's
>>>> immediately intuitively sensible to both (a)people already
>>>> familiar with perl and/or python pack/unpack and maybe C/sprintf,
>>>> as well as (b)people only familiar with C/sprintf. Since I'm a
>>>> b-type myself, and wasn't even aware of the a-types until
>>>> brought to my attention here, the C/sprintf look-alike was
>>>> my sole original goal (which I'd thought was pretty successful,
>>>> modulo the minimal unavoidable syntax differences due to
>>>> fundamental functional requirements differences).
>>>
>>> The main departure from sprintf is that literal characters are not
>>> copied to the destination. That's going to look very odd at first
>>> glance.

>>
>> Impossible to avoid (I think): what are you supposed to do with
>> smemf(mem,"deaf"); ?
>> Is that 4 ascii chars or two hex bytes?

>
> My point was that its 4 chars (the encoding is whatever your C
> implementation decides, it need not be ASCII) in sprintf and not doing
> that causes a difference. You said you'd aimed for (or achieved)
> "minimal unavoidable syntax differences".
>
>> Somehow, the user to has specify what he wants.
>> My solution:
>> smemf(mem,"deaf %s"); or smemf(mem,"deaf %x");
>> where a format specification preceded by a literal
>> is applied to that literal rather than eating the next arg.

>
> In sprintf et. al. the number of required arguments is equal to the
> number of format specifiers and everything else is literal bytes. I am
> not saying this is wrong, I am saying that it does not match my notion
> of "minimal unavoidable syntax differences" from sprintf.
> <snip>


Well, part of what you snipped was my request for an alternative.
Lacking that, it matches the notion of "minimal unavoidable
syntax differences" because of the "unavoidable" part.
You have to somehow deal with "deaf". And, even worse, with "10",
which could be ascii/decimal/hex/bits, and which I deal with
in one consistent way, "10%s"/"10%d"/"10%x"/"10%b".
For someone familiar and comfortable with sprintf formats,
that's the most intuitively sensible way I could think of to
deal with the problem. And it >>must<< be dealt with somehow.
If you've got a better idea, I'd be happy to code it.
Otherwise, it's an unavoidable difference.
--
John Forkosh ( mailto: (E-Mail Removed) where j=john and f=forkosh )
 
Reply With Quote
 
BartC
Guest
Posts: n/a
 
      09-24-2012
"JohnF" <(E-Mail Removed)> wrote in message
news:k3oi08$14i$(E-Mail Removed)...
> Ben Bacarisse <(E-Mail Removed)> wrote:


>> The main departure from pack/unpack is the lack of support for
>> alternative byte orderings. As a result, I'm not sure it's all that
>> close to either familiar "model".


> It was primarily smemf()'s general idea I wanted to get across
> by posting that comment block. Obviously, any shortcomings
> in the specific functional details can be corrected and coded.
> Of course, when I originally posted those comments, I wasn't
> yet aware of python/perl pack and unpack. In that case, I could
> have gotten across the general idea just by mentioning those.


Actually this seems a much better idea to me now, than it did at first ...
if you forget about the purpose of using it in place of C's packed structs.

It's just an alternative, possibly simpler way of writing to a binary stream
than trying to use -printf() functions, or sequences of function calls. And
presumably to read from them too.

But you didn't explain clearly how it would be used. In the GIF example,
presumably you'd have an *unpacked* struct representing the header
information, which allows the program to access fields, now properly
aligned, using the conventional forms of 'p.a' or 'p->a'.

smem() (and it's -scanf() counterpart) would simply be used to write to the
proper packed form, or to read from it.

So they are not a plug-in replacement for '#pragma pack()'.

--
Bartc

 
Reply With Quote
 
JohnF
Guest
Posts: n/a
 
      09-24-2012
BartC <(E-Mail Removed)> wrote:
> "JohnF" <(E-Mail Removed)> wrote
>> Ben Bacarisse <(E-Mail Removed)> wrote:

>
>>> The main departure from pack/unpack is the lack of support for
>>> alternative byte orderings. As a result, I'm not sure it's all that
>>> close to either familiar "model".

>
>> It was primarily smemf()'s general idea I wanted to get across
>> by posting that comment block. Obviously, any shortcomings
>> in the specific functional details can be corrected and coded.
>> Of course, when I originally posted those comments, I wasn't
>> yet aware of python/perl pack and unpack. In that case, I could
>> have gotten across the general idea just by mentioning those.

>
> Actually this seems a much better idea to me now, than it did at first ...
> if you forget about the purpose of using it in place of C's packed structs.
>
> It's just an alternative, possibly simpler way of writing to a binary stream
> than trying to use -printf() functions, or sequences of function calls. And
> presumably to read from them too.
>
> But you didn't explain clearly how it would be used. In the GIF example,
> presumably you'd have an *unpacked* struct representing the header
> information, which allows the program to access fields, now properly
> aligned, using the conventional forms of 'p.a' or 'p->a'.


Well, you'd use it however you liked. But for my gif situation,
I'd envisioned >>doing away with structs entirely<<, packed or not.
The smemf >>format string totally replaces<< the need for any struct.
And, of course, I prototyped that for myself, i.e., wrote some
pseudocode using the as-yet-uncompleted smemf, just to make sure
that idea seems to work.
You can see exhaustively complete comments about the gif block
formats at forkosh.com/gifsave89.html by clicking the Listing link
along the left-hand side under "Related Pages", and scrolling down
to line#493 for the GIFIMAGEDESCRIP struct. My "pseudocode" for that
is just one smemf statement that totally replaces the struct,

nbitsinbuffer = /* whitespace in smemf format string is ignored */
smemf(buffer, " 2C %x " /* Image Descriptor identifier is hex 2C */
" %2ld" /* 2-byte little-endian word for X-pos */
" %2ld" /* 2-byte little-endian word for Top */
" %2ld" /* 2-byte little-endian word for Width */
" %2ld" /* 2-byte little-endian word for Height */
/* following is the "Packed" Byte consisting of five bit fields */
" %3b " /* 3-bits #colorbits */
" 0 %2b " /* 2-bits "reserved bits" */
" 0 %1b " /* 1-bit local colortable sort flag */
" 0 %1b " /* 1-bit interlace flag */
" %1b ", /* 1-bit local colortable flag */
col0,row0, ncols,nrows, ncolorbits, (colortable==NULL?0:1) );

And smemf() returns the size, in #bits, of the buffer it constructs.
That would usually be a multiple of 8, in which case you can just
fwrite(buffer,etc), or do whatever you want with it.

> smem() (and it's -scanf() counterpart) would simply be used to write to the
> proper packed form, or to read from it.
> So they are not a plug-in replacement for '#pragma pack()'.


--
John Forkosh ( mailto: (E-Mail Removed) where j=john and f=forkosh )
 
Reply With Quote
 
Ben Bacarisse
Guest
Posts: n/a
 
      09-24-2012
JohnF <(E-Mail Removed)> writes:

> Ben Bacarisse <(E-Mail Removed)> wrote:
>> JohnF <(E-Mail Removed)> writes:
>>> Ben Bacarisse <(E-Mail Removed)> wrote:
>>>> JohnF <(E-Mail Removed)> writes:
>>>> <snip>
>>>>> By the way, compared with those pack/unpack formats, I think
>>>>> smemf()'s format string syntax is a lot more C/sprintf-like
>>>>> (could you be more specific?). That's, of course, the trick:
>>>>> access all functionality from a format string syntax that's
>>>>> immediately intuitively sensible to both (a)people already
>>>>> familiar with perl and/or python pack/unpack and maybe C/sprintf,
>>>>> as well as (b)people only familiar with C/sprintf. Since I'm a
>>>>> b-type myself, and wasn't even aware of the a-types until
>>>>> brought to my attention here, the C/sprintf look-alike was
>>>>> my sole original goal (which I'd thought was pretty successful,
>>>>> modulo the minimal unavoidable syntax differences due to
>>>>> fundamental functional requirements differences).
>>>>
>>>> The main departure from sprintf is that literal characters are not
>>>> copied to the destination. That's going to look very odd at first
>>>> glance.
>>>
>>> Impossible to avoid (I think): what are you supposed to do with
>>> smemf(mem,"deaf"); ?
>>> Is that 4 ascii chars or two hex bytes?

>>
>> My point was that its 4 chars (the encoding is whatever your C
>> implementation decides, it need not be ASCII) in sprintf and not doing
>> that causes a difference. You said you'd aimed for (or achieved)
>> "minimal unavoidable syntax differences".
>>
>>> Somehow, the user to has specify what he wants.
>>> My solution:
>>> smemf(mem,"deaf %s"); or smemf(mem,"deaf %x");
>>> where a format specification preceded by a literal
>>> is applied to that literal rather than eating the next arg.

>>
>> In sprintf et. al. the number of required arguments is equal to the
>> number of format specifiers and everything else is literal bytes. I am
>> not saying this is wrong, I am saying that it does not match my notion
>> of "minimal unavoidable syntax differences" from sprintf.
>> <snip>

>
> Well, part of what you snipped was my request for an alternative.
> Lacking that, it matches the notion of "minimal unavoidable
> syntax differences" because of the "unavoidable" part.


I gave you my suggestion which is why I snipped the request -- literal
text just represents those characters. That's what sprintf does and
therefor offers the least "surprise".

> You have to somehow deal with "deaf". And, even worse, with "10",
> which could be ascii/decimal/hex/bits, and which I deal with
> in one consistent way, "10%s"/"10%d"/"10%x"/"10%b".
> For someone familiar and comfortable with sprintf formats,
> that's the most intuitively sensible way I could think of to
> deal with the problem. And it >>must<< be dealt with somehow.


Doing what sprintf does must surely be closer than doing something new.
This problem that you insist must be dealt with seems to me to be an
invented one: it's intuitive (to me at least) that "deaf" in a format
means copy a 'd' an 'e' an 'a' and then an 'f' to the destination.

> If you've got a better idea, I'd be happy to code it.
> Otherwise, it's an unavoidable difference.


--
Ben.
 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Packed structs vs. unpacked structs: what's the difference? Daniel Rudy C Programming 15 04-10-2006 08:10 AM
Array of structs instead of an array with pointers to structs? Paminu C Programming 5 10-11-2005 07:18 PM
length of an array in a struct in an array of structs in a struct in an array of structs Tuan Bui Perl Misc 14 07-29-2005 02:39 PM
const structs in other structs Chris Hauxwell C Programming 6 04-27-2004 07:03 PM
structs with fields that are structs Patricia Van Hise C Programming 5 04-05-2004 01:37 AM



Advertisments