Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > C Programming > packed structs

Reply
Thread Tools

packed structs

 
 
JohnF
Guest
Posts: n/a
 
      09-24-2012
Ben Bacarisse <(E-Mail Removed)> wrote:
> JohnF <(E-Mail Removed)> writes:
>>
>> what are you supposed to do with
>> smemf(mem,"deaf"); ?
>> Is that 4 ascii chars or two hex bytes?

>
> it's intuitive (to me at least) that "deaf" in a format
> means copy a 'd' an 'e' an 'a' and then an 'f' to the destination.


Okay, good to know. Thanks so much for your thoughts.
--
John Forkosh ( mailto: http://www.velocityreviews.com/forums/(E-Mail Removed) where j=john and f=forkosh )
 
Reply With Quote
 
 
 
 
Stephen Sprunk
Guest
Posts: n/a
 
      09-30-2012
On 23-Sep-12 18:23, JohnF wrote:
> Stephen Sprunk <(E-Mail Removed)> wrote:
>> On 22-Sep-12 01:37, JohnF wrote:
>>> I'd actually finished (what I think is) a more elegant solution,
>>> that I called smemf() that's like memcpy() but under format
>>> control, including additonal format specifiers for hex, for
>>> bits, and for other stuff. The code actually works fine, but
>>> still uncompleted is 723 lines (though that includes >>many<<
>>> comments), which is somewhat of a tail-wagging-dog situation
>>> which I also want to avoid.

>>
>> Your smemf() looks interesting, but I'm curious why you went with
>> that syntax (which, despite claiming to be similar to printf,
>> doesn't seem to end up looking much like it) rather than leveraging
>> the syntax of an existing system for the same purpose, eg. Perl's
>> pack/unpack.

>
> You didn't read all the followups -- I wasn't aware of perl's (or
> python's) pack/unpack, but they were brought to my attention (thanks
> again, guys). Then I indeed said I'd be reading up more carefully
> about them all, and trying to re-spec smemf() (and maybe rename it --
> pack or spackf???) based on the all that stuff, as the best C variant
> I can come up with.


Fair enough.

> By the way, compared with those pack/unpack formats, I think
> smemf()'s format string syntax is a lot more C/sprintf-like (could
> you be more specific?).


Aside from your use of percent signs for format specifiers, I don't see
much in common, and in fact I find your use of percent signs to be quite
misleading since they behave rather differently than printf()'s.

> That's, of course, the trick: access all functionality from a format
> string syntax that's immediately intuitively sensible to both
> (a)people already familiar with perl and/or python pack/unpack and
> maybe C/sprintf, as well as (b)people only familiar with C/sprintf.


I don't see how (a) could have been a goal if you hadn't been aware of
Perl's pack()/unpack() when you developed your smemf() interface.

> Since I'm a b-type myself, and wasn't even aware of the a-types until
> brought to my attention here, the C/sprintf look-alike was my sole
> original goal (which I'd thought was pretty successful, modulo the
> minimal unavoidable syntax differences due to fundamental functional
> requirements differences).


Some differences were unavoidable, but others were not.

For instance, sprintf() copies the format string verbatim to the output
except for format specifiers, which are replaced by formatted arguments.
smemf() discards whitespace and puts literal arguments in the format
string, which are consumed by subsequent format specifiers.

More detail in my response to your example.

S

--
Stephen Sprunk "God does not play dice." --Albert Einstein
CCIE #3723 "God is an inveterate gambler, and He throws the
K5SSS dice at every possible opportunity." --Stephen Hawking
 
Reply With Quote
 
 
 
 
Stephen Sprunk
Guest
Posts: n/a
 
      09-30-2012
On 24-Sep-12 06:04, JohnF wrote:
> Well, you'd use it however you liked. But for my gif situation,
> I'd envisioned >>doing away with structs entirely<<, packed or not.
> The smemf >>format string totally replaces<< the need for any struct.
> And, of course, I prototyped that for myself, i.e., wrote some
> pseudocode using the as-yet-uncompleted smemf, just to make sure
> that idea seems to work.
> You can see exhaustively complete comments about the gif block
> formats at forkosh.com/gifsave89.html by clicking the Listing link
> along the left-hand side under "Related Pages", and scrolling down
> to line#493 for the GIFIMAGEDESCRIP struct. My "pseudocode" for that
> is just one smemf statement that totally replaces the struct,
>
> nbitsinbuffer = /* whitespace in smemf format string is ignored */
> smemf(buffer, " 2C %x " /* Image Descriptor identifier is hex 2C */
> " %2ld" /* 2-byte little-endian word for X-pos */
> " %2ld" /* 2-byte little-endian word for Top */
> " %2ld" /* 2-byte little-endian word for Width */
> " %2ld" /* 2-byte little-endian word for Height */
> /* following is the "Packed" Byte consisting of five bit fields */
> " %3b " /* 3-bits #colorbits */
> " 0 %2b " /* 2-bits "reserved bits" */
> " 0 %1b " /* 1-bit local colortable sort flag */
> " 0 %1b " /* 1-bit interlace flag */
> " %1b ", /* 1-bit local colortable flag */
> col0,row0, ncols,nrows, ncolorbits, (colortable==NULL?0:1) );
>
> And smemf() returns the size, in #bits, of the buffer it constructs.
> That would usually be a multiple of 8, in which case you can just
> fwrite(buffer,etc), or do whatever you want with it.


This example shows how far you have diverged from sprintf(). From such
a claim, I would have expected something more like this:

nbitsinbuffer = smemf(buffer,
"%1lu" /* Image Descriptor, 8-bit little-endian unsigned int */
"%2lu" /* X-Pos, 16-bit little-endian unsigned int */
"%2lu" /* Top, 16-bit little-endian unsigned int */
"%2lu" /* Width, 16-bit little-endian unsigned int */
"%2lu" /* Height, 16-bit little-endian unsigned int */
/* following is the "Packed" Byte consisting of five bit fields */
"%3b" /* #colorbits, 3-bit field */
"%2b" /* reserved, 2-bit field */
"%1b" /* local colortable sort flag, 1-bit field */
"%1b" /* interlace flag, 1-bit field */
"%1b", /* local colortable flag, 1-bit field */
0x2C, col0, row0, ncols, nrows, ncolorbits, 0, 0, 0,
(colortable==NULL?0:1) );

Notice that whitespace is _not_ ignored and that there are ten arguments
corresponding to ten format specifiers. I also used %u rather than %d
since I'm pretty sure those ints are supposed to be unsigned (but I'm
not sure that makes a difference here).

Of course, your use of "%ld" to mean "little-endian word" rather than
"long signed integer" is a major difference as well, though that's
forgivable since you obviously need more control over representation
than sprintf()'s specifiers offer. When thinking it through, though,
that was the point at which I decided that trying to reuse those
specifiers was probably more trouble than it was worth.

S

--
Stephen Sprunk "God does not play dice." --Albert Einstein
CCIE #3723 "God is an inveterate gambler, and He throws the
K5SSS dice at every possible opportunity." --Stephen Hawking
 
Reply With Quote
 
JohnF
Guest
Posts: n/a
 
      10-01-2012
Stephen Sprunk <(E-Mail Removed)> wrote:
> JohnF wrote:
>> Well, you'd use it however you liked. But for my gif situation,
>> I'd envisioned >>doing away with structs entirely<<, packed or not.
>> The smemf >>format string totally replaces<< the need for any struct.
>> And, of course, I prototyped that for myself, i.e., wrote some
>> pseudocode using the as-yet-uncompleted smemf, just to make sure
>> that idea seems to work.
>> You can see exhaustively complete comments about the gif block
>> formats at forkosh.com/gifsave89.html by clicking the Listing link
>> along the left-hand side under "Related Pages", and scrolling down
>> to line#493 for the GIFIMAGEDESCRIP struct. My "pseudocode" for that
>> is just one smemf statement that totally replaces the struct,
>>
>> nbitsinbuffer = /* whitespace in smemf format string is ignored */
>> smemf(buffer, " 2C %x " /* Image Descriptor identifier is hex 2C */
>> " %2ld" /* 2-byte little-endian word for X-pos */
>> " %2ld" /* 2-byte little-endian word for Top */
>> " %2ld" /* 2-byte little-endian word for Width */
>> " %2ld" /* 2-byte little-endian word for Height */
>> /* following is the "Packed" Byte consisting of five bit fields */
>> " %3b " /* 3-bits #colorbits */
>> " 0 %2b " /* 2-bits "reserved bits" */
>> " 0 %1b " /* 1-bit local colortable sort flag */
>> " 0 %1b " /* 1-bit interlace flag */
>> " %1b ", /* 1-bit local colortable flag */
>> col0,row0, ncols,nrows, ncolorbits, (colortable==NULL?0:1) );
>>
>> And smemf() returns the size, in #bits, of the buffer it constructs.
>> That would usually be a multiple of 8, in which case you can just
>> fwrite(buffer,etc), or do whatever you want with it.

>
> This example shows how far you have diverged from sprintf(). From such
> a claim, I would have expected something more like this:
>
> nbitsinbuffer = smemf(buffer,
> "%1lu" /* Image Descriptor, 8-bit little-endian unsigned int */
> "%2lu" /* X-Pos, 16-bit little-endian unsigned int */
> "%2lu" /* Top, 16-bit little-endian unsigned int */
> "%2lu" /* Width, 16-bit little-endian unsigned int */
> "%2lu" /* Height, 16-bit little-endian unsigned int */
> /* following is the "Packed" Byte consisting of five bit fields */
> "%3b" /* #colorbits, 3-bit field */
> "%2b" /* reserved, 2-bit field */
> "%1b" /* local colortable sort flag, 1-bit field */
> "%1b" /* interlace flag, 1-bit field */
> "%1b", /* local colortable flag, 1-bit field */
> 0x2C, col0, row0, ncols, nrows, ncolorbits, 0, 0, 0,
> (colortable==NULL?0:1) );


Thanks for the alternative spec. I'll keep it in mind.
I agree that your format's more sprintf-like. Maybe I
should "deprecate" that literal requirement. More
about why in response to your specific remarks below...

> Notice that whitespace is _not_ ignored and that there are ten arguments
> corresponding to ten format specifiers.


Both of which I'd consider bad news. Certainly, ignoring
whitespace is different. But notice that your format string
has none. And likely never would, because how often is
whitespace going to be part of a binary packet format
specification? So it's basically no use for this particular
purpose.
And you also don't get constants in your formats.
Instead, you've got that 0x2C and 0,0,0 in the arguments
which "force-feeds" the "%1b" specifiers. And which
the user has to know about. Instead, I'm moving those
same 0x2C and 0's into the format string
preceding the %1b. Different, again, but serves
the purpose better. The user shouldn't have to
remember magic numbers like 0x2C, and where to put
them.
Instead, maybe some .h file with all the format
strings describing the packets for some particular
protocol/whatever would embed all that info and
hide it from the user. He'd just say something like,
#include "packetformatstrings.h"
unsigned char *thispacketbuffer = malloc(whatever);
...
smemf(thispacketbuffer, thispacketformatstring,
all,the,numbers,I,actually,care,about,and,no,other s);
And that would do the entire job, with the user taking
care of his business only, and the binary packet formatting
pretty much entirely handled for him.

> I also used %u rather than %d
> since I'm pretty sure those ints are supposed to be unsigned (but I'm
> not sure that makes a difference here).
>
> Of course, your use of "%ld" to mean "little-endian word" rather than
> "long signed integer" is a major difference as well, though that's
> forgivable since you obviously need more control over representation
> than sprintf()'s specifiers offer. When thinking it through, though,
> that was the point at which I decided that trying to reuse those
> specifiers was probably more trouble than it was worth.


Forget the characters %u vs %d and length modifier, flags, etc.
That's easily/trivially changed. In fact, I'd already changed
my mind to %<d or (%<u if you prefer) for little endian,
and the obvious for big, but might change it again.
Just focus on the functionality. The format string syntax,
at least so far as specifiers, flags, modifiers, etc are
concerned, can be decided later. For now, they're just
illustrative -- you have to write something while discussing it.

> S

--
John Forkosh ( mailto: (E-Mail Removed) where j=john and f=forkosh )
 
Reply With Quote
 
Stephen Sprunk
Guest
Posts: n/a
 
      10-15-2012
On 01-Oct-12 02:34, JohnF wrote:
> Stephen Sprunk <(E-Mail Removed)> wrote:
>> This example shows how far you have diverged from sprintf(). From
>> such a claim, I would have expected something more like this:
>>
>> ...

>
> Thanks for the alternative spec. I'll keep it in mind. I agree that
> your format's more sprintf-like. Maybe I should "deprecate" that
> literal requirement. More about why in response to your specific
> remarks below...


Note that I'm not saying that my syntax is better; I'm just saying that
it is more like a sprintf() format string, which may actually turn out
to be worse than coming up with something completely new.

>> Notice that whitespace is _not_ ignored and that there are ten
>> arguments corresponding to ten format specifiers.

>
> Both of which I'd consider bad news. Certainly, ignoring whitespace
> is different. But notice that your format string has none. And likely
> never would, because how often is whitespace going to be part of a
> binary packet format specification? So it's basically no use for this
> particular purpose.


sprintf() copies everything in the format string (including whitespace)
to its output, replacing format specifiers as it goes. If you're going
to claim to be similar to sprintf(), then you need to do the same.

> And you also don't get constants in your formats. Instead, you've got
> that 0x2C and 0,0,0 in the arguments which "force-feeds" the "%1b"
> specifiers. And which the user has to know about. Instead, I'm moving
> those same 0x2C and 0's into the format string preceding the %1b.
> Different, again, but serves the purpose better. The user shouldn't
> have to remember magic numbers like 0x2C, and where to put them.


I can see cases where I might want to include literal bytes in the
format string, such as the "GIF89a" at the beginning of GIF files,
without having to use a format specifier. In some cases, those literal
bytes might happen to include "whitespace".

Instead, rather that doing the same thing as sprintf(), you chose to
have format specifiers modify the preceding literal text in the format
string and treat whitespace specially, which is completely unlike sprintf().

>> I also used %u rather than %d since I'm pretty sure those ints are
>> supposed to be unsigned (but I'm not sure that makes a difference
>> here).
>>
>> Of course, your use of "%ld" to mean "little-endian word" rather
>> than "long signed integer" is a major difference as well, though
>> that's forgivable since you obviously need more control over
>> representation than sprintf()'s specifiers offer. When thinking it
>> through, though, that was the point at which I decided that trying
>> to reuse those specifiers was probably more trouble than it was
>> worth.

>
> Forget the characters %u vs %d and length modifier, flags, etc.
> That's easily/trivially changed. In fact, I'd already changed my mind
> to %<d or (%<u if you prefer) for little endian, and the obvious for
> big, but might change it again.


I'm curious how you'd specify middle-endian. Presumably, without one of
those three modifiers, it would use native byte order?

> Just focus on the functionality. The format string syntax, at least
> so far as specifiers, flags, modifiers, etc are concerned, can be
> decided later. For now, they're just illustrative -- you have to
> write something while discussing it.


My point was merely to illustrate why I said your syntax didn't seem to
be similar to sprintf() as you had claimed. I'm not debating whether it
is a good syntax without such a claim.

S

--
Stephen Sprunk "God does not play dice." --Albert Einstein
CCIE #3723 "God is an inveterate gambler, and He throws the
K5SSS dice at every possible opportunity." --Stephen Hawking
 
Reply With Quote
 
W Karas
Guest
Posts: n/a
 
      10-16-2012
On Friday, September 21, 2012 9:54:36 PM UTC-4, JohnF wrote:
> Any >>portable<< way to accomplish that in c?
>
> Don't want to use __attribute__((__packed__))
>
> or #pragma pack, etc, nor #ifdef's to choose
>
> among whatever alternatives I happen to know
>
> about. It's a requirement that the code remain
>
> portable.
>
>
>
> In particular, I'm trying to write blocks that
>
> conform to a binary file format (gif), and can
>
> set up structs for them easily enough, but can't
>
> fwrite(blockstruct,sizeof(blockstruct),1,fileptr),
>
> or the like, due to blockstruct's inevitable
>
> padding (which indeed occurs for gif format blocks).
>
>
>
> At the moment, I just have a different func for
>
> each block type that writes out the members of that
>
> particular struct individually... b..o..r..i..n..g.
>
> A generalization of that idea (if portable packing's
>
> not possible) would also be fine: >>if<< there's some
>
> way to reference the members of a struct, passed as
>
> an argument but of unknown (to the func) type, in a
>
> loop, i.e., for(i=0;i<nmembers;i++)thisstruct->member[i].
>
> Then I could offsetof() and sizeof() each member, and
>
> write it out, so just one (much less boring) func
>
> could handle all the different block type structs.
>
>
>
> But, afaik, I don't think that thisstruct->member[i]
>
> thing is possible, nor portable packing. So is there
>
> any "one size fits all" way to handle this problem?
>
> I'm sure people must come across it frequently enough
>
> that it's been thought about, and the best possible
>
> approach (among, possibly, several bad alternatives)
>
> has been identified.
>
> --
>
> John Forkosh ( mailto: (E-Mail Removed) where j=john and f=forkosh )


typedef unsigned char P2b[2];

#define SETP2(PTR, VAL) \
((PTR)[0] = (VAL) >> CHAR_BIT, (PTR)[1] = (VAL) & ((1 << CHAR_BIT) - 1))

#define GETP2(PTR) \
(((PTR)[0] << CHAR_BIT) | (PTR)[0])

Larger "packable ints" are analogous. My choice of big-endian here is arbitrary.
 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Packed structs vs. unpacked structs: what's the difference? Daniel Rudy C Programming 15 04-10-2006 08:10 AM
Array of structs instead of an array with pointers to structs? Paminu C Programming 5 10-11-2005 07:18 PM
length of an array in a struct in an array of structs in a struct in an array of structs Tuan Bui Perl Misc 14 07-29-2005 02:39 PM
const structs in other structs Chris Hauxwell C Programming 6 04-27-2004 07:03 PM
structs with fields that are structs Patricia Van Hise C Programming 5 04-05-2004 01:37 AM



Advertisments