«

On 9/13/11 7:52 PM, BartC wrote:
>
> Suppose *all* addresses have to be word-aligned (which used to be quite
> common). Then you'd have the same problem with:
>
> struct foo {
> char c[2];
> int i;
> };
>
> and trying to take the address of c[1] (or even just trying to access
> it). (This assumes you don't just make char and int the same size.)
>
> The Pascal 'packed' directive perhaps *was* an instructive to pack
> things more tightly than hardware considerations would normally dictate.
>

If you can only take address of word aligned objects then the compiler
can either make char the size of a word, even if this is more that the
"expected" 8 bits, or it can make a char* (and void*) pointer bigger by
adding an extra word to it that tells it which "byte" within the word to
access and adds code to all char accesses to use that extra information.
I have seen both done on systems I have programed for, and both can trip
up people who are used to "normal systems", and who assume CHAR_BIT = 8
and sizeof(int*) == sizeof(char*)

"BartC" <> writes:
> "Keith Thompson" <kst-> wrote in message
> news:...
>> Roberto Waltman <> writes:
>
>>> Even Pascal had packed records.
>>
>> Yes, but Pascal packed records, as I recall, are merely a hint to the
>> compiler that space should be minimized. They're not necessarily
>> suitable for matching a specific externally imposed layout.
>>
>> One example of a problem that would have to be solved:
>>
>> struct foo {
>> char c;
>> int i;
>> };
>> #pragma pack(struct foo) /* or whatever the syntax is */
>>
>> void some_func(int *p);
>>
>> struct foo obj;
>> some_func(&obj.i);
>
>> I've worked on systems where an int* cannot contain an address that
>> isn't word-aligned. Do you forbid taking the address of a member of a
>> packed struct?
>
> Suppose *all* addresses have to be word-aligned (which used to be quite
> common). Then you'd have the same problem with:
>
> struct foo {
> char c[2];
> int i;
> };
>
> and trying to take the address of c[1] (or even just trying to access it).
> (This assumes you don't just make char and int the same size.)

You're making assumptions that make a conforming C implementation
impossible, unless a "word" is 1 byte. c[1] must have a valid address.
That address is of type char*, which may or may not have the same
representation as an int*. On the system I was referring to, a char*
can old a byte-aligned address, but an int* can only hold a word-aligned
address.

(The system was the Cray T90. The word size was 64 bits, and there was
no hardware support for byte addressing. Byte addresses were
implemented in software by storing an offset in the high-order 3 bits of
a 64-bit pointer.)

> The Pascal 'packed' directive perhaps *was* an instructive to pack things
> more tightly than hardware considerations would normally dictate.

--
Keith Thompson (The_Other_Keith) kst- <http://www.ghoti.net/~kst>
"We must do something. This is something. Therefore, we must do this."
-- Antony Jay and Jonathan Lynn, "Yes Minister"

Richard Damon <> writes:

> On 9/13/11 7:52 PM, BartC wrote:
>>
>> Suppose *all* addresses have to be word-aligned (which used to be quite
>> common). Then you'd have the same problem with:
>>
>> struct foo {
>> char c[2];
>> int i;
>> };
>>
>> and trying to take the address of c[1] (or even just trying to access
>> it). (This assumes you don't just make char and int the same size.)
>>
>> The Pascal 'packed' directive perhaps *was* an instructive to pack
>> things more tightly than hardware considerations would normally dictate.
>>
>
> If you can only take address of word aligned objects then the compiler
> can either make char the size of a word, even if this is more that the
> "expected" 8 bits, or it can make a char* (and void*) pointer bigger
> by adding an extra word to it that tells it which "byte" within the
> word to access and adds code to all char accesses to use that extra
> information. I have seen both done on systems I have programed for,
> and both can trip up people who are used to "normal systems", and who
> assume CHAR_BIT = 8 and sizeof(int*) == sizeof(char*)

Another strategy I've seen is to use unused bits of the address to
denote which char from the word a char * (or void *) points to. These
can be high bits or low bits. If high bits are used, bytes addresses
need to be masked, if low bits are used they need to be shifted.

Non-portable code is something of a self-fulfilling prophesy. Machines
with interesting architectures are very likely to fail because there is
so much code that is hard to run on them. Chip designers have one hand
tied behind their backs. Odd since software is supposed to be the
flexible member of the partnership.

--
Ben.

On 9/13/2011 8:24 AM, Roberto Waltman wrote:
> Eric Sosman wrote:
>> ... if you really think you need "pack(1)". Usually (not always,
>> but usually) that's the sign of a short-term hack leading to long-
>> term headaches.
>
> Not when you are dealing with communication protocols, mapping
> hardware control registers in embedded systems, etc.

For communication protocols: Even if you can arrange a struct's
layout to match the foreign format, you still need to deal with
representation. That two-byte integer: Is it big- or little-endian?
Ordinary positional binary or a Gray code? Is the low-order bit a
value, or is it parity? Layout is only one of the issues when mediating
between internal values and external forms, and "solving" them by
arranging a struct merely ignores the others. If you manage to get it
working with a particular compiler on a particular machine, and if "it
works" seduces you into thinking you've found the right approach, I
predict long-term headaches.

For memory-mapped registers: Although it's quite unlikely that
issues like endianness will arise, brand-new problems crop up. For
example, a four-byte register at 0xF00-0xF03 might do nothing at all
in response to accesses at 0xF01,0xF02,0xF03 -- it's not memory,
after all, but a little gadget somewhere that's waiting for 0xF00
to show up on the address lines. So you lay out your struct just like
the manual says: Four flag bits, four must-be-zero bits, a three-bit
command code and five bit sub-command code, and a two-byte integer for
the swizzle selector value. And then you store to the swizzle selector
(at 0xF02), and the hardware device ... ignores you. (Even worse,
maybe it ignores the two low-order address bits and treats the swizzle
selector as flags-and-commands!) The pretty layout of the struct has
not helped in the slightest; again, you need a completely different
approach.

> Without the "pack" pragmas, instead of having a structure that
> directly overlaps the desired memory layout, the packing/unpacking
> would have to be in higher level software.

It has to be so anyhow. If not today, then tomorrow when the
long-term headaches begin to throb.

> This is a much needed functionality, and I am puzzled why it never
> became part of the language standard, instead of a not always
> available compiler extension.

It could be made to work even on machines that have alignment
requirements for memory-resident objects. All you need to do is
fetch and store multi-byte objects one byte at a time, marshalling
them and unmarshalling them with extra instructions. You've got to
do this not only for packed structs, but for any pointer that might
point to a misaligned object somewhere -- yes, qsort() just slowed
down some more. This seems a high price to pay for functionality
which, despite your statement to the contrary, is in no way "needed."

That's my story, and I'm sticking to it.

--
Eric Sosman
d

On Wed, 2011-09-14, Eric Sosman wrote:
> On 9/13/2011 8:24 AM, Roberto Waltman wrote:
>> Eric Sosman wrote:
>>> ... if you really think you need "pack(1)". Usually (not always,
>>> but usually) that's the sign of a short-term hack leading to long-
>>> term headaches.
>>
>> Not when you are dealing with communication protocols, mapping
>> hardware control registers in embedded systems, etc.
>
> For communication protocols: Even if you can arrange a struct's
> layout to match the foreign format, you still need to deal with
> representation. That two-byte integer: Is it big- or little-endian?
> Ordinary positional binary or a Gray code? Is the low-order bit a
> value, or is it parity? Layout is only one of the issues when mediating
> between internal values and external forms, and "solving" them by
> arranging a struct merely ignores the others.

Plus, you now have leaked the concept of foreign endianness into your
code. As soon as you open a hole in the type system

const struct Foo* foo = (struct Foo*)some_octet_buffer;

things which look like integers but /aren't/ are free to leak into any
part of your code, and sooner or later you'll accidentally do
arithmetics on something which is in foreign byte-order.

Foreign byte-order scalars are like zombies. You don't want them
indoors; they have bad manners.

/Jorgen

--
// Jorgen Grahn <grahn@ Oo o. . .
\X/ snipabacken.se> O o .

On Tue, 2011-09-13, James Kuyper wrote:
> On 09/13/2011 08:50 AM, BartC wrote:
>> "Eric Sosman" <> wrote in message
>> news:j4ngch$oqb$...
>>> On 9/13/2011 5:47 AM, Wang WolfLouis wrote:
>>
>>>> #define WIN32_PACKED #pragma pack(1)
>>
>>> ... if you really think you need "pack(1)". Usually (not always,
>>> but usually) that's the sign of a short-term hack leading to long-
>>> term headaches.
>>
>> I would guess that 'usually' it is to match a layout defined outside your
>> control. (In this case, perhaps because it's not practical to rewrite the
>> whole of Windows.)
>
> There's no need to rewrite Windows; just read the data into an array of
> unsigned char, then unpack the array into each member of the structure
> using memcpy().

Or utility functions like

static unsigned eat16(const uint8_t*& p)
{
unsigned n = *p++ << 8;
n |= *p++;
return n;
}

(OK, that was the C++ version, but that was what I had handy.)

/Jorgen

--
// Jorgen Grahn <grahn@ Oo o. . .
\X/ snipabacken.se> O o .

Roberto Waltman <> writes:
> Eric Sosman wrote:
> > ... if you really think you need "pack(1)". Usually (not always,
> >but usually) that's the sign of a short-term hack leading to long-
> >term headaches.
>
> Not when you are dealing with communication protocols, mapping
> hardware control registers in embedded systems, etc.

Packing doesn't address endianness. You have lost already.

Phil
--
"Religion is what keeps the poor from murdering the rich."
-- Napoleon

Keith Thompson <kst-> writes:
> (The system was the Cray T90. The word size was 64 bits, and there was
> no hardware support for byte addressing. Byte addresses were
> implemented in software by storing an offset in the high-order 3 bits of
> a 64-bit pointer.)

I presume the Cray T3E is the similar, as that was based on DEC's
21164 which only had word (64 bit) memory access, and then arbitrary
byte shuffle/extraction operations to extract the byte you're
interested in. That used to throw a few bubbles in the pipeline, so
you didn't do it unless you absolutely needed it.

Not that I ever used mine (21164, not T3E) for anything
apart from FP stuff, so I'm not speaking from experience,
just hearsay.

Phil
--
"Religion is what keeps the poor from murdering the rich."
-- Napoleon

Phil Carmody <> writes:
> Keith Thompson <kst-> writes:
>> (The system was the Cray T90. The word size was 64 bits, and there was
>> no hardware support for byte addressing. Byte addresses were
>> implemented in software by storing an offset in the high-order 3 bits of
>> a 64-bit pointer.)
>
> I presume the Cray T3E is the similar, as that was based on DEC's
> 21164 which only had word (64 bit) memory access, and then arbitrary
> byte shuffle/extraction operations to extract the byte you're
> interested in. That used to throw a few bubbles in the pipeline, so
> you didn't do it unless you absolutely needed it.
>
> Not that I ever used mine (21164, not T3E) for anything
> apart from FP stuff, so I'm not speaking from experience,
> just hearsay.

No, it wasn't (I worked on both). The T3E used the DEC Alpha.
The C compiler had 8-bit char, 32-bit short, and 64-bit int and long.
It may have required extra work to access bytes (I never looked into
it that deeply, but machine addresses pointed to bytes, not words,
so there was no special format for void* vs. int*.

--
Keith Thompson (The_Other_Keith) kst- <http://www.ghoti.net/~kst>
"We must do something. This is something. Therefore, we must do this."
-- Antony Jay and Jonathan Lynn, "Yes Minister"

On Fri, 16 Sep 2011 00:34:20 +0300, Phil Carmody wrote:

>> > ... if you really think you need "pack(1)". Usually (not always,
>> >but usually) that's the sign of a short-term hack leading to long-
>> >term headaches.
>>
>> Not when you are dealing with communication protocols, mapping
>> hardware control registers in embedded systems, etc.
>
> Packing doesn't address endianness. You have lost already.

It doesn't matter for the "hardware control registers" case, as these are
inherently in the native byte order.

It doesn't always matter for the case of communication protocols; some
protocols offer a choice of byte order. Similarly for file formats.

For floating-point values, reading bytes directly into a C float/double
then optionally byte-swapping is rather common. If your platform doesn't
use IEEE-754 representation, you lose.

Even if portability is important enough to justify explicit
(de)serialisation code, performance is often important enough to justify:

#if FILE_FORMAT_IS_NATIVE_FORMAT
return fwrite(&obj, sizeof(obj), 1, fp);
#else
// explicit seralisation
#endif