"Ersek, Laszlo" <> writes:
> On Wed, 14 Jul 2010, Tim Rentsch wrote:
>
>> Franck Bui-Huu <> writes:
>>
>>> "Ersek, Laszlo" <> writes:
>>>
>>>>>
>>>>> struct object *the_obj;
>>>>> the_obj = avltree_container_of(the_node, struct object , node);
>>>>> assert(the_obj->key == 4);
>>>>
>>>> Therein lies my problem:
>>>>
>>>> #define avltree_container_of(node, type, member) ({ \
>>>> const struct avltree_node *__mptr = (node); \
>>>> (type *)( (char *)__mptr - offsetof(type,member) );})
>>>>
>
> [snip]
>
>>>> Second (my main concern), I suspect that this breaks strict
>>>> aliasing rules. That is, after the avltree_container_of()
>>>> function-like macro is expanded and the (statement-)expression is
>>>> evaluated, you end up with two pointers, namely "the_node" and
>>>> "the_obj". They point to storage that overlaps, and none of them
>>>> was computed from the other by following pointers and taking
>>>> addresses of members.
>>>>
>>>> That is, you have two pointers to different types, the pointed-to
>>>> areas overlap, and you obtained one of the pointers by
>>>> type-punning (when you cast the pointer-to-char to
>>>> pointer-to-type).
>
> [snip]
>
>>> Therefore,
>>>
>>> (char *)node - offsetof(...)
>>>
>>> results to the address of the beginning of the structure (whose
>>> type is 'struct my_struct) containing the member 'my_node' pointed
>>> by 'node'.
>>>
>>> So by definition we can assume that this address is correctly
>>> aligned for accessing an object of type 'struct
>>> my_struct'. Therefore we can safely convert back this address to
>>> (struct my_struct *).
>>>
>>> And since we know that the effective type of the underlying object
>>> is 'struct my_struct', it's safe to deference the converted
>>> pointer.
>>
>> Right.
>>
>>> So to sum up, I think:
>>>
>>> struct my_struct obj;
>>> char *p = (char *)&obj;
>>> p += offsetof(struct my_struct, my_node);
>>> p -= offsetof(struct my_struct, my_node);
>>> ((struct my_struct *)p)->a;
>>>
>>> is a defined case of type punning.
>>
>> Actually there isn't any type punning going on here, since the types
>> used to reference objects are the same in each case that the
>> respective object actually holds. (And ignoring that member 'a'
>> hasn't been initialized, which is orthogonal to the question of type
>> punning.) Type punning refers to the case when a region of memory
>> has been stored (or perhaps declared) using one type, then accessed
>> using another. That hasn't happened here. The pointer conversions
>> are legal and portable, but the converted pointers are accessing
>> objects whose types match those of the access in each case -- ergo,
>> no type punning.
>
> I probably misused the term "type punning" (see above what I meant --
> nothing more than manipulating a pointer-to-char and then casting it
> to a pointer-to-struct).
Yes, I realized the confusion and hoped my comments might
clear that up.
> My real problem was, again, that the compiler sees two pointers to
> different types, and the areas occupied by the pointed-to objects
> overlap. I was worried whether this breaks "strict aliasing rules".
>
> Of course, it doesn't; the outer structure declaration ("struct
> object_type") is visible, so if the compiler sees a (struct
> object_type *) and a (struct node_type *), it cannot assume the latter
> doesn't point into the target of the former -- unless "restrict" is in
> effect.
>
>
> I attribute my mistake to two specific "shocks" I suffered earlier:
> (1) the struct hack is UB in C89, and (2) if "arr" is a two-by-two
> "matrix", you can't write "arr[0][3]". These presumably drove me to
> the overcautious extreme.
>
> I have to admit, I still have difficulty determining what constitutes
> a promise to the compiler. Consider:
>
> a) It was derived by multiple illustrious contributors here and in
> c.l.c.m that multi-dimensional arrays can't have holes at all (or so I
> remember). It had to be derived because the Standard doesn't
> explicitly say so (I think). Why oh why can't you write arr[0][3]
> then, when the memory *must* be there and the access is correctly
> aligned? Well, because you made a promise to the compiler wrt. to the
> range of the last subscript, and the addressing instruction it
> generates may be invalid.
>
> Okay, then write "*(&arr[0][0] + 3)". Still undefined, because...
>
> Or, instead of over-indexing a struck hack array in C89, just write
>
> *(el_type *)((char unsigned *)hack_arr + n * sizeof(el_type))
>
> Still undefined, because...
>
> b) Contrast: hacking on a char pointer, casting it and dereferencing
> it is okay, because "we know it is correctly aligned, and the
> pointed-to storage was correctly allocated and written to earlier".
>
> I simply can't put my finger on the distinctive feature between these two.
I find it useful to read the Standard in two distinct modes.
One, what I might call the "comp.std.c mode", is a more literal
and less assuming reading (or interpretation, some might say).
The other, what I might call the "comp.lang.c mode", is less
literal and allows more reading between the lines, in an effort
to discover what the committee expects for how the words will
be read. Mostly these two modes produce the same conclusions,
but there are some obvious counterexamples, and I think pointer
manipulation and conversion qualifies as a candidate here.
Reading in the comp.lang.c mode, indexing an array past its
(last dimension) index limit is wrong, because the compiler
"knows" (more accurately, "is allowed to know") what that
upper bound is. So that explains the 'arr[0][3]' example.
Furthermore, because of how indexing works in C (because of
how arrays convert to pointer-to-first-element) the expression
'*(&arr[0][0]+3)' isn't any different than '*(&arr[0][3])',
which obviously should be the same as 'arr[0][3]'.
The third example (with 'hack_arr') is a little tricker, because
casting a pointer to any kind of character pointer carries a certain
amount of "magic" in the C language, so this is closer to being
allowed. There's a different problem though, which is storing into
any structure member is allowed to mess up any padding bytes, so the
(presumably last) member 'hack_arr' might be messed up by that in
its upper bytes. I think a reasonable argument could be made that
if the size of 'hack_arr' plus its offset in the structure equals
the size of the whole structure (ie, so there are no trailing
padding bytes), then accessing 'hack_arr' in the way you describe
can be expected to work (again reading in the comp.lang.c mode, and
even so I admit it's a gray area).
The difference with converting to a structure pointer is (a) we
know (again, in comp.lang.c mode) that we have to be able to do
pointer arithmetic on a character pointer to get from a
pointer-to-member to pointer-to-structure, because otherwise
what's the point of 'offsetof()', and (b) when we convert a
pointer to a pointer-to-structure, whatever the compiler used to
"know" about the size of the pointed-to object, now it "knows"
something different, ie, the size of the structure, because that
information is explicit in the definition of the structure type,
and C is supposed to "trust the programmer". The explicit
supplying of new size information is the key difference between
the two cases. At least, I think that's a reasonable conclusion,
reading in the comp.lang.c mode.
Some people who read in more of a comp.std.c mode might argue that
going from pointer-to-(nonfirst)member to pointer-to-structure is
always undefined behavior, and I think it might be possible to make
an argument that the Standard supports such a reading. However,
as far as comp.lang.c goes, any sort of suggestion in that direction
is merely evidence that the Standard needs to be revised and/or
clarified, because surely the committee expects that this kind
of character pointer arithmetic works and yields the expected
(and well-defined) results.