«

Hi,

with reference to [0] and [1], please consider the following:

1 #include <string.h>
2 #include <stdlib.h>
3
4 struct x {
5 double *y;
6 };
7
8 int
9 main(void)
10 {
11 struct x *x = malloc(sizeof *x);
12
13 /* suppose the allocation succeeds */
14 (void)memset(x, 0, sizeof *x);
15 (void)(0 == x->y);
16 return 0;
17 }

In my understanding, the evaluation of x->y on line 15 is undefined
behavior in C99.

Consider the following (fictious) extension:

"The all-bits-zero object representation is valid for any
pointer-to-object type. Any pointer-to-object object with the
all-bits-zero representation is a null pointer of the corresponding
pointer-to-object type."

Would this extension make the above program well defined?

In particular, would the following program still break aliasing rules, as
per C99 6.5 p 6-7?

1 #include <stdlib.h>
2
3 int
4 main(void)
5 {
6 double **d = malloc(sizeof *d);
7 size_t pos;
8
9 /* suppose the allocation succeeded */
10
11 for (pos = 0u; pos < sizeof *d; ++pos) {
12 ((char unsigned *)d)[pos] = 0u;
13 }
14
15 (void)*d;
16 return 0;
17 }

(I hope my question corresponds precisely to the austin-group-l topic.)

Thank you very much,
lacos


[0] https://www.opengroup.org/sophocles/...oup-l&id=13687
[1] https://www.opengroup.org/sophocles/...oup-l&id=13690

Ersek, Laszlo wrote:
> Hi,
>
> with reference to [0] and [1], please consider the following:
>
> 1 #include <string.h>
> 2 #include <stdlib.h>
> 3
> 4 struct x {
> 5 double *y;
> 6 };
> 7
> 8 int
> 9 main(void)
> 10 {
> 11 struct x *x = malloc(sizeof *x);
> 12
> 13 /* suppose the allocation succeeds */
> 14 (void)memset(x, 0, sizeof *x);
> 15 (void)(0 == x->y);
> 16 return 0;
> 17 }
>
> In my understanding, the evaluation of x->y on line 15 is undefined
> behavior in C99.
>
> Consider the following (fictious) extension:
>
> "The all-bits-zero object representation is valid for any
> pointer-to-object type. Any pointer-to-object object with the
> all-bits-zero representation is a null pointer of the corresponding
> pointer-to-object type."

That depends upon what you mean by valid. The standard distinguishes
several cases. It talks about pointer objects containing representations
of which can be dereferenced, incremented, decremented, compared for
order, compared for equality, or simply copied as a pointer value. For
each of those operations, the set of pointer representations valid for
that operation is different.

A null pointer value must compare equal to any other null pointer value,
and it must compare unequal to any pointer to an object. A pointer is
not valid for the purpose of dereferencing it, unless it points at an
object. Therefore, in principle, an implementation cannot choose
all-bits-0 to be both a null pointer and a pointer which is valid for
the purpose of dereferencing it. However, offhand I can't come up with
any code with defined behavior which demonstrates the non-conformance of
such an implementation, so it might be permitted, under the "as-if" rule.

> Would this extension make the above program well defined?

Yes. The behavior would be well-defined, by the implementor. The
behavior would, of course, still be undefined as far as the C standard
is concerned, because "undefined behavior" is a specialized piece of
jargon in the C standard. It doesn't carry the apparently obvious
meaning of "behavior that has no definition". Instead, it means
"behavior which is not defined by this standard" (I've paraphrased the
actual wording, for the sake improved clarity in this context).

> In particular, would the following program still break aliasing rules,
> as per C99 6.5 p 6-7?
>
> 1 #include <stdlib.h>
> 2
> 3 int
> 4 main(void)
> 5 {
> 6 double **d = malloc(sizeof *d);
> 7 size_t pos;
> 8
> 9 /* suppose the allocation succeeded */

It's almost as easy to handle the possibility that the allocation
failed, as it is to write a comment explaining that you've decided to
ignore that possibility.

> 11 for (pos = 0u; pos < sizeof *d; ++pos) {
> 12 ((char unsigned *)d)[pos] = 0u;
> 13 }
> 14
> 15 (void)*d;

> 16 return 0;
> 17 }
>
> (I hope my question corresponds precisely to the austin-group-l topic.)

I'm not sure that it does. Aliasing is something that is inherently
impossible for null pointers, for they do not point at an object.

On Mon, 26 Apr 2010, James Kuyper wrote:

> Ersek, Laszlo wrote:
>> Hi,
>>
>> with reference to [0] and [1], please consider the following:
>>
>> 1 #include <string.h>
>> 2 #include <stdlib.h>
>> 3
>> 4 struct x {
>> 5 double *y;
>> 6 };
>> 7
>> 8 int
>> 9 main(void)
>> 10 {
>> 11 struct x *x = malloc(sizeof *x);
>> 12
>> 13 /* suppose the allocation succeeds */
>> 14 (void)memset(x, 0, sizeof *x);
>> 15 (void)(0 == x->y);
>> 16 return 0;
>> 17 }
>>
>> In my understanding, the evaluation of x->y on line 15 is undefined
>> behavior in C99.
>>
>> Consider the following (fictious) extension:
>>
>> "The all-bits-zero object representation is valid for any
>> pointer-to-object type. Any pointer-to-object object with the
>> all-bits-zero representation is a null pointer of the corresponding
>> pointer-to-object type."
>
> That depends upon what you mean by valid.

My apologies. I tried (and failed) to formulate "all-bits-zero implies a
null pointer value" in standardese. Thus I meant all those uses that are
otherwise valid for any given lvalue evaluating to a null pointer value.


> The standard distinguishes several cases. It talks about pointer objects
> containing representations of which can be dereferenced, incremented,
> decremented, compared for order, compared for equality, or simply copied
> as a pointer value. For each of those operations, the set of pointer
> representations valid for that operation is different.
>
> A null pointer value must compare equal to any other null pointer value,
> and it must compare unequal to any pointer to an object. A pointer is
> not valid for the purpose of dereferencing it, unless it points at an
> object. Therefore, in principle, an implementation cannot choose
> all-bits-0 to be both a null pointer and a pointer which is valid for
> the purpose of dereferencing it. However, offhand I can't come up with
> any code with defined behavior which demonstrates the non-conformance of
> such an implementation, so it might be permitted, under the "as-if"
> rule.
>
>> Would this extension make the above program well defined?
>
> Yes. The behavior would be well-defined, by the implementor. The
> behavior would, of course, still be undefined as far as the C standard
> is concerned, because "undefined behavior" is a specialized piece of
> jargon in the C standard. It doesn't carry the apparently obvious
> meaning of "behavior that has no definition". Instead, it means
> "behavior which is not defined by this standard" (I've paraphrased the
> actual wording, for the sake improved clarity in this context).

Thank you.

As I understand it, the question is: after adding this extension to
POSIX(R), would further extensions be necessary, so that the code above
becomes defined? Most probably, this could be answered completely only by
considering all other extensions introduced by POSIX. Assuming, however,
that POSIX only defined otherwise undefined (or unspecified) behavior, and
that it didn't redefine (or weaken/reclassify) already defined behavior, I
think the suggestion ought to be eligible for consideration in isolation
as well.


>> In particular, would the following program still break aliasing rules, as
>> per C99 6.5 p 6-7?
>>
>> 1 #include <stdlib.h>
>> 2
>> 3 int
>> 4 main(void)
>> 5 {
>> 6 double **d = malloc(sizeof *d);
>> 7 size_t pos;
>> 8
>> 9 /* suppose the allocation succeeded */
>
> It's almost as easy to handle the possibility that the allocation failed, as
> it is to write a comment explaining that you've decided to ignore that
> possibility.

I agree absolutely. I had to force myself to omit the error checking and
write a comment instead. I favor examples with complete error checking. I
only wanted to sidestep dead-ends like "if malloc() fails, there is no
undefined behavior, because the first substatement of your *if* statement
is not executed then".


>
>> 11 for (pos = 0u; pos < sizeof *d; ++pos) {
>> 12 ((char unsigned *)d)[pos] = 0u;
>> 13 }
>> 14
>> 15 (void)*d;
>
>> 16 return 0;
>> 17 }
>>
>> (I hope my question corresponds precisely to the austin-group-l topic.)
>
> I'm not sure that it does. Aliasing is something that is inherently
> impossible for null pointers, for they do not point at an object.

I believe it isn't about an object hypothetically aliased by some other
(valid) pointer and a pointer with all-bits-zero representation. It is
about the pointer object with all-bits-zero representation itself, aliased
by differently typed pointers (pointer rvalues); like *d vs. *(char
unsigned *)d in the above.

I think that storing a valid (double*)0 null pointer value representation
in the space allocated by malloc() through either ((char unsigned
*)d)[...] or memset() doesn't force any effective type (eg. a character
type) on the allocated object. That is, the conclusion of the second
sentence of C99 6.5 p6 does not hold, because its premise is false.

--o--

I've downloaded ISO/IEC 9899:1999/Cor.2:2004(E) from
<http://www.open-std.org/jtc1/sc22/wg14/www/docs/9899-1999_cor_2-2004.pdf>.
Entry 9 seems to imply that

{
int *ip;

ip = malloc(sizeof *ip);
if (0 != ip) {
(void)memset(ip, 0, sizeof *ip);
(void)*ip;
}
}

can invoke no undefined behavior, even though TC2 doesn't appear to extend
C99 6.5 with any requirement on memset(). Reformulating the original
question: extending C99 (with all TC's applied) with a requirement on
pointers-to-objects, similar to TC2 entry 9, would the code at the top
*instantly* become defined?


Thank you very much for your answer.
lacos

Ersek, Laszlo wrote:
> On Mon, 26 Apr 2010, James Kuyper wrote:
>
>> Ersek, Laszlo wrote:
....
>>> 11 for (pos = 0u; pos < sizeof *d; ++pos) {
>>> 12 ((char unsigned *)d)[pos] = 0u;
>>> 13 }
>>> 14
>>> 15 (void)*d;
>>
>>> 16 return 0;
>>> 17 }
>>>
>>> (I hope my question corresponds precisely to the austin-group-l topic.)
>>
>> I'm not sure that it does. Aliasing is something that is inherently
>> impossible for null pointers, for they do not point at an object.
>
> I believe it isn't about an object hypothetically aliased by some other
> (valid) pointer and a pointer with all-bits-zero representation. It is
> about the pointer object with all-bits-zero representation itself,
> aliased by differently typed pointers (pointer rvalues); like *d vs.
> *(char unsigned *)d in the above.

The anti-aliasing rules in 6.5p7 explicitly allow for the representation
of any object to be accessed through an lvalue of character type, in
addition to several other possibilities.

> I think that storing a valid (double*)0 null pointer value
> representation in the space allocated by malloc() through either ((char
> unsigned *)d)[...] or memset() doesn't force any effective type (eg. a
> character type) on the allocated object. That is, the conclusion of the
> second sentence of C99 6.5 p6 does not hold, because its premise is false.

Sorry, I didn't realize that it was 'd' itself, rather than *d, that you
were thinking about in terms of aliasing.

C99 describes how the effective type is determined, and I agree that
neither the call to memset() nor writing to that memory as an array of
char gives it an effective type, since that writing did not take the
form of copying from an existing object. However, accessing it through
*d gives it an effective type of 'double*'; and 6.5p6 makes no
distinction between whether the access was for the purpose of writing
the memory, or reading it. If that memory has all bits zero at the time
of the access, then an POSIX-specific promise that such a representation
represents a null pointer seems to me to be sufficient to render the
behavior of that code defined - by POSIX, not by the C standard.

This is really an issue that you should raise in a forum devoted to the
POSIX standard, though I'm not sure what the appropriate one would be.
comp.std.unix has been quiet for so long that someone is starting the
formal process for removing it.

On Mon, 26 Apr 2010, James Kuyper wrote:

> Ersek, Laszlo wrote:
>> On Mon, 26 Apr 2010, James Kuyper wrote:
>>
>>> Ersek, Laszlo wrote:
> ...
>>>> 11 for (pos = 0u; pos < sizeof *d; ++pos) {
>>>> 12 ((char unsigned *)d)[pos] = 0u;
>>>> 13 }
>>>> 14
>>>> 15 (void)*d;
>>>
>>>> 16 return 0;
>>>> 17 }
>>>>

[snip]

>> I think that storing a valid (double*)0 null pointer value
>> representation in the space allocated by malloc() through either ((char
>> unsigned *)d)[...] or memset() doesn't force any effective type (eg. a
>> character type) on the allocated object. That is, the conclusion of the
>> second sentence of C99 6.5 p6 does not hold, because its premise is
>> false.

[snip]

> C99 describes how the effective type is determined, and I agree that
> neither the call to memset() nor writing to that memory as an array of
> char gives it an effective type, since that writing did not take the
> form of copying from an existing object. However, accessing it through
> *d gives it an effective type of 'double*'; and 6.5p6 makes no
> distinction between whether the access was for the purpose of writing
> the memory, or reading it. If that memory has all bits zero at the time
> of the access, then an POSIX-specific promise that such a representation
> represents a null pointer seems to me to be sufficient to render the
> behavior of that code defined - by POSIX, not by the C standard.

Thank you very much for your invaluable input. Much obliged.


> This is really an issue that you should raise in a forum devoted to the
> POSIX standard, though I'm not sure what the appropriate one would be.

As I understand it, austin-group-l is *that* forum. (See [0]/Q0, and [1].)
The issue was raised there and I thought comp.lang.c and comp.std.c
subscribers could contribute authoritatively. Thankfully, you proved the
hunch right.

If you don't mind, I'll forward your message to the corresponding
austin-group-l thread.

Cheers,
lacos

[0] http://www.opengroup.org/austin/faq.html
[1] http://www.opengroup.org/austin/lists.html

On Tue, 27 Apr 2010, Ersek, Laszlo wrote:

> The issue was raised there and I thought comp.lang.c and comp.std.c
> subscribers could contribute authoritatively.

Small fix, with apologies: the comp.std.c idea came from Vincent Lefevre.

Thanks,
lacos

In comp.std.c, article <hr5dd0$8o3$>,
James Kuyper <> wrote:

> C99 describes how the effective type is determined, and I agree that
> neither the call to memset() nor writing to that memory as an array of
> char gives it an effective type, since that writing did not take the
> form of copying from an existing object.

There's a problem with this sentence ("since ... not" while
the C standard uses positive causality - see below). Here's
what I said in the austin-group mailing-list about memset()
used on a dynamically allocated region (but again, the
standard is not clear enough, IMHO):

6.5#6 says:

The effective type of an object for an access to its stored value is
the declared type of the object, if any.75)

Here there is no declared type (I recall the context: the memory
was allocated dynamically). So, this doesn't apply.

If a value is stored into an object having no declared type through
an lvalue having a type that is not a character type, then the type
of the lvalue becomes the effective type of the object for that
access and for subsequent accesses that do not modify the stored
value.

Here the type of the lvalue is a character type, so that this doesn't
apply. Another interpretation is that memset is its own way to store
data (just like memcpy and memmove below); still, the above sentence
doesn't apply here.

If a value is copied into an object having no declared type using
memcpy or memmove, or is copied as an array of character type, then
the effective type of the modified object for that access and for
subsequent accesses that do not modify the value is the effective
type of the object from which the value is copied, if it has one.

Here this is memset, not memcpy or memmove. I don't know what "is
copied as an array of character type" intends to mean. Anyway,
memset doesn't copy an object. So, this doesn't apply.

For all other accesses to an object having no declared type, the
effective type of the object is simply the type of the lvalue used
for the access.

This is an "else" case. This is how I deduce that the effective type
is a character type.

--
Vincent Lefèvre <> - Web: <http://www.vinc17.net/>
100% accessible validated (X)HTML - Blog: <http://www.vinc17.net/blog/>
Work: CR INRIA - computer arithmetic / Arénaire project (LIP, ENS-Lyon)

In comp.std.c, article < lte.hu>,
Ersek, Laszlo <> wrote:

> On Tue, 27 Apr 2010, Ersek, Laszlo wrote:

> > The issue was raised there and I thought comp.lang.c and comp.std.c
> > subscribers could contribute authoritatively.

> Small fix, with apologies: the comp.std.c idea came from Vincent Lefevre.

Actually, my remark in the austin-group list was just about the
effective type due to a memset() on a dynamically allocated region
(not about the representation of a null pointer). This question is
covered by the C standard, not by POSIX. That's why I suggested
comp.std.c.

--
Vincent Lefèvre <> - Web: <http://www.vinc17.net/>
100% accessible validated (X)HTML - Blog: <http://www.vinc17.net/blog/>
Work: CR INRIA - computer arithmetic / Arénaire project (LIP, ENS-Lyon)

On Tue, 27 Apr 2010, Vincent Lefevre wrote:

> Here's what I said in the austin-group mailing-list about memset() used
> on a dynamically allocated region (but again, the standard is not clear
> enough, IMHO):
>
> 6.5#6 says:


(a)

> The effective type of an object for an access to its stored value is
> the declared type of the object, if any.75)
>
> Here there is no declared type (I recall the context: the memory was
> allocated dynamically). So, this doesn't apply.


(b)

> If a value is stored into an object having no declared type through
> an lvalue having a type that is not a character type, then the type
> of the lvalue becomes the effective type of the object for that
> access and for subsequent accesses that do not modify the stored
> value.
>
> Here the type of the lvalue is a character type, so that this doesn't
> apply. Another interpretation is that memset is its own way to store
> data (just like memcpy and memmove below); still, the above sentence
> doesn't apply here.


(c)

> If a value is copied into an object having no declared type using
> memcpy or memmove, or is copied as an array of character type, then
> the effective type of the modified object for that access and for
> subsequent accesses that do not modify the value is the effective
> type of the object from which the value is copied, if it has one.
>
> Here this is memset, not memcpy or memmove. I don't know what "is copied
> as an array of character type" intends to mean. Anyway, memset doesn't
> copy an object. So, this doesn't apply.


(d)

> For all other accesses to an object having no declared type, the
> effective type of the object is simply the type of the lvalue used
> for the access.
>
> This is an "else" case. This is how I deduce that the effective type is
> a character type.

Ah, okay, now I think I see what you mean. Sorry for being dense.

We seem to agree that none of (a) and (b) apply. You say that (c) doesn't
apply either, and thus (d) -- the "else branch" -- must apply. I didn't
understand this previously: I said (or rather, I think I said) "since none
of a-b-c applies, the access establishes no effective type at all". This
is probably a misinterpretation. (Euphemism for "I was wrong".)

But what if we assume for a moment that the all-bits-zero representation
carries a valid null pointer value for all pointer-to-object types? In
that case, wouldn't zeroing out the individual bytes of a
pointer-to-object object through a (char unsigned *) make (c) applicable?

----v----
If a value is copied into an object having no declared type [...] as an
array of character type, then the effective type of the modified object
for that access and for subsequent accesses that do not modify the value
is the effective type of the object from which the value is copied, if it
has one.
----^----

static double *dp; /* suppose all-bits-zero */
static char unsigned zeroes[sizeof dp];

static void
z1(void **dpp)
{
(void)memcpy(dpp, &dp, sizeof dp);
}

static void
z2(void **dpp)
{
size_t pos;

assert(0 == memcmp(&dp, zeroes, sizeof zeroes));
for (pos = 0u; pos < sizeof zeroes; ++pos) {
((char unsigned *)dpp)[pos] = zeroes[pos];
}
}

static void
z3(void **dpp)
{
(void)memset(dpp, 0, sizeof(double *));
}


(c) applies to z1(). z2() copies the exact same bit pattern (object
representation) from a character array to "*dpp". z3() establishes the
exact same bit pattern (object representation) in "*dpp" without a source
object.

Insomuch as TC2 entry 9 has rendered z2() and z3() equivalent to z1() wrt.
integers, without touching 6.5 at all, I think it would only be consequent
if a similar all-bits-zero requirement on object pointers (added as an
extension) made both z2() and z3() equivalent to z1(), wrt. object
pointers, necessitating no change to 6.5 either.

In my opinion, the reason why the standard doesn't explicitly include
memset() in (c), and the char-wise storing of a pattern, is only because
it couldn't do that without restricting the object representations
themselves. Adding a constraint on object representation sufficed to allow
z3() for integers. So should it for pointers-to-objects.

Cheers,
lacos

In comp.std.c, article <. elte.hu>,
Ersek, Laszlo <> wrote:

> But what if we assume for a moment that the all-bits-zero representation
> carries a valid null pointer value for all pointer-to-object types? In
> that case, wouldn't zeroing out the individual bytes of a
> pointer-to-object object through a (char unsigned *) make (c) applicable?

It depends on how this is done. You could apply (c) under "If a value
[...] is copied as an array of character type". But note the words
"value" and "copied". This means that there is a source in memory
(with an effective type). While memcpy and memmove use such a source,
memset doesn't. I'm not such that even a "for" loop falls under this
condition because I don't see how an implementation could recognize
every form of such loops (in the most complicated cases).

> ----v----
> If a value is copied into an object having no declared type [...] as an
> array of character type, then the effective type of the modified object
> for that access and for subsequent accesses that do not modify the value
> is the effective type of the object from which the value is copied, if it
> has one.
> ----^----

> static double *dp; /* suppose all-bits-zero */
> static char unsigned zeroes[sizeof dp];

> static void
> z1(void **dpp)
> {
> (void)memcpy(dpp, &dp, sizeof dp);
> }

> static void
> z2(void **dpp)
> {
> size_t pos;

> assert(0 == memcmp(&dp, zeroes, sizeof zeroes));
> for (pos = 0u; pos < sizeof zeroes; ++pos) {
> ((char unsigned *)dpp)[pos] = zeroes[pos];
> }
> }

> static void
> z3(void **dpp)
> {
> (void)memset(dpp, 0, sizeof(double *));
> }


> (c) applies to z1().

and the effective type of the object in &dp (that is, double *)
is used.

> z2() copies the exact same bit pattern (object representation) from
> a character array to "*dpp".

But the object in zeroes has no effective type (except the individual
unsigned char), thus no value. You first need to force an effective
type (and a value), e.g. with

*((double **) &zeroes) = NULL;

Then I'm not sure that the for loop counts as a copy of such an
object.

> z3() establishes the exact same bit pattern (object representation)
> in "*dpp" without a source object.

Since there is no source, there is no effective type and no value.

> Insomuch as TC2 entry 9 has rendered z2() and z3() equivalent to z1() wrt.
> integers, without touching 6.5 at all,

??? Could you explain? I don't see such a thing on

http://www.open-std.org/jtc1/sc22/wg14/www/docs/tc2.htm

--
Vincent Lefèvre <> - Web: <http://www.vinc17.net/>
100% accessible validated (X)HTML - Blog: <http://www.vinc17.net/blog/>
Work: CR INRIA - computer arithmetic / Arénaire project (LIP, ENS-Lyon)