Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > C++ > Reading Struct not Located at Four-boundary

Reply
Thread Tools

Reading Struct not Located at Four-boundary

 
 
Alf P. Steinbach /Usenet
Guest
Posts: n/a
 
      08-20-2010
* Ian Collins, on 20.08.2010 22:59:
> On 08/21/10 04:04 AM, Francesco S. Carta wrote:
>>
>> OK, let's see if I got this straight. Since the size of those two
>> structs is the same, they will both fit correctly aligned in a space
>> allocated via "new char[sizeof(a)]" (or via "new char[sizeof(b)]", for
>> that matter).

>
> I think you are still missing the point that size and alignment are
> orthogonal.


I'd miss it too, since alignment requirement /is/ a function of size.

That goes down to the hardware level, it's not a matter of formalism.

Regarding the formalism, specific alignment for a given object can be specified,
by language extensions or in C++0x within the standard language; such a specific
alignment must conform to the alignment requirement for the type; the alignment
requirement for the type in turn must conform to the alignment requirement for
the type's size, which is what matter to the HW.


>> But "new short[sizeof(double) / sizeof(short)]", alone and by itself,
>> has to satisfy the alignment requirements of "short", regardless of the
>> size of the array itself.
>>
>> So in some sense, both the type and the size of that type do play an
>> important role in the alignment of the address returned by dynamic
>> allocators - although that role is implementation defined within the
>> limits mandated by the standard.

>
> No, the size is still irrelevant to the guarantee.


In theory yes the size is irrelevant to the new operator's guarantee, since it
guarantees to return a pointer suitably aligned for any item.

However, in practice there can be hardware support for items of larger size than
standard C++ supports, then with special alignment requirements.

E.g., for SIMD instructions there is a 16-byte alignment requirement, and most
likely operator new will use an 8-byte alignment (at best), not sufficient.


> Whether you are
> allocating a double or an array of sizeof(double) char or a single char,
> the alignment of the pointer will be the same. The pointer returned will
> always be appropriately aligned for objects of any type.



Cheers,

- Alf (picking nits)

--
blog at <url: http://alfps.wordpress.com>
 
Reply With Quote
 
 
 
 
Gennaro Prota
Guest
Posts: n/a
 
      08-20-2010
On 20/08/2010 18.47, Francesco S. Carta wrote:
> James Kanze <(E-Mail Removed)>, on 20/08/2010 09:37:15, wrote:
>
>> On Aug 20, 10:43 am, "Francesco S. Carta"<(E-Mail Removed)> wrote:
>>> thomas<(E-Mail Removed)>, on 20/08/2010 02:35:35, wrote:

>>
>>>> OK.. One thing I want to make sure is whether "new/malloc" will
>>>> guarantee that the address of the allocated memory will be aligned
>>>> properly?

>>
>>> I don't know about "malloc", but for "new" the answer is yes, it will be
>>> properly allocated for the type with which "new" is called.

>>
>> If he was comparing to malloc, he probably meant the operator
>> new function. Both malloc and the operator new function are
>> guaranteed to return memory suitably aligned for any object.

>
> Suitably aligned for any object that fits into the allocated space, as I
> seem to have understood, so far, from the other branches of this thread.
>
> In any case, I was surely overlooking (once more) the difference between
> using plain "new" and calling "the operator new function". How were they
> commonly referred to? "new expression" and "operator new"?


Use the terminology of the standard ("new expression" and
"operator new function")!

<asides>
- The reason for the excitement and the consequent exclamation
point is that I do use it, and I feel lonely.

- If you really want to refer to the operator (as opposed to
the expression) you have to think more.
</asides>

I seem to recall someone advocating the usage of "operator new"
for the function(s) and "new operator" for the operator but to
me... Perhaps it's crystal clear to a native speaker? (Anyway,
it may work in some contexts, especially when italics or
different fonts are used properly, and I've seen it used by
Dewhurst with extreme clarity. It's just not something to
advocate in general.)

I think one may also use "new allocation function", which is
unambiguous too.

PS: quite obviously, too, if one says for instance "write
operator delete if you write operator new", like our Scott
Meyers does, it's clear that he's not talking about the
operators Yet I'm picky enough to avoid such language anyway
(no, not avoiding C++, just such terminology ).

--
Gennaro Prota | I'm available for your projects.
Breeze (preview): <https://sourceforge.net/projects/breeze/>
 
Reply With Quote
 
 
 
 
Ian Collins
Guest
Posts: n/a
 
      08-21-2010
On 08/21/10 11:37 AM, Alf P. Steinbach /Usenet wrote:
> * Ian Collins, on 20.08.2010 22:59:
>> On 08/21/10 04:04 AM, Francesco S. Carta wrote:
>>>
>>> OK, let's see if I got this straight. Since the size of those two
>>> structs is the same, they will both fit correctly aligned in a space
>>> allocated via "new char[sizeof(a)]" (or via "new char[sizeof(b)]", for
>>> that matter).

>>
>> I think you are still missing the point that size and alignment are
>> orthogonal.

>
> I'd miss it too, since alignment requirement /is/ a function of size.


No, it isn't, at least not to the size we are debating, that is the size
passed to the allocation function.

> That goes down to the hardware level, it's not a matter of formalism.


Correct. A 32 bit machine may be able to perform efficient byte aligned
operation on any type, so it could have an alignment requirement of one
for any type. Another 32 bit machine may be incapable of byte
addressing and therefore have an alignment requirement of four for any type.

A lot of 32 bit machines have the same alignment requirement (four) for
32 bit float and 64 bit double (or 32 bit long and 64 bit long long).

The alignment used by allocator functions is typically bigger than that
required by the machine. All the hosted environment allocators I've
checked use 16 or 32.

>> No, the size is still irrelevant to the guarantee.

>
> In theory yes the size is irrelevant to the new operator's guarantee,
> since it guarantees to return a pointer suitably aligned for any item.


In practice too. The guarantee is just that, a guarantee.

It might be clearer to say the size /passed/ the the allocation function
is irrelevant. The alignment used is derived from the hardware, not run
time information.

> Cheers,
>
> - Alf (picking nits)


Putting some back!

Cheers,

--
Ian Collins
 
Reply With Quote
 
Gennaro Prota
Guest
Posts: n/a
 
      08-21-2010
On 20/08/2010 17.32, Pete Becker wrote:
[...]
>> The standard says, in [expr.new] 5.3.4p10, that char arrays do have
>> such restrictions, but it says that related to char arrays created
>> using "new", that would mean that "new char[sizeof(double)]" should
>> return an address properly aligned for any object of the same size of a
>> double.
>>
>> Are you saying that the requirements are different for automatic and
>> static char arrays as opposed to dynamically created ones?

>
> Sorry, I made a bit of a muddle here. The key point is that alignment
> is defined by the hardware; whatever the standard says about alignment
> is pretty much handwaving, to avoid putting unnecessary restrictions on
> implementations.
>
> Those words in [expr.new] try to put some logic into what typically
> hasn't been explicitly laid out in the past. The underlying principle
> is that the size of the requested allocation might eliminate some
> types, because they're too large to fit in the requested space; in that
> case, the alignment of the returned pointer doesn't have to be
> appropriate for those types, because the memory block is too small to
> hold them. For example, the result of calling operator new(4) doesn't
> have to be aligned appropriately for an 8-bit double, only for types
> that occupy four bits or less.


Actually this seems in contradiction with 3.7.3.1/2 and with the
note to 5.3.4/10 itself which says "aligned for objects of any
type". When will the standard writers employ the same principles
that hold for programming, such as DRY?

And I think, at this time of the night, the "difference between
the result of the new-expression and the address returned by the
allocation function" matter is just confusing me.

This stuff concretely affects what the programmer knows about
the pointer returned by an operator new function so, if after
some sleep, I still think that there's a contradiction I'll file
a DR (I don't file DRs for logical inconsistencies and such with
no concrete impact on real software. There are just too many of
those, IMHO, and the result is just wasting time, for the
committee and for language users, who would see other important
things delayed).

I'm particularly worried that someone just reads 3.7.3.1 and
assumes "aligned for everything" (which is correct: why should
he suspect that another section says otherwise?).

In this case, where should the DR go? Core or library? Both?
(I'm inclined to "core"... but this lays astride (well, more on
the core side of the things, as I see it)).

--
Gennaro Prota | I'm available for your projects.
Breeze (preview): <https://sourceforge.net/projects/breeze/>
 
Reply With Quote
 
Ian Collins
Guest
Posts: n/a
 
      08-21-2010
On 08/21/10 12:36 PM, Gennaro Prota wrote:
> On 20/08/2010 17.32, Pete Becker wrote:
>>
>> Those words in [expr.new] try to put some logic into what typically
>> hasn't been explicitly laid out in the past. The underlying principle
>> is that the size of the requested allocation might eliminate some
>> types, because they're too large to fit in the requested space; in that
>> case, the alignment of the returned pointer doesn't have to be
>> appropriate for those types, because the memory block is too small to
>> hold them. For example, the result of calling operator new(4) doesn't
>> have to be aligned appropriately for an 8-bit double, only for types
>> that occupy four bits or less.

>
> Actually this seems in contradiction with 3.7.3.1/2 and with the
> note to 5.3.4/10 itself which says "aligned for objects of any
> type".


I agree. 3.7.3.1 states that "The pointer returned shall be suitably
aligned so that it can be converted to a pointer of any complete object
type". The note in 5.3.4p10 is based on that.

The examples in 5.3.4p12 appears to contradict what Pete says:

5.3.4p12 [Example:

new T results in a call of operator new(sizeof(T)),

which follows on form the note in 5.3.4p10

[Note: Because allocation functions are assumed to
return pointers to storage that is appropriately aligned for objects of
any type, this constraint on array allocation overhead permits the
common idiom of allocating character arrays into which objects of other
types will later be placed. ]

Which implies to me that new(4) *does* have to be (or at least will be)
aligned appropriately for an 8-bit double, even though the space is too
small.

> This stuff concretely affects what the programmer knows about
> the pointer returned by an operator new function so, if after
> some sleep, I still think that there's a contradiction I'll file
> a DR (I don't file DRs for logical inconsistencies and such with
> no concrete impact on real software.


All the programmer has to know is the pointer is suitably aligned.

> I'm particularly worried that someone just reads 3.7.3.1 and
> assumes "aligned for everything" (which is correct: why should
> he suspect that another section says otherwise?).


Because none does?

--
Ian Collins
 
Reply With Quote
 
Francesco S. Carta
Guest
Posts: n/a
 
      08-21-2010
I am replying to nobody in particular, right now. It seems to me that
people more experienced than me have not yet found an agreement in this
very thread, so I'll stay out of the discussion because my intervention
will likely add confusion instead of dissipating it.

This is just an update about what I've worked out till now.

I have tracked down a misconception of mine that was influencing my
thought about all of this.

I've created a struct that occupies 64 bytes of storage, then I
dynamically allocated it and I checked its address to see if, as I was
expecting, it was a multiple of 64. As it turned out, it wasn't, showing
me that my assumption was stupid at best.

I have no idea about where I have taken that misconception from, I could
very well have made it up all by myself.

I still need some study to clarify the concepts used by the standard
about "operator new", "new operator", their array versions, the flow of
these operations and so on, so I won't speak any further about a subject
that isn't clear enough to me yet, I'll come back about this later to
ask further informations if reading the relative parts of the standard
will not suffice.

About the implementation-defined part of this discussion, that is, the
actual value of the alignments, finding a method for determining this
value will at least help me understand the implementation's behavior,
which in turn will eventually help me understanding the abstract
behavior mandated by the standard.

I've found a method in the page linked below, and the explanations given
make sense to me, any further reference or eventually a warning about
wrong informations given there (if any) will be more than welcome:

http://www.monkeyspeak.com/alignment/

--
FSC - http://userscripts.org/scripts/show/59948
http://fscode.altervista.org - http://sardinias.com
 
Reply With Quote
 
James Kanze
Guest
Posts: n/a
 
      08-21-2010
On Aug 20, 10:08 pm, Pete Becker <(E-Mail Removed)> wrote:
> On 2010-08-20 12:47:39 -0400, Francesco S. Carta said:


[...]
> If you're really into technicalities, since a class can override
> operator new, the actual code that the compiler generates for new T
> typically calls T's constructor with a null pointer, and T's
> constructor calls operator new.


I know that this is a common implementation, but is it formally
conform? I'm not sure that it is 100% clear, but by my reading,
the following program is conform:

class C
{
void* operator new(size_t);
public:
C();
};

int
main()
{
C aC;
}

With at least some compilers I've used, this (or something
similar---I don't have a compiler installed on this machine to
test) fails to link, because there is no definition of
C:perator new (and the compiler is using the implementation
you suggest).

A similar problem arises with delete.

--
James Kanze
 
Reply With Quote
 
James Kanze
Guest
Posts: n/a
 
      08-21-2010
On Aug 21, 12:37 am, "Alf P. Steinbach /Usenet" <alf.p.steinbach
(E-Mail Removed)> wrote:
> * Ian Collins, on 20.08.2010 22:59:


> > On 08/21/10 04:04 AM, Francesco S. Carta wrote:


> >> OK, let's see if I got this straight. Since the size of those two
> >> structs is the same, they will both fit correctly aligned in a space
> >> allocated via "new char[sizeof(a)]" (or via "new char[sizeof(b)]", for
> >> that matter).


> > I think you are still missing the point that size and alignment are
> > orthogonal.


> I'd miss it too, since alignment requirement /is/ a function of size.


Not "a function of size", but "constrained by size". The
alignment isn't orthogonal, since the constraint exists
(alignment cannot be greater than size), but it's not a function
of size either, since as long as the constraint is met, there's
no problem.

> That goes down to the hardware level, it's not a matter of formalism.


Except that Ian has already posted a contradicting example:
char[sizeof(double)] and double have the same size (required by
the standard), but don't have the same alignment requirements.

[...]
> Regarding the formalism, specific alignment for a given object
> In theory yes the size is irrelevant to the new operator's
> guarantee, since it guarantees to return a pointer suitably
> aligned for any item.


I'm not sure about this. I've heard two explinations (and I
can't remember which is right): a non-member allocation function
must return memory sufficiently aligned for any object, or it
must return memory sufficiently aligned for any object which
will fit.

--
James Kanze
 
Reply With Quote
 
Alf P. Steinbach /Usenet
Guest
Posts: n/a
 
      08-21-2010
* James Kanze, on 21.08.2010 17:14:
> On Aug 21, 12:37 am, "Alf P. Steinbach /Usenet"<alf.p.steinbach
> (E-Mail Removed)> wrote:
>> * Ian Collins, on 20.08.2010 22:59:

>
>>> On 08/21/10 04:04 AM, Francesco S. Carta wrote:

>
>>>> OK, let's see if I got this straight. Since the size of those two
>>>> structs is the same, they will both fit correctly aligned in a space
>>>> allocated via "new char[sizeof(a)]" (or via "new char[sizeof(b)]", for
>>>> that matter).

>
>>> I think you are still missing the point that size and alignment are
>>> orthogonal.

>
>> I'd miss it too, since alignment requirement /is/ a function of size.

>
> Not "a function of size", but "constrained by size". The
> alignment isn't orthogonal, since the constraint exists
> (alignment cannot be greater than size), but it's not a function
> of size either, since as long as the constraint is met, there's
> no problem.


Well it gets thorny in the details. For the general case the basic HW alignment
on a given system is a function of size: size -> alignment requirement. But then
you have things like special instructions for special kinds of data, with
special alignment requirements -- outside of the general case, and not very
well covered, if at all, by C++ rules. Yeah, I could've been more precise.


>> That goes down to the hardware level, it's not a matter of formalism.

>
> Except that Ian has already posted a contradicting example:
> char[sizeof(double)] and double have the same size (required by
> the standard), but don't have the same alignment requirements.


No it's not a contradiction. The data item size for char[N], what you can deal
with via assignment, is 1. Just a matter of picking the Right Size. <g>


> [...]
>> Regarding the formalism, specific alignment for a given object
>> In theory yes the size is irrelevant to the new operator's
>> guarantee, since it guarantees to return a pointer suitably
>> aligned for any item.

>
> I'm not sure about this. I've heard two explinations (and I
> can't remember which is right): a non-member allocation function
> must return memory sufficiently aligned for any object, or it
> must return memory sufficiently aligned for any object which
> will fit.


C++98 3.7.3.1/2 "The pointer returned shall be suitably aligned so that it can
be converted to a pointer of any complete object type and then used to access
the object or array in the storage allocated".

At the ultra-formal level I think this doesn't say anything, since "converted"
isn't defined (at least as far as I can see).

But from a mereley practical-formal point of view it says the address must be
suitably aligned for any complete object type.


Cheers,

- Alf


--
blog at <url: http://alfps.wordpress.com>
 
Reply With Quote
 
Larry Evans
Guest
Posts: n/a
 
      08-21-2010
On 08/21/10 06:30, Francesco S. Carta wrote:
[snip]
> I've created a struct that occupies 64 bytes of storage, then I
> dynamically allocated it and I checked its address to see if, as I was
> expecting, it was a multiple of 64. As it turned out, it wasn't, showing
> me that my assumption was stupid at best.
>
> I have no idea about where I have taken that misconception from, I could
> very well have made it up all by myself.

[snip]

Hi Francesco.

I struggled a few yrs (months?) ago about that and think I've
figured it out.

The alignment of your 64 byte long structure is dependent on the
types contained in the structure.

Using variadic template:

http://www.osl.iu.edu/~dgregor/cpp/v...templates.html

notation, augmented with:

template
< unsigned Index
, typename... T
>

struct at_c
{
typedef
//T...[I] the I-th type in T...
type
;
};

assume:

template
< typename... T
>

struct C
{
typename at_c<1,T...>::type m_1;
typename at_c<2,T...>::type m_2;
...
typename at_c<n,T...>::type m_n;
};

then, again

then alignof(C) is *only* a function of the alignments of
T... .

I'm pretty sure:

eq[1]: alignof(C) == fold(lcm,1,alignof(T)...)

where fold is the haskell foldl or foldr
which is basically the stl::accumulate.
alignof(T)... basically expands to:

alignof(T_1), alignof(T_2), ..., alignof(T_n)

NOTE, sizeof doesn't appear in eq[1]:.

If eq[1] is true, then the
offsets for all the members can
be easily calculated by assuring that
the offset of m_i in C is divisible by the
alignof(T_i). This is because, since the start of C is
at an address that is divisible by alignof(T_i)
(since alignof(C) is a multiple of alignof(T_i),
since alignof(C) was calculated as the lcm of all of
the alignof(T...)) , then m_i will be located at an address divisible by
aligndof(T_i).

All this is implemented with templates here:

http://svn.boost.org/svn/boost/sandb...osite_storage/

Now the one thing missing from this explanation is the role
of size. Now in order for all C's in:

C cvec[2];

to be properly aligned, the sizeof(C) must also be a multiple
of the alignof(C). Thus there may be some padding at the
end of C to achieve this. Likewise, in order to calculates
the offset of m_i to put m_i at the proper alignment, some
padding may also be needed. I'm pretty sure:

Sorry if this seems complicated. It is, at least to me

HTH.

-Larry
 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Can *common* struct-members of 2 different struct-types, that are thesame for the first common members, be accessed via pointer cast to either struct-type? John Reye C Programming 28 05-08-2012 12:24 AM
static struct initialization in a Class:: -- not my struct christian.bongiorno@gmail.com C++ 2 09-20-2006 06:53 PM
Printer not located in new laptop from wireless network =?Utf-8?B?dG1jZ2luNzEwMzk1?= Wireless Networking 3 04-19-2006 02:15 AM
struct my_struct *p = (struct my_struct *)malloc(sizeof(struct my_struct)); Chris Fogelklou C Programming 36 04-20-2004 08:27 AM
To coerce or not...? struct sockaddr vs struct sockaddr_in James Harris C Programming 4 10-09-2003 10:06 PM



Advertisments