Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > C Programming > realloc() implicit free() ?

Reply
Thread Tools

realloc() implicit free() ?

 
 
CBFalconer
Guest
Posts: n/a
 
      05-21-2005
Robert Gamble wrote:
> On Sat, 21 May 2005 16:38:51 +0200, Michael Mair wrote:
>> CBFalconer wrote:
>>> "S.Tobias" wrote:
>>>> CBFalconer <(E-Mail Removed)> wrote earlier:
>>>>
>>>>> In my nmalloc implementation for DJGPP I have chosen to treat
>>>>> a zero block size as a request for one byte, and let the
>>>>> alignment mechanisms raise it as they will.
>>>>
>>>> I'm not criticizing this, there're good reasons to do it like
>>>> that, but the Standard does not require the allocated object to
>>>> have alignment more than its size, right? Subsequent
>>>> `malloc(1)'-s could return ptrs that differ by 1 byte, right?
>>>> (I'm asking because the Std is a bit unclear there, IMHO.)
>>>
>>> No, that can't happen. Any non NULL return from malloc/calloc/
>>> realloc must be suitably aligned for any object whatsoever.

>>
>> Can you provide chapter and verse, please?

>
> 7.20.3p1 sentence 2:
>
> "The pointer returned if the allocation succeeds is suitably
> aligned so that it may be assigned to a pointer to any type
> of object and then usedto access such an object or an array
> of such objects in the space allocated (until the space is
> explicitly deallocated)."
>
>> This seems to be an unnecessary restriction -- why should malloc()
>> return 16-byte aligned memory for a 3-byte request if the
>> implementation "knows" that this implies a necessity for, say,
>> 2-byte alignment _at_most_?

>
> I'm not sure why, the Rationale doesn't give any hints here. It
> doesn't particularly make sense that the allocated space must be
> aligned to store an object that could not fit into the space,
> perhaps there is another reason we are overlooking or maybe I'm
> misinterpreting the above quote.


The malloc subsystem doesn't know, and can't know, what type the
pointer will be used for. It returns a void*, and the only
information it gets is the size required. That size may be for one
item, or for many items.

Similarly the application has no information as to the alignment
requirements for a type. The compiler system does. That is one of
the fundamental reasons that malloc is a system function, and
simply cannot be written in purely portable code.

--
Some informative links:
news:news.announce.newusers
http://www.geocities.com/nnqweb/
http://www.catb.org/~esr/faqs/smart-questions.html
http://www.caliburn.nl/topposting.html
http://www.netmeister.org/news/learn2quote.html


 
Reply With Quote
 
 
 
 
CBFalconer
Guest
Posts: n/a
 
      05-21-2005
Russell Shaw wrote:
> CBFalconer wrote:
>> "S.Tobias" wrote:
>>> CBFalconer <(E-Mail Removed)> wrote earlier:
>>>
>>>> In my nmalloc implementation for DJGPP I have chosen to treat a
>>>> zero block size as a request for one byte, and let the alignment
>>>> mechanisms raise it as they will.
>>>
>>> I'm not criticizing this, there're good reasons to do it like
>>> that, but the Standard does not require the allocated object to
>>> have alignment more than its size, right? Subsequent
>>> `malloc(1)'-s could return ptrs that differ by 1 byte, right?
>>> (I'm asking because the Std is a bit unclear there, IMHO.)

>>
>> No, that can't happen. Any non NULL return from
>> malloc/calloc/realloc must be suitably aligned for any object
>> whatsoever.

>
> I'm writing a malloc thing and compiling it in gcc on a pc.
> How do you know what the alignment should be?


No, and neither can you. Only the system implementor can know.
You can make some educated guesses in some cases, but you can never
find out in a portable manner.

--
Some informative links:
news:news.announce.newusers
http://www.geocities.com/nnqweb/
http://www.catb.org/~esr/faqs/smart-questions.html
http://www.caliburn.nl/topposting.html
http://www.netmeister.org/news/learn2quote.html


 
Reply With Quote
 
 
 
 
Michael Mair
Guest
Posts: n/a
 
      05-21-2005
CBFalconer wrote:
> Robert Gamble wrote:
>
>>On Sat, 21 May 2005 16:38:51 +0200, Michael Mair wrote:
>>
>>>CBFalconer wrote:
>>>
>>>>"S.Tobias" wrote:
>>>>
>>>>>CBFalconer <(E-Mail Removed)> wrote earlier:
>>>>>
>>>>>
>>>>>>In my nmalloc implementation for DJGPP I have chosen to treat
>>>>>>a zero block size as a request for one byte, and let the
>>>>>>alignment mechanisms raise it as they will.
>>>>>
>>>>>I'm not criticizing this, there're good reasons to do it like
>>>>>that, but the Standard does not require the allocated object to
>>>>>have alignment more than its size, right? Subsequent
>>>>>`malloc(1)'-s could return ptrs that differ by 1 byte, right?
>>>>>(I'm asking because the Std is a bit unclear there, IMHO.)
>>>>
>>>>No, that can't happen. Any non NULL return from malloc/calloc/
>>>>realloc must be suitably aligned for any object whatsoever.
>>>
>>>Can you provide chapter and verse, please?

>>
>>7.20.3p1 sentence 2:
>>
>>"The pointer returned if the allocation succeeds is suitably
>> aligned so that it may be assigned to a pointer to any type
>> of object and then usedto access such an object or an array
>> of such objects in the space allocated (until the space is
>> explicitly deallocated)."


Thanks, Rob

>>>This seems to be an unnecessary restriction -- why should malloc()
>>>return 16-byte aligned memory for a 3-byte request if the
>>>implementation "knows" that this implies a necessity for, say,
>>>2-byte alignment _at_most_?

>>
>>I'm not sure why, the Rationale doesn't give any hints here. It
>>doesn't particularly make sense that the allocated space must be
>>aligned to store an object that could not fit into the space,
>>perhaps there is another reason we are overlooking or maybe I'm
>>misinterpreting the above quote.

>
> The malloc subsystem doesn't know, and can't know, what type the
> pointer will be used for. It returns a void*, and the only
> information it gets is the size required. That size may be for one
> item, or for many items.
>
> Similarly the application has no information as to the alignment
> requirements for a type. The compiler system does. That is one of
> the fundamental reasons that malloc is a system function, and
> simply cannot be written in purely portable code.


Hmmm, I can accept this as an additional explanation but still do
not see the _necessity_ for the standard giving the cited
restriction.
Thank you.

Cheers
Michael
--
E-Mail: Mine is an /at/ gmx /dot/ de address.
 
Reply With Quote
 
Michael Mair
Guest
Posts: n/a
 
      05-21-2005
Chris Torek wrote:
>>CBFalconer wrote:
>>
>>>... Any non NULL return from malloc/calloc/realloc must be
>>>suitably aligned for any object whatsoever.

>
>
> In article <(E-Mail Removed)>
> Michael Mair <(E-Mail Removed)> wrote:
>
>>Can you provide chapter and verse, please?
>>This seems to be an unnecessary restriction -- why should malloc()
>>return 16-byte aligned memory for a 3-byte request if the
>>implementation "knows" that this implies a necessity for, say,
>>2-byte alignment _at_most_?

>
>
> Indeed, it does seem unnecessary. It also seems to be what the
> Standard requires. I suspect, however, that an implementor might
> be able to get away with the "as-if" rule and align to just 2 bytes
> in this case, provided the following code fragment still works:
>
> void *vp;
> char *cp;
> double *dp;
> /* add other types if desired */
>
> cp = malloc(3);
> if (cp == NULL) ... handle error ...
> vp = cp;
> dp = vp;
> assert(vp == cp);
> assert(dp == vp);
> cp = (char *)dp;
> assert(cp == vp);
> puts("all OK");
>
> In other words, if we have a system on which we can detect "failure
> to align for any object whatsoever" by just using ordinary pointer
> assignments, *then* that system will have to do 16-byte (or whatever)
> alignment for a 3-byte malloc; but if the system handles "misaligned"
> pointers without trouble, as long as they are not used to access
> their misaligned objects, then -- because there will be no way for
> the user to tell -- the as-if rule will allow us to implement
> malloc(3) with just two-byte alignment.


Thank you -- as usual, I can learn something from your response

Cheers
Michael
--
E-Mail: Mine is an /at/ gmx /dot/ de address.
 
Reply With Quote
 
Eric Sosman
Guest
Posts: n/a
 
      05-21-2005
Robert Gamble wrote:

> On Sat, 21 May 2005 16:38:51 +0200, Michael Mair wrote:
>
>
>>CBFalconer wrote:
>>
>>>"S.Tobias" wrote:
>>>
>>>
>>>>CBFalconer <(E-Mail Removed)> wrote earlier:
>>>>
>>>>
>>>>>In my nmalloc implementation for DJGPP I have chosen to treat a zero
>>>>>block size as a request for one byte, and let the alignment mechanisms
>>>>>raise it as they will.
>>>>
>>>>I'm not criticizing this, there're good reasons to do it like that, but
>>>>the Standard does not require the allocated object to have alignment
>>>>more than its size, right? Subsequent `malloc(1)'-s could return ptrs
>>>>that differ by 1 byte, right? (I'm asking because the Std is a bit
>>>>unclear there, IMHO.)
>>>
>>>No, that can't happen. Any non NULL return from malloc/calloc/realloc
>>>must be suitably aligned for any object whatsoever.

>>
>>Can you provide chapter and verse, please?

>
>
> 7.20.3p1 sentence 2:
>
> "The pointer returned if the allocation succeeds is suitably aligned so
> that it may be assigned to a pointer to any type of object and then used
> to access such an object or an array of such objects in the space
> allocated (until the space is explicitly deallocated)."


Taken in isolation, this sentence says that

struct s { char c[30000]; } x = { 0 };
struct s *p = malloc(1); /* assume success */
*p = x;

should work: The allocation has succeeded, the returned value
is therefore suitably aligned for any type (`struct s' in
particular), the value can be assigned to any type of pointer
(including `struct s*'), and can then be used to access an
object of that type. All the preconditions of the sentence
are met -- and yet, the access is obviously not permitted.

What this means is that snipping a sentence from the rest
of the Standard can distort its meaning. Elsewhere in the
Standard we can learn that `*p = x' produces undefined behavior,
so no conforming program can access a `struct s' through the
value returned by this particular malloc(). It follows that no
conforming program can actually encounter any alignment
problem by trying to access a `struct s' through this pointer;
the access itself is forbidden. Under the "as if" rule, if
nothing else, I believe malloc(1) can perfectly well return a
value that is not `struct s'-aligned because no conforming
program can ever detect the non-alignment.

However, the quoted sentence still places some restrictions
on what malloc(1) could return. For example,

void *p1 = malloc(1); /* assume success */
double *dp = p1;
void *p2 = dp;
assert (p2 == p1);

That is, converting the pointer to `double*' and back again
must not change its value. On machines where pointers to
different types have different representations, malloc(1) is
constrained to return values that can be converted to and from
any data pointer type without damage -- but as long as the
conversion is guaranteed to succeed, I don't think the value
need actually suffice for any data type that won't fit in the
allocated space.

--
Eric Sosman
http://www.velocityreviews.com/forums/(E-Mail Removed)lid

 
Reply With Quote
 
Chris Torek
Guest
Posts: n/a
 
      05-21-2005
In article <(E-Mail Removed)>
Russell Shaw <rjshawN_o@s_pam.netspace.net.au> wrote:
>I'm writing a malloc thing and compiling it in gcc on a pc.
>How do you know what the alignment should be?


Alas, ANSI/ISO C does not have any way to find out.

In the absence of the ability to write "comp.lang.c-conformant" code,
if you still need to get the job done, write non-conformant code.

For the various 4.xBSD systems, we defined a pair of macros (in
one of the <machine/*.h> headers), ALIGN(p) and ALIGNBYTES, that
do the trick.

The "machine" headers are different on each implementation, but
attempt to abstract out as much as possible. Thus, for instance,
<machine/frame.h> gives the stack frame for whatever implementation
you are using, always as a "struct frame" (and additional special
frames for user => kernel boundary crossings, in some cases). This
works fine as long as there is one consistent call-stack frame,
and you can write a simplified stack-trace-dump function that works
on (say) five completely different architectures (with some assistant
functions or macros), as long as they all use just the one "struct
frame". This eases code porting, but is not perfect: as soon as
you try to support (say) both 32- and 64-bit SPARC routines, you
start to need to distinguish between a 32-bit stack frame and a
64-bit stack frame ("struct frame32" and "struct frame64"), and
you must write a SPARC-specific "simplified stack-trace-dump"
module. Still, you get to share the original (single frame) dump
routine across those five machines, so that instead of six source
files to maintain (for six architectures), you have just two.

The idea is simple enough: make common code to solve common problems.
The trick is knowing what is common, what is not, and how to abstract
away irrelevant details without losing the important details.
--
In-Real-Life: Chris Torek, Wind River Systems
Salt Lake City, UT, USA (4039.22'N, 11150.29'W) +1 801 277 2603
email: forget about it http://web.torek.net/torek/index.html
Reading email is like searching for food in the garbage, thanks to spammers.
 
Reply With Quote
 
Robert Gamble
Guest
Posts: n/a
 
      05-21-2005
On Sat, 21 May 2005 13:33:23 -0400, Eric Sosman wrote:

> Robert Gamble wrote:
>
>> On Sat, 21 May 2005 16:38:51 +0200, Michael Mair wrote:
>>
>>
>>>CBFalconer wrote:
>>>
>>>>"S.Tobias" wrote:
>>>>
>>>>
>>>>>CBFalconer <(E-Mail Removed)> wrote earlier:
>>>>>
>>>>>
>>>>>>In my nmalloc implementation for DJGPP I have chosen to treat a zero
>>>>>>block size as a request for one byte, and let the alignment
>>>>>>mechanisms raise it as they will.
>>>>>
>>>>>I'm not criticizing this, there're good reasons to do it like that,
>>>>>but the Standard does not require the allocated object to have
>>>>>alignment more than its size, right? Subsequent `malloc(1)'-s could
>>>>>return ptrs that differ by 1 byte, right? (I'm asking because the
>>>>>Std is a bit unclear there, IMHO.)
>>>>
>>>>No, that can't happen. Any non NULL return from malloc/calloc/realloc
>>>>must be suitably aligned for any object whatsoever.
>>>
>>>Can you provide chapter and verse, please?

>>
>>
>> 7.20.3p1 sentence 2:
>>
>> "The pointer returned if the allocation succeeds is suitably aligned so
>> that it may be assigned to a pointer to any type of object and then
>> used to access such an object or an array of such objects in the space
>> allocated (until the space is explicitly deallocated)."

>
> Taken in isolation, this sentence says that
>
> struct s { char c[30000]; } x = { 0 };
> struct s *p = malloc(1); /* assume success */
> *p = x;
>
> should work:


No, it doesn't.

> The allocation has succeeded, the returned value is
> therefore suitably aligned for any type (`struct s' in particular), the
> value can be assigned to any type of pointer (including `struct s*'),
> and can then be used to access an object of that type. All the
> preconditions of the sentence are met -- and yet, the access is
> obviously not permitted.


The only thing being discussed here is alignmentment requirements. The
above quote states that your object created with malloc will be properly
*aligned* to hold any type, not that any type can actually be stored in it
if the space doesn't exist.

Rob Gamble
 
Reply With Quote
 
S.Tobias
Guest
Posts: n/a
 
      05-21-2005
Eric Sosman <(E-Mail Removed)> wrote:
> Robert Gamble wrote:
> > On Sat, 21 May 2005 16:38:51 +0200, Michael Mair wrote:
> >>CBFalconer wrote:


> >>>No, that can't happen. Any non NULL return from malloc/calloc/realloc
> >>>must be suitably aligned for any object whatsoever.
> >>
> >>Can you provide chapter and verse, please?

> >
> >
> > 7.20.3p1 sentence 2:
> >

(1)
> > "The pointer returned if the allocation succeeds is suitably aligned so
> > that it may be assigned to a pointer to any type of object

(2)
> >and then used
> > to access such an object or an array of such objects in the space
> > allocated (until the space is explicitly deallocated)."


> Taken in isolation, this sentence says that


> struct s { char c[30000]; } x = { 0 };
> struct s *p = malloc(1); /* assume success */
> *p = x;


> should work: The allocation has succeeded, the returned value
> is therefore suitably aligned for any type (`struct s' in
> particular), the value can be assigned to any type of pointer
> (including `struct s*'), and can then be used to access an
> object of that type. All the preconditions of the sentence
> are met -- and yet, the access is obviously not permitted.


> What this means is that snipping a sentence from the rest
> of the Standard can distort its meaning.

(3)
>Elsewhere in the
> Standard we can learn that `*p = x' produces undefined behavior,
> so no conforming program can access a `struct s' through the
> value returned by this particular malloc().


[ snipped implications which follow from (1) and (3),
but somehow (2) was lost, because it didn't fit into the theory;
rejected only 30% of facts - congratulations of a success! ]

So, under your interpretation, the Standard in one place *requires*
something that it forbids at another, and yet you're happy about it?
What do you people drink to keep you out of depression?


How about this: I say the Standard requires `malloc(1)' to return
at least enough space to accommodate `struct s'.
Stretched? Yes, but fits your assumptions better.

In another post I offered a different interpretation of the Standard,
which IMHO is more rational (I mean the implications are more rational,
not the interpretation itself), but I won't argue for its credibility,
because I'm not completely convinced myself.

Yet another resolution could be to acknowledge the Standard has
simply an error there and no conclusions can be drawn.

--
Stan Tobias
mailx `echo (E-Mail Removed)LID | sed s/[[:upper:]]//g`
 
Reply With Quote
 
Stephen Sprunk
Guest
Posts: n/a
 
      05-21-2005
"Michael Mair" <(E-Mail Removed)> wrote in message
news:(E-Mail Removed)...
> CBFalconer wrote:
> > The malloc subsystem doesn't know, and can't know, what type the
> > pointer will be used for. It returns a void*, and the only
> > information it gets is the size required. That size may be for one
> > item, or for many items.
> >
> > Similarly the application has no information as to the alignment
> > requirements for a type. The compiler system does. That is one of
> > the fundamental reasons that malloc is a system function, and
> > simply cannot be written in purely portable code.

>
> Hmmm, I can accept this as an additional explanation but still do
> not see the _necessity_ for the standard giving the cited
> restriction.


If malloc() were allowed to inspect the size of the request and
determine that it was allowed to return a pointer to something of a
different alignment than normal, there would be nothing wrong with that
per se.

However, it is reasonable to assume that no existing implementation did
that at the time C89 was written, nor that any new implementation after
that time would take advantage of that "feature" since it'd be extra
work for virtually no gain, so why make a point of allowing it? Letting
malloc() deal with a single alignment for all requests, regardless of
size, is the simpler option.

S

--
Stephen Sprunk "Those people who think they know everything
CCIE #3723 are a great annoyance to those of us who do."
K5SSS --Isaac Asimov


 
Reply With Quote
 
Walter Roberson
Guest
Posts: n/a
 
      05-21-2005
In article <(E-Mail Removed)>,
SM Ryan <(E-Mail Removed)> wrote:
># The question is not over-specified if one is concerned about
># whether one is leaking memory. When your datasets are ~1 Gb each
># and you are handling them in a loop, you can't afford to allow memory
># to leak.


>All you can do to prevent leaks is to match frees to allocs; you don't need to
>know how the library is implemented to do that much. It's still leaking
>you're stuck unless you can get the library source code.


That's not ALL you can do: you can also ask questions about paragraphs
that one might have overlooked or which might have been clarified
in addendums that one hasn't seen, which proscribe the behaviour
of library routines so as to remove the ambiguity.

If no-one happens to know the answer, or if the answer is
indeterminate, then one can sometimes write one's own routine that
avoids the indeterminate behaviour. In the case of realloc(),
since it is never -required- that the address be left unchanged,
one can write a replacement for realloc() in terms of malloc(),
a memory copy, and a free(), provided that one knows the current
size of the object at hand.
--
Entropy is the logarithm of probability -- Boltzmann
 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Implicit localization Progman ASP .Net 2 03-02-2006 08:06 PM
Implicit rule PIX Nieuws Xs4all Cisco 3 11-30-2005 02:14 AM
Implicit object creation RHC Java 3 11-10-2004 09:22 AM
Passing implicit (standard) "config" object from jsp to java class (not servlet) cvelusc Java 5 06-03-2004 04:14 PM
Question on additional decimals in implicit conversion Jacob Java 7 10-03-2003 10:23 PM



Advertisments