Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > C Programming > Freeing memory - will it be available immediately

Reply
Thread Tools

Freeing memory - will it be available immediately

 
 
Micah Cowan
Guest
Posts: n/a
 
      02-27-2008
user923005 wrote:
> On Feb 26, 5:20 pm, Micah Cowan <(E-Mail Removed)> wrote:
>> Micah Cowan wrote:
>>> This is not an accurate statement, in that the theory is neither
>>> recent, nor
>>> original to Kelsey. While watching this thread, I've had the
>>> very strong feeling that I'd seen Kelsey's argument before, and that
>>> it was
>>> accepted by respected regulars to this list (which was why I was
>>> somewhat
>>> surprised to find it largely spurned by the same).

>> Arg. Apologies for the nasty formatting. I guess I wasn't confident that
>> Google would break my lines, so I did it by hand, which was apparently
>> shooting myself in the foot.

>
> The links you provided are adequate proof that even experts are not
> agreed on as to the definite meaning of the wording.
>
> I think that the DR should have been followed through. After all,
> here we are in a big debate again over the same thing.
>
> I guess that three years hence we will see another cycle. When it is
> something as important as "How does this computer language work?" it
> seems to me that we should make it totally unambiguous.


Well, given that my system, a commonly-used development platform,
violates the standard's guarantees wrt allocated memory in a rather more
egregious way - namely, to allow malloc() to return a pointer, and then
decide later, upon that pointer's use, whether it should actually make
memory available through that pointer or simply kill the process for
attempting to use a "valid" pointer... the text of the standard in this
matter ends up not distressing me too much, given that, any way you look
at it, my implementation's busted.

(I'm speaking, of course, of Linux's OOM killer.)

Still, yeah, seems like it should be addressed. By someone other than
me, of course - I'm too lazy.

--
Micah J. Cowan
Programmer, musician, typesetting enthusiast, gamer...
http://micah.cowan.name/
 
Reply With Quote
 
 
 
 
Walter Roberson
Guest
Posts: n/a
 
      02-27-2008
In article <(E-Mail Removed)>,
Kelsey Bjarnason <(E-Mail Removed)> wrote:

>On Tue, 26 Feb 2008 18:31:45 +0000, Walter Roberson wrote:



>> You seem to be making at least two assumptions:


>> 1) That it is not the operating system that is responsible for the
>> allocation of memory within a running program; and


>I make no such assumption. I merely point out that an implementation, to
>be conforming according to the definition of free, cannot release the
>memory back to the OS, or, if it does, it must do so in some manner which
>keeps the memory "reserved" for further allocation by the C program.


If the implementation relies upon the operating system to manage
memory, then releasing the memory back to the OS would be the proper
thing to do for that implementation. After all, if the implementation
relies upon the operating system to manage memory and it does -not-
release the memory back to the operating system, then the operating
system is not going to be allocate that memory to the program in
response to a further malloc()/calloc() call, since the operating
system will not know that the memory is free. Thus if the
implementation does not have a memory allocator that is independant
of the OS (and there is no requirement that it does) then your
interpretation (that memory is never released back to the OS) would
render the implementation non-conformant.
--
amazon.com's top 8 books about "walter" are Kotzwinkle/ Gundy/ Colman's
"Walter the Farting Dog"
 
Reply With Quote
 
 
 
 
ymuntyan@gmail.com
Guest
Posts: n/a
 
      02-27-2008
On Feb 26, 6:53 pm, Kelsey Bjarnason <(E-Mail Removed)> wrote:
> [snips]
>
> On Tue, 26 Feb 2008 09:38:01 -0800, Keith Thompson wrote:
> >> If you can find a clause in the definition of free which allows for
> >> some other behaviour, such as handing the memory back to the OS, please
> >> trot it out; I can't find it anywhere.

> > Show us where the standard says that an implementation in which storage
> > deallocated by free() is made available for further allocation by
> > another program invoked via the standard system() function would be
> > non-conforming.

>
> I'll turn that around: where in the definition of, say, the pre-increment
> operator as applied to a variable of type int, does it say that if the
> variable's value before increment was 3 and the value after increment is
> 19,471, the implementation is non-conforming? Now try again, but for
> values of 97 before and 65 after.
>
> Right; there's no such definition. Doesn't exist. There's a reason for
> this; it's pointless.
>
> What matters is not that there is a clause for every possible case, but
> that there is a rule, or set of rules, for determining what to expect.
> There is; the standard goes to some length to differentiate between
> defined, undefined, unspecified and implementation defined behaviours,
> with the assumption the reader will apply those rules when determining
> the implications of the rest of the document.
>
> Thus, for example, one need not have a specific clause which disallows
> the pre-increment operator to produce a result of 19,471 from a previous
> value of 3; it is sufficient to have a definition of how pre-increment is
> defined to work, plus the rules for determining what to expect: the
> behaviour is defined _thus_, any implementation which does otherwise is
> non-conforming.
>
> Therefore, by defining a behaviour for free, the standard defines what
> free is expected to do, how it is expected to behave. The up-front rules
> about interpreting the behaviour do not need to expressly disallow the
> case you mention, as the defined behaviour of free does not include such
> a case *nor* does it include any allowance for implementation-defined
> behaviour or the like; it behaves as documented, or the implementation
> fails to meet the requirements of the standard and thus is non-conforming.
>
> And what *is* the defined behaviour? Right - to make the memory
> available for further allocation.
>
> Ah, but wait. To *whom* does it define this behaviour? Is the C
> standard suddenly the defining document for POSIX-based OSen? How about
> non-POSIX-based OSen? How about process management in general? Cars?
> Toaster ovens?
>
> Nope, none of these apply. The behaviour defined is the contract between
> implementation and developer, in the context of writing C programs to be
> used in conjunction with a C implementation. That is *it*. No OSen, no
> cars, no toaster ovens, just C, all C, nothing but C.
>
> And what is the defined behaviour granted by the standard to the C
> programmer using a conforming implementation? Right - that the memory is
> made available for further allocation.


If two subsequent malloc() calls never succeed (both of them, that
is), is it a conforming implementation? "Further allocations" are
fine, it's just the very next that doesn't work. And to break the
Dan Pop's example: two subsequent malloc() calls do not succeed
if they have the same argument. And now the "as if" rule kicks in.

To make it easier for you, here's a simpler example:
objects are stored in toasters which are delivered to the
program in cars when it calls malloc(), and are carried
away when the program calls free(). The toasters are
of different size and shape, and if one is being carried
away, it can't be returned until it's been delivered back
to the toasters store. But they are "available" for further
allocation. For further allocation at some point later,
that is. Can you quote the standard saying such an
implementation would be non-conforming?

You make an important assumption here, that an implementation
must behave as conventional implementation do (at least
for small N, since conventional implementations do not actually
always behave this way): the sequence p = malloc(N); free(p);
*immediately* returns the program to its original state as it
was before the malloc() call. But the standard doesn't demand
that, does it?

Yevgen
 
Reply With Quote
 
santosh
Guest
Posts: n/a
 
      02-27-2008
Herbert Rosenau wrote:

<snip>

> So, look on:
>
> 1 mashine will run only one single program. This single program
> (commonly known as operation system) loads many different extensions -
> you sees them as differen programs, but they are nothing else than
> plugins like modern browsers use to extend theyr abilities.


This isn't true for microkernel based systems and those with
virtualisation.

<snip>

 
Reply With Quote
 
Richard Bos
Guest
Posts: n/a
 
      02-27-2008
Kelsey Bjarnason <(E-Mail Removed)> wrote:

> On Tue, 26 Feb 2008 07:25:14 +0000, Richard Bos wrote:
>
> > Kelsey Bjarnason <(E-Mail Removed)> wrote:
> >
> >> Is it? I see nothing whatsoever in the definition of free which allows
> >> for any implementation-defined behaviour; its behaviour, quite the
> >> contrary, is absolutely and clearly defined, and *does not permit*
> >> handing the memory back to the OS.

> >
> > You keep opining that, and you continue to be wrong.

>
> If I'm wrong, please point out the clause which allows the defined
> behaviour to work other than as the standard defines it


There is none. Your interpretation of "as the standard defines it" is
wrong.

Richard
 
Reply With Quote
 
Kelsey Bjarnason
Guest
Posts: n/a
 
      02-27-2008
[snips]

On Wed, 27 Feb 2008 00:27:10 +0000, Gordon Burditt wrote:

> Tell me what prohibits this implementation:


[irrelevances snipped]

> A C program may free memory and the implementation may return it to the
> OS. The OS may then let a non-C program (or a C program running in
> non-standard-compliant mode)


Non-conforming mode? Yes, well, you just answered your question: you're
using a non-conforming implementation, which means it's either broken, or
non-conforming by intent; if the former, file a bug report. If the
latter, consider it a QoI issue.

 
Reply With Quote
 
Kelsey Bjarnason
Guest
Posts: n/a
 
      02-27-2008
On Wed, 27 Feb 2008 10:19:59 +0000, Richard Bos wrote:

> Kelsey Bjarnason <(E-Mail Removed)> wrote:
>
>> On Tue, 26 Feb 2008 07:25:14 +0000, Richard Bos wrote:
>>
>> > Kelsey Bjarnason <(E-Mail Removed)> wrote:
>> >
>> >> Is it? I see nothing whatsoever in the definition of free which
>> >> allows for any implementation-defined behaviour; its behaviour,
>> >> quite the contrary, is absolutely and clearly defined, and *does not
>> >> permit* handing the memory back to the OS.
>> >
>> > You keep opining that, and you continue to be wrong.

>>
>> If I'm wrong, please point out the clause which allows the defined
>> behaviour to work other than as the standard defines it

>
> There is none. Your interpretation of "as the standard defines it" is
> wrong.


You keep saying that, why is it you seem unable to demonstrate it?

 
Reply With Quote
 
Kelsey Bjarnason
Guest
Posts: n/a
 
      02-27-2008
[snips]

On Tue, 26 Feb 2008 19:30:50 -0800, ymuntyan wrote:

> If two subsequent malloc() calls never succeed (both of them, that is),
> is it a conforming implementation?


We cannot tell from this.

> "Further allocations" are fine, it's
> just the very next that doesn't work. And to break the Dan Pop's
> example: two subsequent malloc() calls do not succeed if they have the
> same argument. And now the "as if" rule kicks in.


void *ptra = malloc(100);
void *ptrb = malloc(100);

You're saying that if either allocation fails, there's a problem? Aside
from the simple lack of available memory, what, exactly, is the problem?

> To make it easier for you, here's a simpler example: objects are stored
> in toasters which are delivered to the program in cars when it calls
> malloc(), and are carried away when the program calls free().


No, they're not; if this occurred, the implementation cannot meet the
behaviour mandated by the definition of free, as the car may not return.
Thus the objects - the blocks of memory - are not allowed to be carried
away when free is called.

> further allocation at some point later, that is. Can you quote the
> standard saying such an implementation would be non-conforming?


I can point out that the implementation does not know how many toasters
are at the toaster store, thus it cannot guarantee there are - or ever
will be - enough to meet the requests, therefore it has no choice, under
the wording of the standard but to store them itself.

The definition of free says *nothing* about the memory being made
available for later allocation *if available* or *if not consumed by some
other process*. There is zero - *zero* - allowance in the wording for
any wiggle room whatsoever.

> You make an important assumption here, that an implementation must
> behave as conventional implementation do (at least for small N, since
> conventional implementations do not actually always behave this way):
> the sequence p = malloc(N); free(p); *immediately* returns the program
> to its original state as it was before the malloc() call. But the
> standard doesn't demand that, does it?


I make no such assumption, and nothing I've said thus far requires such
an assumption. The standard says the memory is made available for
further allocation, it does not say when - it could be next Tuesday.

The point at hand is that once a piece of memory has been allocated by
the program, free cannot release it back to the OS, because, even if the
implementation's memory management is so inefficient it cannot make the
memory available for further allocation until Tuesday, it nevertheless is
required to do so, while handing the memory back to the OS means it
cannot ensure that it meets the requirements of the standard, no matter
how inefficiently.

 
Reply With Quote
 
Kelsey Bjarnason
Guest
Posts: n/a
 
      02-27-2008
On Wed, 27 Feb 2008 00:44:02 +0000, Gordon Burditt wrote:

>>Correct; *if* the memory is available, there's no problem. However, the
>>implementation cannot guarantee that this will be the case; the OS may
>>well hand the memory off to some other process.

>
> There's nothing wrong with that unless the memory is handed off to some
> other process *PERMANENTLY*.


Which it may well be. Since the implementation is *required* to make the
memory available for further allocation, it cannot therefore rely on the
OS which may well hand that memory off to something else until the end of
time.

>>Thus, to be conforming,
>>the implementation *cannot* hand the memory back to the OS; it must
>>retain it, reserve it, for further allocation by the program.


> No, if the C program wants it back, the OS can simply wait until the
> memory is available again.


Which it may never be, thus violating the requirements of the standard.

 
Reply With Quote
 
Kelsey Bjarnason
Guest
Posts: n/a
 
      02-27-2008
On Wed, 27 Feb 2008 03:03:37 +0000, Walter Roberson wrote:

> In article <(E-Mail Removed)>, Kelsey Bjarnason
> <(E-Mail Removed)> wrote:
>
>>On Tue, 26 Feb 2008 18:31:45 +0000, Walter Roberson wrote:

>
>
>>> You seem to be making at least two assumptions:

>
>>> 1) That it is not the operating system that is responsible for the
>>> allocation of memory within a running program; and

>
>>I make no such assumption. I merely point out that an implementation,
>>to be conforming according to the definition of free, cannot release the
>>memory back to the OS, or, if it does, it must do so in some manner
>>which keeps the memory "reserved" for further allocation by the C
>>program.

>
> If the implementation relies upon the operating system to manage memory,
> then releasing the memory back to the OS would be the proper thing to do
> for that implementation.


Careful, "proper" here is the very issue of contention.

You and I (presumably) think that "proper" behaviour is that if memory is
freed, it should go back to the OS to be used by whatever may need
memory; this is certainly the desirable operation, especially in a
multitasking environment.

However, "proper" behaviour as defined by the standard is that the memory
is made available for further allocation, in the context of defining the
behaviour of a C program and the libraries upon which it depends - thus
"proper" is that it can only hand the memory back to the OS if the
implementation can guarantee the memory will, in fact, be available for
later allocation, something which, in the general case, it cannot
guarantee. Thus "proper" here mandates that the implementation _not_
return the memory back to the OS.

> After all, if the implementation relies upon
> the operating system to manage memory and it does -not- release the
> memory back to the operating system, then the operating system is not
> going to be allocate that memory to the program in response to a further
> malloc()/calloc() call, since the operating system will not know that
> the memory is free.


This is an implementation detail. There's nothing to prevent an
implementation using OS services to allocate blocks of memory, then using
its own memory manager to handle subsequent reallocations. The same sort
of thing is sometimes done when the provided allocator is known to be
excessively slow; rather that constantly calling it, the application
instead retains the allocated memory, doling out portions as needed,
going back to the OS only when the previously allocated pool is empty.
This is the same sort of thing, just the implementation doing it.

Put it this way. Suppose the first thing I do on program startup is to
allocate 100MB of memory. I immediately free the memory. The
implementation, knowing that the OS's allocation system is painfully
slow, does *not* give the 100MB back to the OS, it keeps it reserved for
itself. A later allocation - say of 1MB - comes from the previously
allocated pool, quickly and efficiently. When the pool is exhausted,
then the implementation goes back to the OS for more.

The OS need not know the 100MB was ever freed; indeed, as far as it's
concerned, it wasn't freed.

All of which, however, is essentially beside the point - implementation
details. However the implementation does it, as long as it ensures the
memory *is* available, all's good. In the general case, this means it
cannot ever return the memory back to the OS, as there's no way to tag
the memory as "reserved for this program", but again, it's an
implementation detail.

 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
memory allocation and freeing memory Rodrigo Dominguez C Programming 11 06-14-2005 11:54 PM
freeing memory Harald Kirsch Java 0 04-22-2005 09:17 AM
freeing memory Rajesh.Rapaka Java 17 04-21-2005 10:11 PM
Freeing more Virtual Memory in Compaq Presario S4000NX Bun Mui Computer Support 2 05-22-2004 08:46 PM
some queries on freeing memory allocated using malloc Hassan Iqbal C Programming 3 09-25-2003 02:53 PM



Advertisments