Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > C Programming > Will 'free' return the memory Immediately to the OS ?

Reply
Thread Tools

Will 'free' return the memory Immediately to the OS ?

 
 
Raj Pashwar
Guest
Posts: n/a
 
      05-15-2012
As per subject-line, thank-you
 
Reply With Quote
 
 
 
 
Ian Collins
Guest
Posts: n/a
 
      05-15-2012
On 05/16/12 09:34 AM, Raj Pashwar wrote:
> As per subject-line, thank-you


Why couldn't you just write

Will 'free' return the memory Immediately to the OS ?

Assuming there is an OS, Yes and no. It depends on how the OS works.
Memory is typically mapped to a process in pages and the allocator
allocates memory from these. Most operating systems will leave unused
pages mapped to the process unless they are needed elsewhere.

--
Ian Collins
 
Reply With Quote
 
 
 
 
James Kuyper
Guest
Posts: n/a
 
      05-15-2012
On 05/15/2012 05:34 PM, Raj Pashwar wrote:
> As per subject-line, thank-you


The malloc family is often implemented by allocating large blocks from
the operating system, and handing out only a small portion of each block
at a time. free() cannot possibly return a block of memory to the OS
until all allocations from that block are free()'d, so you shouldn't
expect an immediate increase in memory available to other processes.

Even when free() could return a block of memory to the OS, it might not
choose to do so; the standard doesn't require it.

On many systems, it doesn't matter much whether or not the memory is
returned; if a sufficiently large block of memory remains unused for a
sufficiently long period of time, the OS may have means of detecting
that fact and swapping it out to disk, transparently to the user. The
memory needn't even have been free()d yet.
 
Reply With Quote
 
Keith Thompson
Guest
Posts: n/a
 
      05-15-2012
Raj Pashwar <(E-Mail Removed)> writes:
> As per subject-line, thank-you


Please include the question in the body of your message:

Will 'free' return the memory Immediately to the OS ?

Here's what the C standard says:

The free function causes the space pointed to by ptr to be
deallocated, that is, made available for further allocation.

It's unspecified whether "made available for further allocation"
means that it's returned to the OS. Typically it isn't.
Some implementations might return free()d memory to the OS, but
it's not practical to do so in all cases. For example, if you have
three allocated chunks of memory that happen to be contiguous, and
you free() the second one, it's likely to be difficult to remove
it from the current program's ownership while keeping its neighbors.

--
Keith Thompson (The_Other_Keith) http://www.velocityreviews.com/forums/(E-Mail Removed) <http://www.ghoti.net/~kst>
Will write code for food.
"We must do something. This is something. Therefore, we must do this."
-- Antony Jay and Jonathan Lynn, "Yes Minister"
 
Reply With Quote
 
Ben Pfaff
Guest
Posts: n/a
 
      05-15-2012
The FAQ says:

7.25: I have a program which mallocs and later frees a lot of memory,
but I can see from the operating system that memory usage
doesn't actually go back down.

A: Most implementations of malloc/free do not return freed memory
to the operating system, but merely make it available for future
malloc() calls within the same program.
 
Reply With Quote
 
Eric Sosman
Guest
Posts: n/a
 
      05-16-2012
On 5/15/2012 5:34 PM, Raj Pashwar wrote:
> As per subject-line, thank-you


As per signature, you're welcome.

--
Eric Sosman
(E-Mail Removed)d
Question 7.25 in <http://www.c-faq.com/>
 
Reply With Quote
 
Stephen Sprunk
Guest
Posts: n/a
 
      05-16-2012
On 16-May-12 12:29, David T. Ashley wrote:
> No [modern PC] OS is likely to have the capability to hand the run-time
> library a chunk of memory smaller than the page size supported by the
> memory management hardware. I don't know what this figure is nowadays,
> but I'd guess maybe between 256K and 1M.


On x86, the page sizes are 4kB (normal) and 2MB/4MB (large), with the
latter depending on whether PAE is enabled. Other architectures vary.

> You typically wouldn't be able to allocate from the operating system,
> for example, a 30-byte chunk of memory.


Nor would you want them to, even if they were able, because that would
greatly increase the number of (relatively expensive) system calls.

> The run-time library is likely to take these large chunks allocated by
> the operating system and subdivide them so it can hand out smaller
> chunks via malloc().


Exactly; it's more efficient to do it that way because it reduces the
number of (relatively expensive) system calls.

> There is no guarantee that the algorithm used by malloc() and free()
> would allow any memory to be returned to the OS until every block had
> been free()'d.


Depending on the system calls available, it may not even be possible.
For instance, the traditional Unix brk()/sbrk() interface dictates a
fixed starting address and variable ending address for the entire heap.
The newer anonymous mmap()/munmap() interface allows allocating and
deallocating individual pages or groups of pages. I don't know how the
Windows interface works, but I'd guess it's similar to the latter.

S

--
Stephen Sprunk "God does not play dice." --Albert Einstein
CCIE #3723 "God is an inveterate gambler, and He throws the
K5SSS dice at every possible opportunity." --Stephen Hawking
 
Reply With Quote
 
Stephen Sprunk
Guest
Posts: n/a
 
      05-17-2012
On 17-May-12 08:47, David T. Ashley wrote:
> On Wed, 16 May 2012 18:36:57 -0500, Stephen Sprunk
> <(E-Mail Removed)> wrote:
>> On 16-May-12 12:29, David T. Ashley wrote:
>>> No [modern PC] OS is likely to have the capability to hand the run-time
>>> library a chunk of memory smaller than the page size supported by the
>>> memory management hardware. I don't know what this figure is nowadays,
>>> but I'd guess maybe between 256K and 1M.

>>
>> On x86, the page sizes are 4kB (normal) and 2MB/4MB (large), with the
>> latter depending on whether PAE is enabled. Other architectures vary.

>
> I believe you, but I'm somewhat surprised at this figure. Hardware
> memory management is usually a tradeoff between silicon complexity,
> complexity of setting up the hardware by the software, and
> granularity.
>
> I figured with a modern PC having maybe 4G of memory, you might want
> to divide the memory into maybe 1,000 - 10,000 pages, so I would have
> figured page sizes in the range of 512K - 4M. 4K surprises me.


Remember, these page sizes were dictated by the i386, when 4kB was still
a fairly big chunk of memory. Most PCs at the time had far less than
4MB of RAM. Virtual memory was mostly about swapping to disk, so the
smaller the pages, the faster a page fault is handled. Also, smaller
pages meant you could get more of the active set in RAM instead of
constantly swapping in and out large pages that were mostly inactive.

The only real cost to small pages is the need for large TLBs to avoid
walking the page table more than once per page, but in most cases that
simply isn't a problem these days for most workloads--or at least not
enough of a problem to justify the incredibly high cost of changing the
ISA. For certain workloads, eg. database servers, there are special OS
APIs to get large pages. This can be a major performance improvement,
but only if you have enough RAM to prevent them from being evicted.

S

--
Stephen Sprunk "God does not play dice." --Albert Einstein
CCIE #3723 "God is an inveterate gambler, and He throws the
K5SSS dice at every possible opportunity." --Stephen Hawking
 
Reply With Quote
 
Kaz Kylheku
Guest
Posts: n/a
 
      05-17-2012
On 2012-05-17, David T Ashley <(E-Mail Removed)> wrote:
> On Wed, 16 May 2012 18:36:57 -0500, Stephen Sprunk
><(E-Mail Removed)> wrote:
>
>>On 16-May-12 12:29, David T. Ashley wrote:
>>> No [modern PC] OS is likely to have the capability to hand the run-time
>>> library a chunk of memory smaller than the page size supported by the
>>> memory management hardware. I don't know what this figure is nowadays,
>>> but I'd guess maybe between 256K and 1M.

>>
>>On x86, the page sizes are 4kB (normal) and 2MB/4MB (large), with the
>>latter depending on whether PAE is enabled. Other architectures vary.

>
> I believe you, but I'm somewhat surprised at this figure. Hardware
> memory management is usually a tradeoff between silicon complexity,
> complexity of setting up the hardware by the software, and
> granularity.
>
> I figured with a modern PC having maybe 4G of memory, you might want
> to divide the memory into maybe 1,000 - 10,000 pages, so I would have
> figured page sizes in the range of 512K - 4M. 4K surprises me.


At 1000 pages, the external fragmentation would be bad. The nice thing
about small page sizes is that it is easier for a process to return
unneeded pages to the operating system. It's easier to find a contiguous
page-aligned 4K chunk of memory which is not in use and free it, than to
find such a 4Mb chunk of memory.

But, never mind that, the real problem is that address spaces do not share
pages, and neither do distinct memory mappings within address spaces.

Firstly, look at your "ps" or "task manager" and see how many processes are
running. You need at least that many pages because you would never allocate two
processes into the same page: you lose protection (as well as the abilility to
link the base executables to run at the same fixed virtual address).

Then, within a process, it is customary to use different page ranges for
different memory mappings, like stack (main one and per-thread), executable, or
shared library. For large malloc requests, it is good to use a function like
mmap. When free is called on that, it can be entirely returned to the operating system.

Every single one of these uses would eat into your allotment of 1000 pages,
quickly gobbling up all of it.

> I'm too lazy to look it all up. The x86 world at the low levels has
> an entry cost. Right now I'm in the ARM world.


4K pages sizes are common in the ARM world, the MIPS, world.

Linux/MIPS only goes up to 64K page sizes (at kernel compile time)
though the R8000 TLB goes from 4K to 16M.

It doesn't make sense to use very large page sizes globally.

--
If you ever need any coding done, I'm your goto man!
 
Reply With Quote
 
Keith Thompson
Guest
Posts: n/a
 
      05-17-2012
David T. Ashley <(E-Mail Removed)> writes:
> On Wed, 16 May 2012 18:36:57 -0500, Stephen Sprunk
> <(E-Mail Removed)> wrote:
>>On 16-May-12 12:29, David T. Ashley wrote:
>>> No [modern PC] OS is likely to have the capability to hand the run-time
>>> library a chunk of memory smaller than the page size supported by the
>>> memory management hardware. I don't know what this figure is nowadays,
>>> but I'd guess maybe between 256K and 1M.

>>
>>On x86, the page sizes are 4kB (normal) and 2MB/4MB (large), with the
>>latter depending on whether PAE is enabled. Other architectures vary.

>
> I believe you, but I'm somewhat surprised at this figure. Hardware
> memory management is usually a tradeoff between silicon complexity,
> complexity of setting up the hardware by the software, and
> granularity.
>
> I figured with a modern PC having maybe 4G of memory, you might want
> to divide the memory into maybe 1,000 - 10,000 pages, so I would have
> figured page sizes in the range of 512K - 4M. 4K surprises me.
>
> I'm too lazy to look it all up. The x86 world at the low levels has
> an entry cost. Right now I'm in the ARM world.


An x86 system might have hundreds of processes running simultaneously,
and some of those are going to be quite small. A large page size would
substantially increase total memory usage.

--
Keith Thompson (The_Other_Keith) (E-Mail Removed) <http://www.ghoti.net/~kst>
Will write code for food.
"We must do something. This is something. Therefore, we must do this."
-- Antony Jay and Jonathan Lynn, "Yes Minister"
 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
[Q] how can I start a shell process and return immediately? Stephen Bannasch Ruby 16 07-10-2009 04:45 PM
Freeing memory - will it be available immediately karthikbalaguru C Programming 119 03-05-2008 01:35 AM
Run the program and return immediately Victor 'Zverok' Shepelev Ruby 2 02-22-2008 05:22 PM
what value does lack of return or empty "return;" return Greenhorn C Programming 15 03-06-2005 08:19 PM
How I free memory and return value from that memory area? sam C++ 2 06-27-2003 01:29 PM



Advertisments