Velocity Reviews > how to figure out the size of buffer from malloc

# how to figure out the size of buffer from malloc

pembed2012
Guest
Posts: n/a

 02-16-2012
I am wondering how does 'free' know how many bytes to free? For example:

char* s = (char*)malloc(10);
free(s); // how many bytes to free?

How does free know how many bytes to free? I have heard that malloc
can find that out?

Thanks!

James Kuyper
Guest
Posts: n/a

 02-16-2012
On 02/16/2012 03:21 PM, pembed2012 wrote:
> I am wondering how does 'free' know how many bytes to free? For example:
>
> char* s = (char*)malloc(10);
> free(s); // how many bytes to free?
>
> How does free know how many bytes to free? I have heard that malloc
> can find that out?

Howe that information is stored is an implementation detail, likely to
be different on different implementations. As a result, there's no
portable way you can obtain that information.

One common technique is to have malloc() allocate a little bit more than
the requested amount of memory, and to store the size of the allocation
in that extra piece of memory. This extra memory is usually stored at a
fixed negative offset from the pointer that you get from malloc().

Another technique is round each allocation request up to a power of 2,
and to maintain a separate pool of memory for each power of 2. By
looking at the address, free() can tell which pool it came from, and
therefore, how much memory to free.

There's several other, more sophisticated techniques that are in actual
use, but I hope that those two are sufficient for you to understand why
there's no portable way to get this information.
--
James Kuyper

John Gordon
Guest
Posts: n/a

 02-16-2012
In <jhjofu\$jer\$(E-Mail Removed)> pembed2012 <(E-Mail Removed)> writes:

> I am wondering how does 'free' know how many bytes to free? For example:

> char* s = (char*)malloc(10);
> free(s); // how many bytes to free?

A common method is for malloc to allocate a few extra bytes, store the
size in the extra space at the beginning, and return a pointer to the
remaining area.

In your sample code above, malloc might actually allocate fourteen bytes,
store the size information in the first four bytes and return a pointer
to the remaining ten bytes.

Then when free is called, it knows it can back up four bytes from the
given address to find the size of the area to be freed.

> How does free know how many bytes to free? I have heard that malloc
> can find that out?

It's all compiler- and library-specific. The language itself doesn't
specify how it is done.

--
John Gordon A is for Amy, who fell down the stairs
http://www.velocityreviews.com/forums/(E-Mail Removed) B is for Basil, assaulted by bears
-- Edward Gorey, "The Gashlycrumb Tinies"

Lew Pitcher
Guest
Posts: n/a

 02-16-2012
On Thursday 16 February 2012 15:21, in comp.lang.c, (E-Mail Removed)d
wrote:

> I am wondering how does 'free' know how many bytes to free?

As others have said, malloc() and free() are part of the language
definition, and /how/ they work is considered to be "magic" as far as we
mortal userspace programmers go.

OTOH, you can get a feel for what sort of code /might/ go into malloc() and
free() by checking out K&R "The C Programming Language". The original K&R
(and, I believe, the K&R rewritten for C89) has a chapter called "The Unix
System Interface", in which an example "Storage Allocator" is detailed.

HTH
--
Lew Pitcher

Keith Thompson
Guest
Posts: n/a

 02-16-2012
pembed2012 <(E-Mail Removed)> writes:
> I am wondering how does 'free' know how many bytes to free? For example:
>
> char* s = (char*)malloc(10);

The cast is unnecessary and can hide errors.

char *s = malloc(10);

Note that this can fail; you should always check whether malloc()
returned a null pointer.

> free(s); // how many bytes to free?
>
> How does free know how many bytes to free? I have heard that malloc
> can find that out?

The comp.lang.c FAQ is at <http://c-faq.com/>. You've just asked
question 7.26.

--
Keith Thompson (The_Other_Keith) (E-Mail Removed) <http://www.ghoti.net/~kst>
Will write code for food.
"We must do something. This is something. Therefore, we must do this."
-- Antony Jay and Jonathan Lynn, "Yes Minister"

pembed2012
Guest
Posts: n/a

 02-16-2012
On Thu, 16 Feb 2012 20:48:06 +0000, John Gordon wrote:

> In <jhjofu\$jer\$(E-Mail Removed)> pembed2012 <(E-Mail Removed)>
> writes:
>
>> I am wondering how does 'free' know how many bytes to free? For
>> example:

>
>> char* s = (char*)malloc(10);
>> free(s); // how many bytes to free?

>
> A common method is for malloc to allocate a few extra bytes, store the
> size in the extra space at the beginning, and return a pointer to the
> remaining area.
>
> In your sample code above, malloc might actually allocate fourteen
> bytes, store the size information in the first four bytes and return a
> pointer to the remaining ten bytes.
>
> Then when free is called, it knows it can back up four bytes from the
> given address to find the size of the area to be freed.
>
>> How does free know how many bytes to free? I have heard that malloc
>> can find that out?

>
> It's all compiler- and library-specific. The language itself doesn't
> specify how it is done.

Thanks, I've tried this but it does NOT work

This code prints: 0, it should print: 10

int get_malloc_size(void *vp)
{
char *cp = (char*)vp;
static int __done = -1;
static int __offset = -1;
if(__done != 0) {
char *p;
int n,i;
while(1) {
n=random()%10000;
p=(char*)malloc(n);
for(i=1;i<100;i++) {
if(*(int*)(p-i)==n) {
__offset=i;
break;
}
}
free(p);
if(__offset != -1)
break;
}
__done=0;
}
return *(int*)(cp-__offset);
}

main()
{
char *p=(char*)malloc(10);
printf("%d\n", get_malloc_size((void*)p));
free(p);
}

John Gordon
Guest
Posts: n/a

 02-16-2012
In <jhk0oi\$8cu\$(E-Mail Removed)> pembed2012 <(E-Mail Removed)> writes:

> Thanks, I've tried this but it does NOT work

I suggested one possible way that malloc might store the size of the
allocated area. There are lots of different ways this can be done. Your
malloc implementation must use a different mechanism.

I thought I was clear when I said:

> > It's all compiler- and library-specific. The language itself doesn't
> > specify how it is done.

If you want to know how it's done on your system, you'll have to look
at the system documentation.

--
John Gordon A is for Amy, who fell down the stairs
(E-Mail Removed) B is for Basil, assaulted by bears
-- Edward Gorey, "The Gashlycrumb Tinies"

Keith Thompson
Guest
Posts: n/a

 02-16-2012
pembed2012 <(E-Mail Removed)> writes:
> On Thu, 16 Feb 2012 20:48:06 +0000, John Gordon wrote:
>
>> In <jhjofu\$jer\$(E-Mail Removed)> pembed2012 <(E-Mail Removed)>
>> writes:

[...]
>>> How does free know how many bytes to free? I have heard that malloc
>>> can find that out?

>>
>> It's all compiler- and library-specific. The language itself doesn't
>> specify how it is done.

>
> Thanks, I've tried this but it does NOT work
>
> This code prints: 0, it should print: 10
>
> int get_malloc_size(void *vp)
> {
> char *cp = (char*)vp;
> static int __done = -1;
> static int __offset = -1;

Identifiers starting with two underscores are reserved to the
implementation. You should *never* define such identifiers in
your own code. The rules are a little more complex for identifiers
starting with a single underscore but you should still avoid them.

[snip]

> return *(int*)(cp-__offset);
> }
>
> main()

Should be "int main(void)".

> {
> char *p=(char*)malloc(10);
> printf("%d\n", get_malloc_size((void*)p));
> free(p);
> }

Why would you expect it to work? There is *no* portable way to
determine the size of a malloc()ed chunk of memory.

Actually, there is: just remember the argument you passed to malloc():

size_t requested_size = 10;
char *p = malloc(requested_size);
if (p != NULL) {
/* You know you've allocated requested_size bytes. */
}

Note that the implementation could allocate more memory than you
request; it won't necessarily store the requested size anywhere.
But that shouldn't matter to your program.

If you think it does matter, explain what you're trying to accomplish

--
Keith Thompson (The_Other_Keith) (E-Mail Removed) <http://www.ghoti.net/~kst>
Will write code for food.
"We must do something. This is something. Therefore, we must do this."
-- Antony Jay and Jonathan Lynn, "Yes Minister"

Shao Miller
Guest
Posts: n/a

 02-17-2012
On 2/16/2012 17:42, pembed2012 wrote:
> On Thu, 16 Feb 2012 20:48:06 +0000, John Gordon wrote:
>
>> In<jhjofu\$jer\$(E-Mail Removed)> pembed2012<(E-Mail Removed)>
>> writes:
>>
>>> I am wondering how does 'free' know how many bytes to free? For
>>> example:

>>
>>> char* s = (char*)malloc(10);
>>> free(s); // how many bytes to free?

>>
>> A common method is for malloc to allocate a few extra bytes, store the
>> size in the extra space at the beginning, and return a pointer to the
>> remaining area.
>>
>> In your sample code above, malloc might actually allocate fourteen
>> bytes, store the size information in the first four bytes and return a
>> pointer to the remaining ten bytes.
>>
>> Then when free is called, it knows it can back up four bytes from the
>> given address to find the size of the area to be freed.
>>
>>> How does free know how many bytes to free? I have heard that malloc
>>> can find that out?

>>
>> It's all compiler- and library-specific. The language itself doesn't
>> specify how it is done.

>
> Thanks, I've tried this but it does NOT work
>

You can make your own allocator and freer and use them instead of
'malloc' and 'free'. In fact, you could even do:

#include <standard_stuff1.h>
#include <standard_stuff2.h>

#include <system_stuff.h>

#include "my_stuff.h"

#define malloc my_malloc
#define free my_free

Then you could use 'malloc' and 'free' but the result would be calls to

Here's an example alternative to using 'malloc' that I haven't looked at
for a while:

/**
* Delete all lines containing
* 'ALL_IN_ONE' if you split this example
* into separate files.
*/
#define ALL_IN_ONE 1

/****
* pfxalloc.h
*
* It took several minutes to find
* a free *alloc. (No pun intended.)
*/
#ifndef PFXALLOC_H_

/* For 'offsetof' and 'size_t' */
#include <stddef.h>

/*** Macros */
#define PFXALLOC_H_

#ifndef alignof

/* Derived from Mr. Chris M. Thomasson */
#define alignof(type) \
(offsetof(struct {char c; type t;}, t))

#ifdef __alignof_is_defined
#undef __alignof_is_defined
#endif /* __alignof_is_defined */

#define __alignof_is_defined 1
#endif /* alignof */

/*** Functions */

/**
* Allocate memory for an object and an
* associated "prefix" object.
*
* @v pfx_size The "prefix" size.
* @v obj_alignment The object's alignment
* requirement.
* @v obj_size The object's size.
* @ret void * Points to the object,
* or is NULL for failure.
*/
extern void * pfxalloc(size_t, size_t, size_t);

/**
* Get an object's associated "prefix" object.
*
* @v obj The object to fetch the
* associated "prefix" for.
* @ret void * Points to the "prefix",
* or is NULL for failure.
*/
extern void * pfxget(void *);

#endif /* PFXALLOC_H_ */

/**** pfxalloc.c */

/* For 'malloc' */
#include <stdlib.h>
#if !ALL_IN_ONE
/* For 'size_t', 'ptrdiff_t' and 'NULL' */
#include <stddef.h>
/* For 'alignof' */
#include "pfxalloc.h"
#endif /* !ALL_IN_ONE */

/*** Constants */
enum cv {
cv_ptrdiff_alignment =
alignof(ptrdiff_t),
cv_zero = 0
};

/*** Functions */

/**
* The layout looks like:
* +-----+-------------+--------+--------+
* | pfx | opt_padding | offset | |
* +- -+- -+- -+ object |
* +----------------------------+--------+
*/
void * pfxalloc(
size_t pfx_size,
size_t obj_alignment,
size_t obj_size
) {
size_t offset_at;
ptrdiff_t * offset;
size_t total_size;
unsigned char * mem;
unsigned char * obj;

/* Sanity-check alignment value */
if (obj_alignment & (obj_alignment - 1))
return NULL;

/* Calculate the offset position */
offset_at = pfx_size;
offset_at += sizeof *offset;
/* Check for wrap-around */
if (offset_at < pfx_size)
return NULL;
--offset_at;
offset_at /= sizeof *offset;

/* Calculate the header size */
header_size = (offset_at + 1) * sizeof *offset;
/* Check for wrap-around */
return NULL;

if (obj_alignment > cv_ptrdiff_alignment) {
new_hdr_size += obj_alignment;
/* Check for wrap-around */
return NULL;
--new_hdr_size;
new_hdr_size /= obj_alignment;
new_hdr_size *= obj_alignment;
}

/* Allocate storage */
/* Check for wrap-around */
if (total_size < pfx_size || total_size < obj_size)
return NULL;
mem = malloc(total_size);
if (!mem)
return mem;

/* Point to the object */

/* Note the offset to the prefix */
offset = (ptrdiff_t *) obj - 1;
*offset = obj - mem;

return obj;
}

/* Fetch an asociated prefix for an object */
void * pfxget(void * obj) {
ptrdiff_t * offset;
unsigned char * prefix;

if (!obj)
return obj;

offset = obj;
--offset;
prefix = obj;
prefix -= *offset;

return prefix;
}

(It could probably be improved.)

Guest
Posts: n/a

 02-17-2012
If you're not too worried about portability, you can use
malloc_usable_size(), malloc_size(), or _msize(), depending on the
system. The value returned is the actual size of the block, which may
be a bit larger than what was requested.

Mark