Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > C Programming > Implementing Malloc()

Reply
Thread Tools

Implementing Malloc()

 
 
santosh
Guest
Posts: n/a
 
      11-27-2007
CJ wrote:

> On 26 Nov 2007 at 22:34, Eric Sosman wrote:
>> Not only does this implementation avoid the processing
>> overhead of maintaining potentially large data structures
>> describing the state of memory pools, but it also reduces
>> the "memory footprint" of every program that uses it, thus
>> lowering page fault rates, swap I/O rates, and out-of-memory
>> problems.

>
> All very amusing, but I think you and some of the other posters have
> misunderstood the context of the discussion.
>
> Of course an implementation of malloc should return a pointer to as
> much memory as requested whenever that's possible! The discussion was
> about the rare failure case. Typically, programs assume malloc returns
> a non-null pointer, so if it returns null on failure then the program
> will crash and burn.


A properly written non-trivial program is not going to "crash and burn"
when malloc() returns NULL. It will shutdown gracefully, or even notify
the user to free up some memory and perhaps try again.

> The idea is that instead of a guaranteed crash,


No. There is no guaranteed crash. Consider:

#include <stdlib.h>

int main(void) {
char *p = malloc(SIZE_MAX);
if (p) { free(p); return 0; }
else return EXIT_FAILURE;
}

Where is the "crash"?

> isn't it better to
> make a last-ditch effort to save the program's bacon by returning a
> pointer to what memory there _is_ available? If the program's going to
> crash anyway, it's got to be worth a shot.


If malloc() returns NULL for failure, the program can at least exit
gracefully or even try and recover some memory and keep going, but if
it is going to indicate success but return insufficient space, then a
memory overwrite is almost certainly going to lead to either hard to
debug data corruption or a core dump from a segmentation violation.
This is (for any serious program) worse than a clean, controlled exit.

Besides your function (for whatever it's worth) can easily be written on
top of malloc() for anyone mad enough to want it. No need to include
another gets() into the Standard.

 
Reply With Quote
 
 
 
 
Eric Sosman
Guest
Posts: n/a
 
      11-27-2007
CJ wrote On 11/27/07 12:58,:
> On 26 Nov 2007 at 22:34, Eric Sosman wrote:
>
>> Not only does this implementation avoid the processing
>>overhead of maintaining potentially large data structures
>>describing the state of memory pools, but it also reduces
>>the "memory footprint" of every program that uses it, thus
>>lowering page fault rates, swap I/O rates, and out-of-memory
>>problems.

>
>
> All very amusing, but I think you and some of the other posters have
> misunderstood the context of the discussion.


Well, yes: I understood it as a joke. But if
you're actually serious ...

> Of course an implementation of malloc should return a pointer to as much
> memory as requested whenever that's possible! The discussion was about
> the rare failure case. Typically, programs assume malloc returns a
> non-null pointer, so if it returns null on failure then the program will
> crash and burn.


One could take issue with your assertion about
"typically," and even about "rare." But that's a
side-issue.

> The idea is that instead of a guaranteed crash, isn't it better to make
> a last-ditch effort to save the program's bacon by returning a pointer
> to what memory there _is_ available?


No. For starters, this makes what you consider an
"atypical" program as bad as a "typical" one, because
it becomes impossible to detect a malloc() failure: if
malloc() returns a non-NULL pointer, how can the caller
tell whether it was or wasn't able to supply the memory?

> If the program's going to crash
> anyway, it's got to be worth a shot.


Your unstated assumption is that a crash is worse
than all other outcomes, and I dispute that assumption.
A program that ceases to operate also ceases to produce
wrong answers. It probably also draws attention to the
fact that something has gone wrong, instead of plunging
silently ahead doing who-knows-what.

In my house I have smoke alarms that use nine-volt
batteries to supply their modest electrical needs. The
service life of the battery in this application is quite
long, well over a year, so the battery's "failure rate"
is quite low: When the alarm "requests more electricity"
from the battery, it nearly always succeeds. Yet on the
extremely rare occasion where the battery cannot deliver
enough, the alarm starts beeping at intervals to alert me
to the fact. Then I change the battery, and all is well.

If the battery were designed to work the way you want
malloc() to operate, this wouldn't happen. The battery
would (somehow) pretend to be able to supply the requested
voltage even when it could not, and the alarm would never
discover that the battery was dead. Instead, the alarm
would sit silently on my wall, giving the illusion of
performing its function without actually doing so, pretending
to protect me and my family while leaving us at risk. You
may not admire me much, but you know nothing of my family
and I request that you not condemn them to the flames.

--
http://www.velocityreviews.com/forums/(E-Mail Removed)

 
Reply With Quote
 
 
 
 
Flash Gordon
Guest
Posts: n/a
 
      11-27-2007
CJ wrote, On 27/11/07 17:58:
> On 26 Nov 2007 at 22:34, Eric Sosman wrote:
>> Not only does this implementation avoid the processing
>> overhead of maintaining potentially large data structures
>> describing the state of memory pools, but it also reduces
>> the "memory footprint" of every program that uses it, thus
>> lowering page fault rates, swap I/O rates, and out-of-memory
>> problems.

>
> All very amusing, but I think you and some of the other posters have
> misunderstood the context of the discussion.


No, I believe that Eric and most of the other respondents understood it
perfectly, and that is why they all considered it a terrible idea.

> Of course an implementation of malloc should return a pointer to as much
> memory as requested whenever that's possible! The discussion was about
> the rare failure case.


Rare to you maybe, but I actually use machines fairly hard and it is not
uncommon for them to run out of memory.

> Typically, programs assume malloc returns a
> non-null pointer, so if it returns null on failure then the program will
> crash and burn.


Not if it is written properly, and a lot of programs *are* written properly.

> The idea is that instead of a guaranteed crash, isn't it better to make
> a last-ditch effort to save the program's bacon by returning a pointer
> to what memory there _is_ available? If the program's going to crash
> anyway, it's got to be worth a shot.


Oh dear, you have just caused my copy of VMWare to crash possibly
causing corruption of the guest systems file system and forcing me to
restart a long and complex set of tests involving a number of VMware
sessions. You have also prevented one of the applications I develop from
tidying up cleanly behind itself and notifying the user, possibly
causing corruption of a companies corporate accounts.

No, it is FAR better to do what the standard requires and actually give
the application a shot at doing something sensible when it runs out of
memory, such as warning the user and giving them a chance to free some
memory up, or shutting down tidily so that data is not corrupted etc.
--
Flash Gordon
 
Reply With Quote
 
John Gordon
Guest
Posts: n/a
 
      11-27-2007
In <(E-Mail Removed)> CJ <(E-Mail Removed)> writes:

> Typically, programs assume malloc returns a non-null pointer, so if it

< returns null on failure then the program will crash and burn.

Huh? If malloc returns null, the program detects it handles it as best
it can. It certainly doesn't *use* the null pointer gotten from malloc.

Where's this "guaranteed crash?"

--
John Gordon A is for Amy, who fell down the stairs
(E-Mail Removed) B is for Basil, assaulted by bears
-- Edward Gorey, "The Gashlycrumb Tinies"

 
Reply With Quote
 
Eric Sosman
Guest
Posts: n/a
 
      11-27-2007
jacob navia wrote:
>
> Well, I proposed the function
> mallocTry
> that could give you the best of both worlds. Why do
> you ignore that suggestion?


He's begun by assuming a caller who's too [insert
derogatory adjective] even to compare malloc's value
to NULL. Would this [ida] person have the smarts to
make effective use of mallocTry?

--
Eric Sosman
(E-Mail Removed)lid
 
Reply With Quote
 
Shadowman
Guest
Posts: n/a
 
      11-27-2007
Al Balmer wrote:
> On Mon, 26 Nov 2007 22:40:52 +0100 (CET), CJ <(E-Mail Removed)>
> wrote:
>
>> The worst thing that can happen is that the programmer _does_ write to
>> the end of the mallocated block. In this case, either there's a SIGSEGV
>> again (no worse off than before), or if the 512Kb is in the middle of
>> the heap malloc() is drawing from then the writes might well succeed,
>> and the program can continue albeit with some possible minor data
>> corruption.
>>
>> Do any implementations of malloc() use a strategy like this?
>>

> This is all a joke, isn't it?
>


On that note:

http://groups.google.com/group/comp....4?dmode=source

http://groups.google.com/group/comp....6?dmode=source

Same guy?

--
SM
rot13 for email
 
Reply With Quote
 
Kenneth Brody
Guest
Posts: n/a
 
      11-27-2007
CJ wrote:
>
> We were discussing implementing malloc(), in particular the following
> situation.
>
> Suppose the user requests 1Mb of memory. Unfortunately, we only have
> 512Kb available. In this situation, most mallocs() would return null.


AFAIK, _all_ conforming mallocs return NULL. (With the possible
exception of things like Linux's overcommit scheme. However, it is my
understanding that it is the kernel that lies to malloc, and malloc
does, in fact, believe that the memory is available.)

> The huge majority of programmers won't bother to check malloc() failure


I have to take exception with that statement.

> for such a small allocation, so the program will crash with a SIGSEGV as
> soon as the NULL pointer is dereferenced.


Someone who doesn't check for malloc() failure, especially on such
a large size, deserves what he gets.

Consider, too, the ease in debugging the NULL pointer dereference.

> So why not just return a pointer to the 512Kb that's available? It's


Eww...

> quite possible that the user will never actually write into the upper
> half of the memory he's allocated, in which case the program will have
> continued successfully where before it would have crashed.


I would consider that strategy "asking for trouble".

> The worst thing that can happen is that the programmer _does_ write to
> the end of the mallocated block. In this case, either there's a SIGSEGV
> again (no worse off than before), or if the 512Kb is in the middle of
> the heap malloc() is drawing from then the writes might well succeed,
> and the program can continue albeit with some possible minor data
> corruption.


So continuing "with some possible minor data corruption" is a
viable programming strategy? You want to make buffer overruns a
design strategy?

Consider the difference in difficulty in "malloc returned NULL" to
"I malloced a million bytes, but something else is mysteriously
writing into my buffer, and writing into my buffer causes some
other code to mysteriously crash".

How many Windows updates were to fix "malicious code could allow an
attacker to take over your computer" bugs which were caused by such
buffer overruns?

> Do any implementations of malloc() use a strategy like this?


I sincerely hope not. I don't want my implementation lying to me.
("Lazy mallocs" are bad enough. We don't need "outright lying
mallocs".)

--
+-------------------------+--------------------+-----------------------+
| Kenneth J. Brody | www.hvcomputer.com | #include |
| kenbrody/at\spamcop.net | www.fptech.com | <std_disclaimer.h> |
+-------------------------+--------------------+-----------------------+
Don't e-mail me at: <(E-Mail Removed)>

 
Reply With Quote
 
Default User
Guest
Posts: n/a
 
      11-27-2007
CJ wrote:


> Of course an implementation of malloc should return a pointer to as
> much memory as requested whenever that's possible! The discussion was
> about the rare failure case. Typically, programs assume malloc
> returns a non-null pointer, so if it returns null on failure then the
> program will crash and burn.


You're either an idiot or a troll. You haven't understood, or have
chosen to ignore all the follow-ups.

As such, you're a waste of my time (the greatest usenet sin).

*plonk*





Brian
 
Reply With Quote
 
user923005
Guest
Posts: n/a
 
      11-27-2007
On Nov 27, 9:58 am, CJ <(E-Mail Removed)> wrote:
> On 26 Nov 2007 at 22:34, Eric Sosman wrote:
>
> > Not only does this implementation avoid the processing
> > overhead of maintaining potentially large data structures
> > describing the state of memory pools, but it also reduces
> > the "memory footprint" of every program that uses it, thus
> > lowering page fault rates, swap I/O rates, and out-of-memory
> > problems.

>
> All very amusing, but I think you and some of the other posters have
> misunderstood the context of the discussion.
>
> Of course an implementation of malloc should return a pointer to as much
> memory as requested whenever that's possible! The discussion was about
> the rare failure case. Typically, programs assume malloc returns a
> non-null pointer, so if it returns null on failure then the program will
> crash and burn.
>
> The idea is that instead of a guaranteed crash, isn't it better to make
> a last-ditch effort to save the program's bacon by returning a pointer
> to what memory there _is_ available? If the program's going to crash
> anyway, it's got to be worth a shot.


Surely, surely. This is a troll.

If I asked for 4 MB and the implementation returned 2 MB, I can't
really imagine anything worse than that. Not only is it undefined
behavior, but the behavior is very unlikely even to be repeatable. Why
(on earth) would the program 'crash anyway'? Anyone who does not
check the return of malloc() in a production application is an
incompetent dimwit of the highest order. So there is no way that the
application is going to crash. But of course, anyone who ever did
real-life programming in C knows all of this already.

If malloc() fails, and I could succeed with less memory (e.g. if I am
creating a hash table with a small fill factor, I can increase it),
then I am going to try to allocate less memory. To simply return a
smaller block... you're a riot.

Quite an effective troll, sir. I congratulate you.
 
Reply With Quote
 
jameskuyper@verizon.net
Guest
Posts: n/a
 
      11-27-2007
CJ wrote:
> On 26 Nov 2007 at 22:34, Eric Sosman wrote:
> > Not only does this implementation avoid the processing
> > overhead of maintaining potentially large data structures
> > describing the state of memory pools, but it also reduces
> > the "memory footprint" of every program that uses it, thus
> > lowering page fault rates, swap I/O rates, and out-of-memory
> > problems.

>
> All very amusing, but I think you and some of the other posters have
> misunderstood the context of the discussion.


No, we understood the context; we just have a well-justified lack of
respect for the attitudes that could create such a context

> Of course an implementation of malloc should return a pointer to as much
> memory as requested whenever that's possible! The discussion was about
> the rare failure case. Typically, programs assume malloc returns a
> non-null pointer, so if it returns null on failure then the program will
> crash and burn.
>
> The idea is that instead of a guaranteed crash, isn't it better to make
> a last-ditch effort to save the program's bacon by returning a pointer
> to what memory there _is_ available? If the program's going to crash
> anyway, it's got to be worth a shot.


No, it is not. The sooner and more frequently such incompetently
written code fails, the sooner the programmer will realize why it was
a bad idea to ignore null pointers returned by malloc(). If the
programmer never realizes this, then the sooner and more frequently
the programs fail, the sooner the programmer will be fired and
replaced with someone more competent. Everyone who deserves to be will
be better off as a result. Even the fired programmer will be better
off in the long run, because getting fired will give the ex-programmer
an opportunity to change to a career which doesn't require as much
attention to detail as programming, which should make the ex-
programmer happier in the long run.

Competent programmers check for failed memory allocation, and take
appropriate action. If allocation fails, their programs will either
fail gracefully (NOT crash-and-burn), or they will free up memory
elsewhere and retry the allocation. Your proposed change would cause
competently written programs that rely upon the failure indication to
fail without warning, in order to provide very questionable protection
for incompetently written code.
 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Implementing Interface Gomathi ASP .Net 1 11-17-2005 03:09 PM
Need help implementing a proj on SPARTAN3 Riccardo Fregonese VHDL 2 01-03-2005 01:21 PM
Implementing the CORDIC algorithm without using Real Data Type Johnsy Joseph VHDL 2 10-29-2004 10:49 AM
Implementing E1 - E3 Dev VHDL 1 09-09-2004 09:06 AM
vhdl for implementing pre-fetch and an instruction cache Eqbal Z VHDL 3 11-16-2003 06:07 AM



Advertisments