Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > C Programming > Recovering from out of memory on the stack

Reply
Thread Tools

Recovering from out of memory on the stack

 
 
jacob navia
Guest
Posts: n/a
 
      01-10-2008
Flash Gordon wrote:
> jacob navia wrote, On 10/01/08 20:27:
>> You can have an idea of how much stack you are using if you do
>> the following
>>
>> char *StartOfStack;
>> int main(...)
>> {
>> char foo;
>>
>> StartOfStack = &foo;

>
> Of course, if you do not call main recursively the compiler might decide
> to allocate foo where it allocates "global" variables. I can see reasons
> why this optimisation might make sense.
>


You are daydreaming.
I bet you can't name a single compiler that does that kind of nonsense!

>> }
>>
>> This will tell you approximately WHERE the stack begins.
>>
>> You can see how much you have used if in the function "fn" you do:
>>
>> int fn(void)
>> {
>> char foo;
>> char *CurrentStack = &foo;
>>
>> size_t StackSize = CurrentStack - StartOfStack;

>
> Of course, if the stack grows in the opposite direction (as it does on
> the PowerPC) it could give very unexpected results.


And the fix is to take the absolute value... We are interested in the
distance.

> Not to mention on
> embedded compilers which people have posted about as recently as this
> week that do things like allocating *all* local variables statically.
>


Who cares?

We are speaking about STACK OVERFLOW here, so those compilers are
OFF TOPIC for this thread.

>>
>> You can then TEST before you call a function that uses a lot of stack
>> if you are beyond some limit.

>
> However you have no way of knowing (in general) what limit you need to
> test against, so it does not help you.
>


Of course you know, and if you don't you can measure it with a program
like the above one.

#include <stdio.h>
char *StartOfStack;
int main(int argc,char *argv[])
{
char foo,*p;
char m[8192*16];
if (argc > 0)
p = StartOfStack = &foo;
else
p = &foo;
printf("Stack %d\n",StartOfStack-p);
main(0,NULL);
// Not reached
}

Output:
Stack 0
Stack 131100
Stack 262200
Stack 393300
Stack 524400
Stack 655500
Stack 786600

You can adjust the size of the array to make more finer measures.

>> Of course this is very compiler specific too, but should work
>> in most systems.

>
> Apart from PowerPC and related processors (because you have the stack
> direction wrong), so it won't work on big iron (or even relatively small
> servers like those we run our accounts system on in my company) from IBM
> or Apples from before they switched to x86 processors, or the afore
> mentioned embedded compilers...
>



HELLO HELLO

This thread is about stack overflows!

We are speaking about stack overflows here.

http://dictionary.reference.com/browse/pedant

pedant

ľnoun
1.a person who makes an excessive or inappropriate display of learning.
2.a person who overemphasizes rules or minor details.
3.a person who adheres rigidly to book knowledge without regard to
common sense.


--
jacob navia
jacob at jacob point remcomp point fr
logiciels/informatique
http://www.cs.virginia.edu/~lcc-win32
 
Reply With Quote
 
 
 
 
Default User
Guest
Posts: n/a
 
      01-10-2008
http://www.velocityreviews.com/forums/(E-Mail Removed) wrote:

> Interesting.


Please don't top-post. Your replies belong following or interspersed
with properly trimmed quotes. See the majority of other posts in the
newsgroup, or:
<http://www.caliburn.nl/topposting.html>
 
Reply With Quote
 
 
 
 
Walter Roberson
Guest
Posts: n/a
 
      01-11-2008
In article <fm6ajv$pat$(E-Mail Removed)>, jacob navia <(E-Mail Removed)> wrote:
>Flash Gordon wrote:


>> Of course, if you do not call main recursively the compiler might decide
>> to allocate foo where it allocates "global" variables. I can see reasons
>> why this optimisation might make sense.


>You are daydreaming.
>I bet you can't name a single compiler that does that kind of nonsense!


Thad Smith indicated their existance yesterday, in the context
of a thread where the original poster had indicated that the system
in use "had no stack".

http://groups.google.ca/group/comp.l...4174f98d5d485a
--
"Is there any thing whereof it may be said, See, this is new? It hath
been already of old time, which was before us." -- Ecclesiastes
 
Reply With Quote
 
Walter Roberson
Guest
Posts: n/a
 
      01-11-2008
In article <(E-Mail Removed)>,
<(E-Mail Removed)> wrote:
>Interesting.


>I know that the stack grows down from the high addresses while the
>heap grows up the address space


On some implementations. On other implementations, it is the
opposite. Some implementations have them both grow in the same
direction. And on yet other implementations, there is no stack as such.

--
"Any sufficiently advanced bug is indistinguishable from a feature."
-- Rich Kulawiec
 
Reply With Quote
 
William Ahern
Guest
Posts: n/a
 
      01-11-2008
(E-Mail Removed) <(E-Mail Removed)> wrote:
<snip>
> It's relatively rare that a program can successfully recover from a
> failing memory allocation. Even when a program does it, it's usually
> only for a very limited portion of the allocations (for example,
> storage allocated for buffering a large data file). In most cases
> avoidance is a much better idea.


Defining "rare" can be difficult. For the application domains I'm involved
with (network application servers/daemons), after my programs enter their
event loops malloc() could fail all day long, sporadically or continually,
without necessitating that the application exit. Exiting would be the last
thing I'd want. It got that far; why would I want to chance not being to get
that far again on re-instantiation of the program?

I do regression tests on memory failure recovery, which works well for me
since most of the error paths for memory allocation failure overlap w/
general application error paths. So I can just interpose malloc() and
interject random allocation failures. Much easier than simulating arbitrary
network exchanges.

I also try to minimize stack use and avoid recursive function calls. My
async I/O library supports recursion but implements a poorman's tail call
device which limits the depth (no shenanigans; pure C).

As a practical matter, on many systems, notwithstanding any VM paging issues
or the like, there are guarantees that you can rely on regarding the stack.
On Unix you can (or have the capacity to, w/ proper configuration) guarantee
that you have a stack of, say, roughly 512K, or 1MB. And by limiting
recursion, as a practical matter, you can indeed assure yourself that you
won't overrun the stack. This is without overly intimate knowledge of the
implementation. Just due diligence and testing. (And I have seen Unix
programs which can actually do proper cleanup on stack overruns, in
conjunction w/ Unix' sigaltstack() facility, so on some systems stack
overruns aren't always a total wash, either.)

Memory management might be a pain, but its also a sort of environmental
stimulus which forces application design and implementation to make good
measure of the efficacy and necessity of various internal devices and
methods. If you don't want to deal with it, then don't use C. Use another
language, maybe a GC'd language. When memory management is merely a burden,
then C is manifestly an inappropriate tool. If you find yourself ignoring
the return value of malloc(), that's about the clearest signal possible that
your project is better implemented with a different tool.

I prefer Lua, which is noteworthy itself because, among other reasons, it
actually does check for memory failures in its ANSI C implementation, and
can throw VM exceptions (if you chose). Imagine that!

So, yes, there are people who actually manage memory, and they're not
confined to rarefied (by some measure) embedded platforms.

 
Reply With Quote
 
jacob navia
Guest
Posts: n/a
 
      01-11-2008
Walter Roberson wrote:
> In article <fm6ajv$pat$(E-Mail Removed)>, jacob navia <(E-Mail Removed)> wrote:
>> Flash Gordon wrote:

>
>>> Of course, if you do not call main recursively the compiler might decide
>>> to allocate foo where it allocates "global" variables. I can see reasons
>>> why this optimisation might make sense.

>
>> You are daydreaming.
>> I bet you can't name a single compiler that does that kind of nonsense!

>
> Thad Smith indicated their existance yesterday, in the context
> of a thread where the original poster had indicated that the system
> in use "had no stack".
>
> http://groups.google.ca/group/comp.l...4174f98d5d485a


That's completely different. It is an embedded system where
there is no RAM for the stack. This is clearly NOT the
case of the user that started this thread.

Mentioning that some obscure implementation in some
embedded chip doesn't have a stack and doesn't implement
recursion doesn't mean that C is not recursive or that
most implementation do use a stack.

LET'S STOP BEING PEDANTIC!
We are speaking about stack overflows here.

http://dictionary.reference.com/browse/pedant

pedant

ľnoun
1.a person who makes an excessive or inappropriate display of learning.
2.a person who overemphasizes rules or minor details.
3.a person who adheres rigidly to book knowledge without regard to
common sense.


--
jacob navia
jacob at jacob point remcomp point fr
logiciels/informatique
http://www.cs.virginia.edu/~lcc-win32
 
Reply With Quote
 
Ian Collins
Guest
Posts: n/a
 
      01-11-2008
(E-Mail Removed) wrote:
> Hi,
>
> I know that it's good practise to check the returned pointer from
> malloc() against NULL when making large allocations on the heap, so as
> to be able to attempt recovery in case of an out-of-memory error.
>
> But my question is: what about if you run out of memory while trying
> to allocated things on the stack, e.g. a big local array variable? Is
> there some way to hook in a function to run in these circumstances, to
> be able to try to clean up?
>

Avoid the problem rather than trying to fix it, use dynamic allocation
for large objects. Then you can check for failure.

Your implementation should document the system stack size limits
(assuming it has a stack!).

--
Ian Collins.
 
Reply With Quote
 
Chris Torek
Guest
Posts: n/a
 
      01-11-2008
In article <(E-Mail Removed)>
<(E-Mail Removed)> wrote:
>I know that the stack grows down from the high addresses while the
>heap grows up the address space


It does some or all of the time on some systems, but sometimes
not (e.g., when using threads) on some systems, and never so on
other systems (where it grows in the other direction, or uses
the malloc() arena).

>- I wonder if it would be possible to have a #pragma or something
>similar that would tell the compiler "Hey I've got some very big
>local arrays, so I'm gonna need plenty of stack space - try to
>allow more room than usual for the stack at the expense of the
>heap"?


Some systems do provide an option of this sort (usually not via
#pragma -- more often it is a linker directive or a "magic" global
variable).

The biggest problem of all comes in when you use various extensions
like threads, so that there is no longer "a" stack, but rather many
stacks, at least one per thread. (One can also have two or more
stacks per thread, as in Forth-like systems, for instance.)
--
In-Real-Life: Chris Torek, Wind River Systems
Salt Lake City, UT, USA (40░39.22'N, 111░50.29'W) +1 801 277 2603
email: forget about it http://web.torek.net/torek/index.html
Reading email is like searching for food in the garbage, thanks to spammers.
 
Reply With Quote
 
Flash Gordon
Guest
Posts: n/a
 
      01-11-2008
jacob navia wrote, On 10/01/08 23:43:
> Flash Gordon wrote:
>> jacob navia wrote, On 10/01/08 20:27:
>>> You can have an idea of how much stack you are using if you do
>>> the following
>>>
>>> char *StartOfStack;
>>> int main(...)
>>> {
>>> char foo;
>>>
>>> StartOfStack = &foo;

>>
>> Of course, if you do not call main recursively the compiler might
>> decide to allocate foo where it allocates "global" variables. I can
>> see reasons why this optimisation might make sense.

>
> You are daydreaming.
> I bet you can't name a single compiler that does that kind of nonsense!


Since I don't rely on the behaviour you are claiming I have no reason to
check what the compilers I use do.

>>> }
>>>
>>> This will tell you approximately WHERE the stack begins.
>>>
>>> You can see how much you have used if in the function "fn" you do:
>>>
>>> int fn(void)
>>> {
>>> char foo;
>>> char *CurrentStack = &foo;
>>>
>>> size_t StackSize = CurrentStack - StartOfStack;

>>
>> Of course, if the stack grows in the opposite direction (as it does on
>> the PowerPC) it could give very unexpected results.

>
> And the fix is to take the absolute value... We are interested in the
> distance.


Where you hit the problems described below and the problem of systems
that implement the C calling conventions using something that is very
much like a heap.

>> Not to mention on embedded compilers which people have posted about as
>> recently as this week that do things like allocating *all* local
>> variables statically.

>
> Who cares?
>
> We are speaking about STACK OVERFLOW here, so those compilers are
> OFF TOPIC for this thread.


Either these implementations do not have a stack, in which case you
should stop complaining about people saying that C does not require a
stack (and admit that you were wrong) or they do have a stack so these
compilers are relevant (and you should admit that you were wrong). You
can't have it both ways.

>>> You can then TEST before you call a function that uses a lot of stack
>>> if you are beyond some limit.

>>
>> However you have no way of knowing (in general) what limit you need to
>> test against, so it does not help you.

>
> Of course you know, and if you don't you can measure it with a program
> like the above one.


At which point the system administrator changed your limits. Or you
upgrade your implementation and it changes the default allocation. Or
you are using a system where the stack is allowed to grow until it meets
the heap, so how much stack space is available depends on the current
heap usage (I've used such systems).

> #include <stdio.h>
> char *StartOfStack;
> int main(int argc,char *argv[])
> {
> char foo,*p;
> char m[8192*16];
> if (argc > 0)
> p = StartOfStack = &foo;
> else
> p = &foo;
> printf("Stack %d\n",StartOfStack-p);
> main(0,NULL);
> // Not reached
> }


It doesn't seem to reliable to me since the final output I get depends
on the optimisation options I pass to the compiler.

Stack 8130432
Segmentation fault (core dumped)

Stack 8374336
Segmentation fault (core dumped)

Stack 8381056
Segmentation fault (core dumped)

Stack 8374848
Segmentation fault (core dumped)

How do we know that on a real program the effects might not be even larger?

<snip>

>>> Of course this is very compiler specific too, but should work
>>> in most systems.

>>
>> Apart from PowerPC and related processors (because you have the stack
>> direction wrong), so it won't work on big iron (or even relatively
>> small servers like those we run our accounts system on in my company)
>> from IBM or Apples from before they switched to x86 processors, or the
>> afore mentioned embedded compilers...

>
> HELLO HELLO
>
> This thread is about stack overflows!
>
> We are speaking about stack overflows here.


Actually I thought we were talking about running out of memory on the
stack. Since you always claim that C requires a stack even if it is not
implemented as a hardware stack must include all the systems everyone
has mentioned. It also includes the other types of system I have just
mentioned (one of which can occur on systems with a hardware stack).

> http://dictionary.reference.com/browse/pedant


Ah, so now you are changing tactics and calling everyone who disagrees
with you and provides evidence of your errors a pedant.
--
Flash Gordon
 
Reply With Quote
 
Ben Pfaff
Guest
Posts: n/a
 
      01-11-2008
(E-Mail Removed) writes:

> I know that it's good practise to check the returned pointer from
> malloc() against NULL when making large allocations on the heap, so as
> to be able to attempt recovery in case of an out-of-memory error.


I would contend that you should do so for every call to malloc(),
not just the calls that request a large amount of memory.
--
char a[]="\n .CJacehknorstu";int putchar(int);int main(void){unsigned long b[]
={0x67dffdff,0x9aa9aa6a,0xa77ffda9,0x7da6aa6a,0xa6 7f6aaa,0xaa9aa9f6,0x11f6},*p
=b,i=24;for(;p+=!*p;*p/=4)switch(0[p]&3)case 0:{return 0;for(p--;i--;i--)case+
2:{i++;if(i)break;else default:continue;if(0)case 1utchar(a[i&15]);break;}}}
 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
C/C++ compilers have one stack for local variables and return addresses and then another stack for array allocations on the stack. Casey Hawthorne C Programming 3 11-01-2009 08:23 PM
Recovering Memory Card Data Davy Digital Photography 1 09-10-2007 08:19 PM
Recovering Memory Collins Computer Support 6 06-01-2006 10:52 PM
Out Of Memory Error and Stack Size karthikeyan.jambulingam@gmail.com Java 11 02-17-2006 03:05 PM
JVM thread stack (out of memory) Chenxi Java 2 11-30-2005 12:36 PM



Advertisments