«

Bugs of memory allocation will make you mad.

Bugs *in* memory allocation will put you in the cuckoo people place.

On 2/16/2012 2:21 PM, Devil with the China Blue Dress wrote:
> In article<jhjr14$rjk$>, (Joe keane) wrote:
>
>> Bugs of memory allocation will make you mad.
>>
>> Bugs *in* memory allocation will put you in the cuckoo people place.
>
> Which is why they are exceedingly rare. Nearly all allocation problems are due
> to the program storing outside array bounds.
>

which are annoyingly difficult to track down sometimes...


it would be nice if compilers would offer an option (as a debugging
feature) to optionally put in bounds checking for many operations (it is
also possible to do this without adding fat pointers or fundamentally
changing how the language works, although it would still introduce a
run-time cost).

simple idea:
on a array access, identify the memory object pointed-to by the base
pointer (heap-lookup);
determine if the address being assigned to is the same object (possibly,
another heap lookup, or checking the target against the first object);
if not, blow up.

now, if the person can't afford the cost, they don't have to use the
feature.


some other use cases are a little harder, say:
double *fp;
fp=...
while(fp) { *fp=*fp+1; fp++; }

the issue would be that, potentially, the pointer could jump from one
object to the next without the run-time checks noticing.

a secondary defense could be to place "trip wires" in the heap between
objects, and if a memory-write check determines that the pointer points
to such a trip-wire, then an exception can be raised (one option for
such trip-wires is to have them occupy the same physical space as
memory-object headers or similar, and maybe also several for words
following the end of a memory-object).


although, without the explicit testing or throwing (kind of a problem
with a standard compiler), I have used similar before as a means to
attempt to debug array overruns with several custom memory managers of
mine (the memory managers will make some attempt to detect and
diagnose/report these sorts of problems).


or such...

On 2/16/2012 4:21 PM, Devil with the China Blue Dress wrote:
> In article<jhjr14$rjk$>, (Joe keane) wrote:
>
>> Bugs of memory allocation will make you mad.
>>
>> Bugs *in* memory allocation will put you in the cuckoo people place.
>
> Which is why they are exceedingly rare. Nearly all allocation problems are due
> to the program storing outside array bounds.

The only allocator bug I have personally encountered was with
a malloc() implementation that never, never returned NULL. When it
ought to have returned NULL, it crashed the program instead ...

--
Eric Sosman
d

On Feb 17, 2:27*am, Eric Sosman <esos...@ieee-dot-org.invalid> wrote:
> On 2/16/2012 4:21 PM, Devil with the China Blue Dress wrote:
>
> > In article<jhjr14$rj...@reader1.panix.com>, j...@panix.com (Joe keane) wrote:
>
> >> Bugs of memory allocation will make you mad.
>
> >> Bugs *in* memory allocation will put you in the cuckoo people place.
>
> > Which is why they are exceedingly rare. Nearly all allocation problems are due
> > to the program storing outside array bounds.
>
> * * *The only allocator bug I have personally encountered was with
> a malloc() implementation that never, never returned NULL. *When it
> ought to have returned NULL, it crashed the program instead ...

If you don't provide a way to verify that, it didn't happen , and
there was a bug in your code.

Goran.

On Feb 16, 10:04*pm, j...@panix.com (Joe keane) wrote:
> Bugs of memory allocation will make you mad.
>
> Bugs *in* memory allocation will put you in the cuckoo people place.

.... Provided you really encountered one. If you don't provide
verifiable evidence that you did, you didn't.

Goran.

Goran <> writes:
> On Feb 17, 2:27Â*am, Eric Sosman <esos...@ieee-dot-org.invalid> wrote:
>> On 2/16/2012 4:21 PM, Devil with the China Blue Dress wrote:
>>
>> > In article<jhjr14$rj...@reader1.panix.com>, j...@panix.com (Joe keane) wrote:
>>
>> >> Bugs of memory allocation will make you mad.
>>
>> >> Bugs *in* memory allocation will put you in the cuckoo people place.
>>
>> > Which is why they are exceedingly rare. Nearly all allocation problems are due
>> > to the program storing outside array bounds.
>>
>> Â* Â* Â*The only allocator bug I have personally encountered was with
>> a malloc() implementation that never, never returned NULL. Â*When it
>> ought to have returned NULL, it crashed the program instead ...
>
> If you don't provide a way to verify that, it didn't happen , and
> there was a bug in your code.

I suspect he's referring to the malloc() implementation
on typical Linux systems, which overcommits memory by default.
It can allocate a large chunk of address space for which no actual
memory is available. The memory isn't actually allocated until
the process attempts to access it. If there isn't enough memory
available for the allocation, the "OOM killer" kills some process
(not necessarily the one that did the allocation).

--
Keith Thompson (The_Other_Keith) kst- <http://www.ghoti.net/~kst>
Will write code for food.
"We must do something. This is something. Therefore, we must do this."
-- Antony Jay and Jonathan Lynn, "Yes Minister"

On Feb 17, 11:26*am, Keith Thompson <ks...@mib.org> wrote:
> Goran <goran.pu...@gmail.com> writes:
> > On Feb 17, 2:27*am, Eric Sosman <esos...@ieee-dot-org.invalid> wrote:
> >> On 2/16/2012 4:21 PM, Devil with the China Blue Dress wrote:
>
> >> > In article<jhjr14$rj...@reader1.panix.com>, j...@panix.com (Joe keane) wrote:
>
> >> >> Bugs of memory allocation will make you mad.
>
> >> >> Bugs *in* memory allocation will put you in the cuckoo people place..
>
> >> > Which is why they are exceedingly rare. Nearly all allocation problems are due
> >> > to the program storing outside array bounds.
>
> >> * * *The only allocator bug I have personally encountered was with
> >> a malloc() implementation that never, never returned NULL. *When it
> >> ought to have returned NULL, it crashed the program instead ...
>
> > If you don't provide a way to verify that, it didn't happen , and
> > there was a bug in your code.
>
> I suspect he's referring to the malloc() implementation
> on typical Linux systems, which overcommits memory by default.
> It can allocate a large chunk of address space for which no actual
> memory is available. *The memory isn't actually allocated until
> the process attempts to access it. *If there isn't enough memory
> available for the allocation, the "OOM killer" kills some process
> (not necessarily the one that did the allocation).

That crossed my mind, but what he said doesn't correspond with what
happens: malloc does return something and __doesn't__ crash the
program. OOM killer kills the code upon an attempt to access that
memory.

But given the way he explained it, it's possible that he's affected by
OOM killer, and he forgot, or never knew, what really happened.

Goran.

On 2/17/2012 2:40 AM, Goran wrote:
> On Feb 17, 2:27 am, Eric Sosman<esos...@ieee-dot-org.invalid> wrote:
>> On 2/16/2012 4:21 PM, Devil with the China Blue Dress wrote:
>>
>>> In article<jhjr14$rj...@reader1.panix.com>, j...@panix.com (Joe keane) wrote:
>>
>>>> Bugs of memory allocation will make you mad.
>>
>>>> Bugs *in* memory allocation will put you in the cuckoo people place.
>>
>>> Which is why they are exceedingly rare. Nearly all allocation problems are due
>>> to the program storing outside array bounds.
>>
>> The only allocator bug I have personally encountered was with
>> a malloc() implementation that never, never returned NULL. When it
>> ought to have returned NULL, it crashed the program instead ...
>
> If you don't provide a way to verify that, it didn't happen , and
> there was a bug in your code.

Yeah, it was probably my code. Awfully nice of DEC to fix it
for me by patching VMS' C library, don't you think?

--
Eric Sosman
d

Goran <> writes:
> On Feb 17, 11:26Â*am, Keith Thompson <ks...@mib.org> wrote:
[...]
>> I suspect he's referring to the malloc() implementation
>> on typical Linux systems, which overcommits memory by default.
>> It can allocate a large chunk of address space for which no actual
>> memory is available. Â*The memory isn't actually allocated until
>> the process attempts to access it. Â*If there isn't enough memory
>> available for the allocation, the "OOM killer" kills some process
>> (not necessarily the one that did the allocation).
>
> That crossed my mind, but what he said doesn't correspond with what
> happens: malloc does return something and __doesn't__ crash the
> program. OOM killer kills the code upon an attempt to access that
> memory.

If you call malloc() and it overcommits, it won't crash the
program until you access the allocated memory. (The rationale for
overcommitting is that most programs don't actually use most of
the memory the memory the allocate. I find that odd)

> But given the way he explained it, it's possible that he's affected by
> OOM killer, and he forgot, or never knew, what really happened.

Given the post you're responding to, I would find it more likely that
he knows exactly what happened and didn't mention it. Reading his
more recent followup, apparently it was on VMS, not Linux, and was
likely a bug in DEC's C library.

--
Keith Thompson (The_Other_Keith) kst- <http://www.ghoti.net/~kst>
Will write code for food.
"We must do something. This is something. Therefore, we must do this."
-- Antony Jay and Jonathan Lynn, "Yes Minister"

Keith Thompsonæ–¼ 2012å¹´2月18日星期å…*UTC+8上å�ˆ1時25分19秒 寫é�“：
> Goran <> writes:
> > On Feb 17, 11:26Â*am, Keith Thompson <ks...@mib.org> wrote:
> [...]
> >> I suspect he's referring to the malloc() implementation
> >> on typical Linux systems, which overcommits memory by default.
> >> It can allocate a large chunk of address space for which no actual
> >> memory is available. Â*The memory isn't actually allocated until
> >> the process attempts to access it. Â*If there isn't enough memory
> >> available for the allocation, the "OOM killer" kills some process
> >> (not necessarily the one that did the allocation).
> >
> > That crossed my mind, but what he said doesn't correspond with what
> > happens: malloc does return something and __doesn't__ crash the
> > program. OOM killer kills the code upon an attempt to access that
> > memory.
>
> If you call malloc() and it overcommits, it won't crash the
> program until you access the allocated memory. (The rationale for
> overcommitting is that most programs don't actually use most of
> the memory the memory the allocate. I find that odd)
>
> > But given the way he explained it, it's possible that he's affected by
> > OOM killer, and he forgot, or never knew, what really happened.
>
> Given the post you're responding to, I would find it more likely that
> he knows exactly what happened and didn't mention it. Reading his
> more recent followup, apparently it was on VMS, not Linux, and was
> likely a bug in DEC's C library.
>
> --
> Keith Thompson (The_Other_Keith) kst- <http://www.ghoti.net/~kst>
> Will write code for food.
> "We must do something. This is something. Therefore, we must do this."
> -- Antony Jay and Jonathan Lynn, "Yes Minister"

Nowadays malloc is too primitive for managing objects.

I have to compress some rarely used objects from time to time. Now the question is shared references which
sould be stored including the data state and dynamical access methods for switching between the native mode and the compressed mode in the heap space..