Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > C Programming > xmalloc string functions

Reply
Thread Tools

xmalloc string functions

 
 
Malcolm McLean
Guest
Posts: n/a
 
      01-27-2008

"Ian Collins" <(E-Mail Removed)> wrote in message
> Kelsey Bjarnason wrote:
>> [snips]
>>
>> On Sun, 27 Jan 2008 15:42:58 +0000, Malcolm McLean wrote:
>>
>>> That's exactly how xmalloc() works. Except that the user callback can
>>> never say "return null". User must either supply some memory or
>>> terminate the program.

>>
>> So the user - in this case, the coder using your function - has
>> absolutely no option to do something as basic as reducing the memory
>> allocation request and trying again, or scheduling the operation for
>> later processing and continuing on to some other action.
>>

> Doesn't that meet the requirements for "supply some memory"? That
> requirement appears sufficiently vague to cover just about anything!
>

Rescheduling would be possible. Reducing the amount allocated not, because
then caller would have to fall back on some alternative algorithm. That
tends not to make sense except in special situations, where malloc() is the
appropriate call.

--
Free games and programming goodies.
http://www.personal.leeds.ac.uk/~bgy1mm

 
Reply With Quote
 
 
 
 
Malcolm McLean
Guest
Posts: n/a
 
      01-27-2008

"Kelsey Bjarnason" <(E-Mail Removed)> wrote in message
> Yes, that's right, it's a trap door function: you can enter, but you can
> never leave, unless and until the memory is available. No error
> reporting, no recovery, no options at all. That is the intent of the
> code, as Malcolm has expressed more than once, that you *cannot*
> recover from any allocation failure. You get the memory, or you exit.
> Period.
>

Exactly. Which is what these string functions will do. They will never
return null for out of memory conditions.
>
> And does it do single quotes? Double quotes? Both?
>

The quote character is passed, together with the character to escape the
quotes (which can be the quote itself). It doesn't support different opening
and closing quotes, which maybe it ought.

> No, no, it makes error handling *easier*, because you don't ever need to
> *do* error handling. xmalloc does that for you - by ensuring you get the
> memory, or the app aborts. See how wonderful it is? No need for all
> that error handling code at the calling point. Also no way to ever do
> anything like, oh, reduce the allocation request and try again, or
> reschedule the operation for later, etc, etc, etc, but hey, it's not like
> any application would ever actually do things like that.
>

You've got it. If machine won't give us a hundred bytes, and the end user
cannot free up any more applications, what can we do, seriously? The
ambitious might attempt an emergency save in the callback function/
>
> Except it's of little use in toy apps. Such an app probably doesn't do a
> hell of a lot of complex allocations in the first place, so wouldn't
> benefit significantly from "easy malloc". A complex app where there may
> be thousands or millions of allocations done from hundreds or thousands
> of places in the code, now this would benefit from "easy malloc" - if it
> were actually useful.
>
> The xmalloc notion presented is of little interest in a toy app, and is
> unusable in a real app. One wonders where, exactly, it would actually be
> used. One place it won't be used: my code. I actually like to be able
> to choose a course of action on allocation failure, not be stuck with
> some asinine notion that allocation will succeed or there's no point in
> continuing.
>

Single user apps. Or database apps where there isn't a robust memory
recovery strategy in place - e.g. anything which uses an X server. Not for
an app where it doesn;t matter if you crash or not on the null pointer
dereference that will follow a failed, unchecked malloc(), not for an app
which absolutely must not terminate but can abort jobs if memory gets low.
Between that low end and high end there's an enormous number of programs.

--
Free games and programming goodies.
http://www.personal.leeds.ac.uk/~bgy1mm

 
Reply With Quote
 
 
 
 
William Ahern
Guest
Posts: n/a
 
      01-27-2008
Yevgen Muntyan <(E-Mail Removed)> wrote:
> Malcolm McLean wrote:
> > Here are six functions implemented on top of xmalloc(). No C programmer
> > should have any triouble providing the implemetations, though replace
> > and getquote are non-trivial.


> [snip]


> > I've think we've got something quite powerful here, purely because none
> > of these functions can ever return null for out of memory conditions. It
> > massively simplifies string handling.


> Take a look at glib,
> http://library.gnome.org/devel/glib/...llocation.html


glib is where bad ideas go to die. Now, if somebody just had the nerve to
tell them....

 
Reply With Quote
 
Yevgen Muntyan
Guest
Posts: n/a
 
      01-27-2008
William Ahern wrote:
> Yevgen Muntyan <(E-Mail Removed)> wrote:
>> Malcolm McLean wrote:
>>> Here are six functions implemented on top of xmalloc(). No C programmer
>>> should have any triouble providing the implemetations, though replace
>>> and getquote are non-trivial.

>
>> [snip]

>
>>> I've think we've got something quite powerful here, purely because none
>>> of these functions can ever return null for out of memory conditions. It
>>> massively simplifies string handling.

>
>> Take a look at glib,
>> http://library.gnome.org/devel/glib/...llocation.html

>
> glib is where bad ideas go to die. Now, if somebody just had the nerve to
> tell them....


There are at least two desktop environments built on glib, and
bunch of independent applications, and they actually work. Have
you got something more substantial than "bad ideas"?

Best regards,
Yevgen
 
Reply With Quote
 
William Ahern
Guest
Posts: n/a
 
      01-27-2008
Yevgen Muntyan <(E-Mail Removed)> wrote:
> William Ahern wrote:
> > Yevgen Muntyan <(E-Mail Removed)> wrote:
> >> Malcolm McLean wrote:
> >>> Here are six functions implemented on top of xmalloc(). No C programmer
> >>> should have any triouble providing the implemetations, though replace
> >>> and getquote are non-trivial.

> >
> >> [snip]

> >
> >>> I've think we've got something quite powerful here, purely because none
> >>> of these functions can ever return null for out of memory conditions. It
> >>> massively simplifies string handling.

> >
> >> Take a look at glib,
> >> http://library.gnome.org/devel/glib/...llocation.html

> >
> > glib is where bad ideas go to die. Now, if somebody just had the nerve to
> > tell them....

>
> There are at least two desktop environments built on glib, and
> bunch of independent applications, and they actually work. Have
> you got something more substantial than "bad ideas"?


I only brought it up because glib uses an xmalloc() wrapper which dies on
allocation failure, and Evolution and similar glib-based applications loved
to crash when, for instance, I would [try to] open a message with an
especially long marked-up diff. Of course, it was worse (as in more
frequent) before I disabled the Linux OOM killer. Never infrequent enough,
however.

In my experience, glib-based applications "mostly" work. Not sure whether
that has more to do w/ the xmalloc() wrapper, or other issues. I'll grant
you that the organization and design of glib source interfaces is... a
matter of taste.

One of my rules of thumb is that if a network daemon uses glib, I
automatically exclude it from consideration. I can deal when an application
crashes and destroys my work. I don't want to be responsible for installing
an application which crashes and destroys or interrupts _other_ people's
work.

 
Reply With Quote
 
Randy Howard
Guest
Posts: n/a
 
      01-27-2008
On Sun, 27 Jan 2008 16:12:01 -0600, William Ahern wrote
(in article <h6cv65-grq.ln1@wilbur.25thandClement.com>):

> Yevgen Muntyan <(E-Mail Removed)> wrote:
>> Malcolm McLean wrote:
>>> Here are six functions implemented on top of xmalloc(). No C programmer
>>> should have any triouble providing the implemetations, though replace
>>> and getquote are non-trivial.

>
>> [snip]

>
>>> I've think we've got something quite powerful here, purely because none
>>> of these functions can ever return null for out of memory conditions. It
>>> massively simplifies string handling.

>
>> Take a look at glib,
>> http://library.gnome.org/devel/glib/...llocation.html

>
> glib is where bad ideas go to die. Now, if somebody just had the nerve to
> tell them....




You gdon't glike ghaving gall gyour gvariables gprexfed gwith g?



--
Randy Howard (2reply remove FOOBAR)
"The power of accurate observation is called cynicism by those
who have not got it." - George Bernard Shaw





 
Reply With Quote
 
Yevgen Muntyan
Guest
Posts: n/a
 
      01-27-2008
William Ahern wrote:
> Yevgen Muntyan <(E-Mail Removed)> wrote:
>> William Ahern wrote:
>>> Yevgen Muntyan <(E-Mail Removed)> wrote:
>>>> Malcolm McLean wrote:
>>>>> Here are six functions implemented on top of xmalloc(). No C programmer
>>>>> should have any triouble providing the implemetations, though replace
>>>>> and getquote are non-trivial.
>>>> [snip]
>>>>> I've think we've got something quite powerful here, purely because none
>>>>> of these functions can ever return null for out of memory conditions. It
>>>>> massively simplifies string handling.
>>>> Take a look at glib,
>>>> http://library.gnome.org/devel/glib/...llocation.html
>>> glib is where bad ideas go to die. Now, if somebody just had the nerve to
>>> tell them....

>> There are at least two desktop environments built on glib, and
>> bunch of independent applications, and they actually work. Have
>> you got something more substantial than "bad ideas"?

>
> I only brought it up because glib uses an xmalloc() wrapper which dies on
> allocation failure, and Evolution and similar glib-based applications loved
> to crash when, for instance, I would [try to] open a message with an
> especially long marked-up diff. Of course, it was worse (as in more
> frequent) before I disabled the Linux OOM killer. Never infrequent enough,
> however.


I'd think evolution crashed because it crashed. You can look at
its list of bugs to see what I mean. If it was abort() inside
g_malloc(), then it probably leaked so much that it indeed has
eaten everything and asked for more. In which case you can't
really blame glib
(Here I'd be actually grateful if such applications were killed
immediately, because they first freeze X, and I've got to wait
for ten minutes to switch to console and kill the application.
They just don't die themselves.)

>
> In my experience, glib-based applications "mostly" work. Not sure whether
> that has more to do w/ the xmalloc() wrapper, or other issues.


There is no xmalloc() wrapper in glib. Anyway, have you never
seen "mostly working" application which do not use glib? Bugs
are everywhere, on all platforms. Do you have a real base for
saying that g_malloc() is somehow responsible for crashes you
have seen? Something other than "evolution crashed", that is,
or "similar applications" (similar glib-based applications
which open messages, huh?).

You know, I have heard things like you are saying only from
people who talk about xmalloc() and related things. Never from
users. Why is that? Perhaps because buggy applications are
buggy applications, not some poor creatures crashing because
glib memory handling is broken?

> I'll grant
> you that the organization and design of glib source interfaces is... a
> matter of taste.


Yes it is, certainly. And it's irrelevant.

>
> One of my rules of thumb is that if a network daemon uses glib, I
> automatically exclude it from consideration. I can deal when an application
> crashes and destroys my work. I don't want to be responsible for installing
> an application which crashes and destroys or interrupts _other_ people's
> work.


It's fine, nobody promiced glib will work for any program. It certainly
won't; but nevertheless it doesn't make abort-on-failing-malloc less
sensible strategy for a whole class of applications.

Besides, a glib application can set up some sort of emergency memory
pool or something, so that failed malloc doesn't necessarily lead to
immediate abort(). Same sort of science fiction as "graceful exit with
saving data on *any* failed malloc() call in any possible application
in any possible situation" which seems to be so popular here

Best regards,
Yevgen
 
Reply With Quote
 
Yevgen Muntyan
Guest
Posts: n/a
 
      01-27-2008
Randy Howard wrote:
> On Sun, 27 Jan 2008 16:12:01 -0600, William Ahern wrote
> (in article <h6cv65-grq.ln1@wilbur.25thandClement.com>):
>
>> Yevgen Muntyan <(E-Mail Removed)> wrote:
>>> Malcolm McLean wrote:
>>>> Here are six functions implemented on top of xmalloc(). No C programmer
>>>> should have any triouble providing the implemetations, though replace
>>>> and getquote are non-trivial.
>>> [snip]
>>>> I've think we've got something quite powerful here, purely because none
>>>> of these functions can ever return null for out of memory conditions. It
>>>> massively simplifies string handling.
>>> Take a look at glib,
>>> http://library.gnome.org/devel/glib/...llocation.html

>> glib is where bad ideas go to die. Now, if somebody just had the nerve to
>> tell them....

>
>
>
> You gdon't glike ghaving gall gyour gvariables gprexfed gwith g?


Why, you don't like the following code?

#include <glib.h>

gint main (gint argc, gchar **argv)
{
gchar *s = g_strdup ("Hello there!");
g_print ("%s\n", s);
g_free (s);
}
 
Reply With Quote
 
Yevgen Muntyan
Guest
Posts: n/a
 
      01-27-2008
Kelsey Bjarnason wrote:
> On Sun, 27 Jan 2008 20:34:58 +0000, Yevgen Muntyan wrote:
>
>> Malcolm McLean wrote:
>>> Here are six functions implemented on top of xmalloc(). No C programmer
>>> should have any triouble providing the implemetations, though replace
>>> and getquote are non-trivial.

>> [snip]
>>
>>> I've think we've got something quite powerful here, purely because none
>>> of these functions can ever return null for out of memory conditions.
>>> It massively simplifies string handling.

>> Take a look at glib,
>> http://library.gnome.org/devel/glib/...llocation.html

>
> Oh, good God. They didn't. Tell me they didn't.
>
> One wonders how many applications they've screwed over with that bit of
> asinine idiocy.


One wonders why one wonders about that only after he learns about
g_malloc. Perhaps because those applications aren't actually screwed?
 
Reply With Quote
 
cr88192
Guest
Posts: n/a
 
      01-28-2008

"Eric Sosman" <(E-Mail Removed)> wrote in message
news:(E-Mail Removed)...
> Malcolm McLean wrote:
>> Here are six functions implemented on top of xmalloc(). No C programmer
>> should have any triouble providing the implemetations, though replace and
>> getquote are non-trivial.
>>
>> char *dup(const char *str);
>> char *cat(const char *str1, const char *str2);
>> char *catandkill(char *str1, const char *str2);
>> char *tok(const char *str, const char *delims, char **end);
>> char *midstr(const char *str, int idx, int len);
>> char *replace(const char *str, const char *pattern, const char *rep);
>> char *getquote(const char *str, char quote, char escape, char **end);
>> char *getline(FILE *fp);
>>
>> All return strings allocated with xmalloc().
>>
>> dup() is strdup(), and duplicates a string. cat() concatenates two
>> strings. catandkill concatenagtes two strings, and frees the first. It
>> also returns dup(str2) is the first pointer is null. It is designed for
>> building strings in conveneint loops.
>> tok() is strtok() that takes a string argument instead of using a global.
>> midstr() takes out a central portion of the string. replace() is a find
>> and replace, getquote() retrives the next quoted string, allowing for the
>> quotes themselves to be escaped. You will have to escape non-quote
>> escaped characters manually, of course. getline() will be a familiar
>> friend.

>
> The specifications are too weak to drive implementations.
> What does xmalloc() do when unable to allocate memory? Does
> replace() replace all occurrences, or just the first, or just
> the last, or what? What does getquote() do with unbalanced
> quotes? Does getline() keep or discard the '\n', and what does
> it do on an I/O error? (See the recent thread on design of line-
> input functions to see how divergent people's opinions are on
> how this "familiar friend" should behave.)
>
>> I've think we've got something quite powerful here, purely because none
>> of these functions can ever return null for out of memory conditions. It
>> massively simplifies string handling.

>
> ... and massively complicates error handling. If xmalloc()
> can unilaterally terminate the program, the whole suite is
> unusable except in toy programs. If there's a fancier framework,
> all functions that use the suite or that call other functions
> that might use the suite need to be plugged in to the framework.
>
> Design consists of more than writing a few declarations.
>


I usually use a garbage collector.


if the app runs out of memory (rare, since I usually use primarily static
memory-management approaches, but I can be lazy sometimes), the allocator
returns NULL.

now, crashing the app is just lazy, but it is possibly justified on the
grounds that on modern systems, running out of memory is rare enough that
this is justified (actually, it may well be justified to crash the app
before this limit, setting some sort of critical upper bound on memory
usage, and provoking the dev to reduce the footprint).

however, often NULL works just as well, because, as a general rule accessing
memory through a NULL pointer will crash the app anyways (most of the time,
address 0 being set to no-access).

this would be, assuming the issue is not handled.


better though, would be an exception handling system, namely, we crash only
if the out-of-memory exception goes unhandled. now, this is less nice in C,
which lacks a good exception system (nor the language features to really
implement one).

that is not to say, however, that a crude one could not be built around
setjmp/longjmp (library feature), or could be added, for example, as a
compiler syntax extension that rewrites exception handlers into the library
feature's form.


thus, in this case, an unhandled exception thus results in an abort (sadly,
an actual 'crash' at this point is pointless, since the context of the
exception is already lost).

of course, another approach would be threaded or recursive exception
handling (in C, however, there would be present other problems, and
semantics which would be problematic to reconcile with the unwinding
variety).

the system could, likely, be made to handle both, with some handlers set to
behave in a recursive manner (more like signal) and others in an unwinding
manner (longjmp). for semantic reasons, likely an unwinding handler would
silently ignore any previous recursive handlers (them being invoked in an
essentially meaningless context).

throw:
if last handler is signal-style, invoke callback;
otherwise, unwind to last handler.

handler:
if correct exception, do handling actions, and resume execution post
handler;
if there is another unwinding handler, unwind to that handler;
else, rethrow an unhandled exception.


> --
> Eric Sosman
> http://www.velocityreviews.com/forums/(E-Mail Removed)lid


 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
xmalloc.c - my xmalloc vippstar@gmail.com C Programming 11 02-20-2008 12:34 AM
xmalloc Malcolm McLean C Programming 52 06-27-2007 08:44 PM
Multi-threaded C++, new, and xmalloc Tim Hollingsworth Ruby 4 11-24-2006 05:02 PM
please help me in distinguish redefining functions, overloading functions and overriding functions. Xiangliang Meng C++ 1 06-21-2004 03:11 AM
[OT?] xmalloc Jeff C Programming 10 07-07-2003 05:14 PM



Advertisments