«

On Jun 16, 3:16 pm, "Alf P. Steinbach" <al...@start.no> wrote:
> [thread cross-posted to comp.lang.c++ and comp.lang.c]
>
> * Paul Hsieh
>
> > On Jun 16, 12:10 pm, Ian Collins <ian-n...@hotmail.com> wrote:
> >> Paul Hsieh wrote:
>
> >>> Using new and delete invoke
> >>> constructors which you might not want to happen.
>
> No, as class designer you have full control over that. Or from a class user
> perpective, if the class in question has at least one user defined constructor,
> you usually really want some constructor invoked, so that you have some
> invariants established. What C++ does here is to automate and check what you'd
> do manually in C, and automation is great.

Its also enforced and the timing has to be at or *before* the time of
first instantiation, unless you do run-time deference tricks (which
have a performance and design impact.)

> > Furthermore, its
> >>> easy to show that STL's vector templates have either ridiculously bad
> >>> performance in comparison to hand managed realloc()'s precisely
> >>> because of the RAII overhead or else compromise your design to the
> >>> point that you might as well use realloc().
> >> Care to demonstrate?
>
> > Sure. Lets make a class of mail messages. Note that its impossible
> > to have an empty mail message (because there is always at least a
> > header), hence a mail message can only be initialized based on some
> > input text stream or string; there is no well defined concept of a
> > default mail message constructor. Further it makes very little sense
> > to mutate a mail message by changing its contents after the fact. So
> > its a well motivated read-only class without an empty or default
> > constructor.
>
> Sure.
>
> > Now lets say you want to have a dynamic vector of mail messages (this
> > is exactly what you would expect a deserialized mailbox essentially to
> > be). The implementation of STL vectors require that the class have a
> > default constructor if the vector is modified (which it would be as a
> > result of incrementally reading the mailbox).
>
> Direct use of a vector would probably be inappropriate, but accepting that for
> the sake of argument.
>
> Then, sorry, the information you have is incorrect: std::vector has no
> requirement of a default constructor for the element type.
>
> The C++98 requirement of a standard container element class is that it is
> assignable[1] and copy constructible.

Even for a mutatible vector? I am pretty sure MSVC and WATCOM C/C++
both have problems with this and for good reason. You definitely do
not want to implement a vector as a linked list.

> > There are numerous work arounds to this such as creating a wrapper
> > class which does have an empty constructor which hides a pointer to a
> > mail message class that starts out NULL.
>
> Hm, now you're talking about a vector of pointers. That imposes no requirements
> on the class pointed to. Pointers are already assignable and copy constructible.

That's why I called it a work around.

> > But individual new()s to
> > each one is still going to take extra overhead (performance + memory)
> > so you would prefer to point into a memory pool of your own which you
> > maintain with malloc() or realloc() anyways,
>
> It seems you're now talking about a free list or more general custom allocator,
> and mixing the requirements of the memory allocator abstraction level with the
> requirements of the C++ class type object abstraction level.

Uhh ... I was just hoping that C++'s std::vector did some "magic" that
made it as fast as I can do with late initialization and realloc()
without requiring I essentially perform work equal or worse than doing
it the C way in the first place.

It turns out, of course, that MSVC and WATCOM C/C++ do no such
thing. When it does a resize it allocates new space, and
simultaneously instantiating extra entries to mitigate constant
resizing thrashing, then copies the old contents into the early part
of its buffer. The first part of this appears to instantiate the
default constructor for your objects before it does the eventual copy
that you want performed.

> That confusion is unfortunately easy to be led into when programming in either C
> or C++, because C does not (let you) properly restrict you, and C++ accepts
> almost all of C as a subset, modulo some teeny tiny small differences.

Well in this case, I don't understand what you are saying. I was
*TRYING* to let C++ solve my problems for me. It didn't and I had to
learn some really dirty grungy details of STL implementations just to
know why. Abstractions only really save you when they work.

> So when one's first instinct is to Do It Myself then the toolset the language
> presents to you overwhelmingly consists of the Wrong Tools, such as exploiting
> class level information at the allocator level. Choosing the Right Tools from
> that multitude of apparently plausible tools, is difficult, especially, I think,
> when one has been misinformed. At the allocator level you should only be dealing
> with untyped raw chunks of memory.
>
> With proper use of C++ you do the custom allocation stuff by defining, for the
> class, a custom allocation function (unfortunately called 'operator new') and a
> ditto custom deallocation function, which deal with untyped raw storage.

And if you want your "allocation" to be a side effect of a vector
block allocation? The point is that I don't get to use STL's
vectors. I get to make up my own vector class if I have no default
constructor and I want the vector to be growable.

> Or you might, better, inherit from a class that does that, which means you can
> use a general already existing solution, such as e.g. the Loki small-object
> allocator -- one little base class specification, and all that stuff's taken
> care of.

I'm sure I'm not the first person to run into this problem, and there
must be plenty of "solutions" out there.

> > in which case you have
> > not saved or improved anything by using these C++ constructs.
>
> Right, if you mix too far apart abstraction levels then you get little or no
> advantage from the abstraction.
>
> And you can't fault a car for its tendency to run off the road or into walls
> when you, as opposed to others, drive it.
>
> Sorry, couldn't resist that nag.

Yeah, I guess so. I just must be such a poor programmer. Odd that I
have not had such problems in any other programming language that I
have ever encountered. I don't think even Ada is as twisted as C++
for things like this.

--
Paul Hsieh
http://www.pobox.com/~qed/
http://bstring.sf.net/

Paul Hsieh wrote:

> On Jun 16, 3:16 pm, "Alf P. Steinbach" <al...@start.no> wrote:
>> [thread cross-posted to comp.lang.c++ and comp.lang.c]
>>
>> * Paul Hsieh
>>
>> > On Jun 16, 12:10 pm, Ian Collins <ian-n...@hotmail.com> wrote:
>> >> Paul Hsieh wrote:
>>
>> >>> Using new and delete invoke
>> >>> constructors which you might not want to happen.
>>
>> No, as class designer you have full control over that. Or from a class
>> user perpective, if the class in question has at least one user defined
>> constructor, you usually really want some constructor invoked, so that
>> you have some invariants established. What C++ does here is to automate
>> and check what you'd do manually in C, and automation is great.
>
> Its also enforced and the timing has to be at or *before* the time of
> first instantiation, unless you do run-time deference tricks (which
> have a performance and design impact.)
>
>> > Furthermore, its
>> >>> easy to show that STL's vector templates have either ridiculously bad
>> >>> performance in comparison to hand managed realloc()'s precisely
>> >>> because of the RAII overhead or else compromise your design to the
>> >>> point that you might as well use realloc().
>> >> Care to demonstrate?
>>
>> > Sure. Lets make a class of mail messages. Note that its impossible
>> > to have an empty mail message (because there is always at least a
>> > header), hence a mail message can only be initialized based on some
>> > input text stream or string; there is no well defined concept of a
>> > default mail message constructor. Further it makes very little sense
>> > to mutate a mail message by changing its contents after the fact. So
>> > its a well motivated read-only class without an empty or default
>> > constructor.
>>
>> Sure.
>>
>> > Now lets say you want to have a dynamic vector of mail messages (this
>> > is exactly what you would expect a deserialized mailbox essentially to
>> > be). The implementation of STL vectors require that the class have a
>> > default constructor if the vector is modified (which it would be as a
>> > result of incrementally reading the mailbox).
>>
>> Direct use of a vector would probably be inappropriate, but accepting
>> that for the sake of argument.
>>
>> Then, sorry, the information you have is incorrect: std::vector has no
>> requirement of a default constructor for the element type.
>>
>> The C++98 requirement of a standard container element class is that it is
>> assignable[1] and copy constructible.
>
> Even for a mutatible vector?

No not even for those.

> I am pretty sure MSVC and WATCOM C/C++
> both have problems with this and for good reason. You definitely do
> not want to implement a vector as a linked list.

No need for that. You seem to think that reallocation of the vectors
contents requires default construction of some elements (probably because
you think that vector uses new internally). This is not the case. When a
vector resizes, it allocates new memory through an allocator; by the
default, that is std::allocator<T>. The allocator itself obtains raw memory
(the default allocator uses operator new for that, which is not the same as
new; in particular, operator new does not construct anything). Then the
vector copy-constructs the given elements into that raw memory. Finally, it
destroys the elements at the previous location.

I
>> > There are numerous work arounds to this such as creating a wrapper
>> > class which does have an empty constructor which hides a pointer to a
>> > mail message class that starts out NULL.
>>
>> Hm, now you're talking about a vector of pointers. That imposes no
>> requirements on the class pointed to. Pointers are already assignable and
>> copy constructible.
>
> That's why I called it a work around.
>
>> > But individual new()s to
>> > each one is still going to take extra overhead (performance + memory)
>> > so you would prefer to point into a memory pool of your own which you
>> > maintain with malloc() or realloc() anyways,
>>
>> It seems you're now talking about a free list or more general custom
>> allocator, and mixing the requirements of the memory allocator
>> abstraction level with the requirements of the C++ class type object
>> abstraction level.
>
> Uhh ... I was just hoping that C++'s std::vector did some "magic" that
> made it as fast as I can do with late initialization and realloc()
> without requiring I essentially perform work equal or worse than doing
> it the C way in the first place.
>
> It turns out, of course, that MSVC and WATCOM C/C++ do no such
> thing. When it does a resize it allocates new space, and
> simultaneously instantiating extra entries to mitigate constant
> resizing thrashing, then copies the old contents into the early part
> of its buffer. The first part of this appears to instantiate the
> default constructor for your objects before it does the eventual copy
> that you want performed.

That would not be standard compliant. The following compiles and executes
fine with g++ (as required by the standard). Please test it on your
compilers:

class X {

// private, unimplemented default constructor
// ==========================================
X ( void );

int x;

public:

X ( int i )
: x ( i )
{}

X ( X const & other )
: x ( other.x )
{}

X & operator= ( X const & rhs ) {
x = rhs.x;
return ( *this );
}

int get ( void ) const {
return ( x );
}

};

#include <vector>

int main ( void ) {
std::vector<X> iv;
for ( unsigned int i = 0; i < 1000; ++i ) {
iv.push_back( X(i) );
}
}


>> That confusion is unfortunately easy to be led into when programming in
>> either C or C++, because C does not (let you) properly restrict you, and
>> C++ accepts almost all of C as a subset, modulo some teeny tiny small
>> differences.
>
> Well in this case, I don't understand what you are saying. I was
> *TRYING* to let C++ solve my problems for me. It didn't and I had to
> learn some really dirty grungy details of STL implementations just to
> know why. Abstractions only really save you when they work.

I really wonder what happened. Maybe, you should switch to an implementation
of the STL that actually implements the standard.


>> So when one's first instinct is to Do It Myself then the toolset the
>> language presents to you overwhelmingly consists of the Wrong Tools, such
>> as exploiting class level information at the allocator level. Choosing
>> the Right Tools from that multitude of apparently plausible tools, is
>> difficult, especially, I think, when one has been misinformed. At the
>> allocator level you should only be dealing with untyped raw chunks of
>> memory.
>>
>> With proper use of C++ you do the custom allocation stuff by defining,
>> for the class, a custom allocation function (unfortunately called
>> 'operator new') and a ditto custom deallocation function, which deal with
>> untyped raw storage.
>
> And if you want your "allocation" to be a side effect of a vector
> block allocation?

Then, you can use a custom allocator instead of the default
std::allocator<T>. There are pooling allocators and other tricks to
increase performance.


> The point is that I don't get to use STL's
> vectors. I get to make up my own vector class if I have no default
> constructor and I want the vector to be growable.
[snip]

See above: I cannot reproduce your problem. Could you paste some code that
illustrates it?


Best

Kai-Uwe Bux

On Jun 17, 2:14 am, Paul Hsieh <websn...@gmail.com> wrote:
> On Jun 16, 3:16 pm, "Alf P. Steinbach" <al...@start.no> wrote:
> > * Paul Hsieh

> > > On Jun 16, 12:10 pm, Ian Collins <ian-n...@hotmail.com> wrote:
> > >> Paul Hsieh wrote:

> > >>> Using new and delete invoke
> > >>> constructors which you might not want to happen.

> > No, as class designer you have full control over that. Or
> > from a class user perpective, if the class in question has
> > at least one user defined constructor, you usually really
> > want some constructor invoked, so that you have some
> > invariants established. What C++ does here is to automate
> > and check what you'd do manually in C, and automation is
> > great.

> Its also enforced and the timing has to be at or *before* the
> time of first instantiation, unless you do run-time deference
> tricks (which have a performance and design impact.)

I'm not sure I understand what you are saying. In C++, if a
class needs a constructor, the implementor of the class provides
it, and it will be called. And the code wouldn't work correctly
if it wasn't. If the class doesn't need a constructor, the
author of the class doesn't provide it, and there's basically no
difference with respect to C.

> > > Furthermore, its
> > >>> easy to show that STL's vector templates have either ridiculously bad
> > >>> performance in comparison to hand managed realloc()'s precisely
> > >>> because of the RAII overhead or else compromise your design to the
> > >>> point that you might as well use realloc().

Actual measurements on real implementations don't bear that out.
For dynamically sized arrays, std::vector is typically
considerably faster than anything you can do with
malloc/realloc.

> > >> Care to demonstrate?

> > > Sure. Lets make a class of mail messages. Note that its
> > > impossible to have an empty mail message (because there is
> > > always at least a header), hence a mail message can only
> > > be initialized based on some input text stream or string;
> > > there is no well defined concept of a default mail message
> > > constructor. Further it makes very little sense to mutate
> > > a mail message by changing its contents after the fact.
> > > So its a well motivated read-only class without an empty
> > > or default constructor.

> > Sure.

> > > Now lets say you want to have a dynamic vector of mail
> > > messages (this is exactly what you would expect a
> > > deserialized mailbox essentially to be). The
> > > implementation of STL vectors require that the class have
> > > a default constructor if the vector is modified (which it
> > > would be as a result of incrementally reading the
> > > mailbox).

> > Direct use of a vector would probably be inappropriate, but
> > accepting that for the sake of argument.

> > Then, sorry, the information you have is incorrect:
> > std::vector has no requirement of a default constructor for
> > the element type.

> > The C++98 requirement of a standard container element class
> > is that it is assignable[1] and copy constructible.

> Even for a mutatible vector? I am pretty sure MSVC and WATCOM
> C/C++ both have problems with this and for good reason. You
> definitely do not want to implement a vector as a linked list.

VC++ certainly doesn't. I often have vectors of objects without
default constructors, and in code which compiles with Sun CC,
g++ and VC++. (This has worked at least since VC++ 6.0, since
g++ 2.95.2, and since Sun CC 5.1. And those are all very old
compilers. In fact, I've never seen an implementation of the
STL where it didn't work.)

> > > There are numerous work arounds to this such as creating a
> > > wrapper class which does have an empty constructor which
> > > hides a pointer to a mail message class that starts out
> > > NULL.

> > Hm, now you're talking about a vector of pointers. That
> > imposes no requirements on the class pointed to. Pointers
> > are already assignable and copy constructible.

> That's why I called it a work around.

> > > But individual new()s to each one is still going to take
> > > extra overhead (performance + memory) so you would prefer
> > > to point into a memory pool of your own which you maintain
> > > with malloc() or realloc() anyways,

I'm not sure I understand the objection here, either. Mail
messages are going to have variable lengths, so you can't put
them directly (as an image of the message) into a vector or a C
style array. In C, you'd probably have to use something like
char*[], with careful memory management when copying, etc. In
C++, the simplest solution would be to use
std::vector< MailMessage >, with MailMessage basically a wrapper
around std::string to start with; if the copying does end up
being too expensive, then you can easily fix it.

From actual experience: if there's any risk of performance being
an issue, you must encapsulate. The result is that if there's
any risk of performance being an issue, C++ is essential.

> > It seems you're now talking about a free list or more
> > general custom allocator, and mixing the requirements of the
> > memory allocator abstraction level with the requirements of
> > the C++ class type object abstraction level.

> Uhh ... I was just hoping that C++'s std::vector did some
> "magic" that made it as fast as I can do with late
> initialization and realloc() without requiring I essentially
> perform work equal or worse than doing it the C way in the
> first place.

It does. In fact, because it has been optimized by some real
experts, it typically does a lot better than you or I could do.

> It turns out, of course, that MSVC and WATCOM C/C++ do no
> such thing. When it does a resize it allocates new space, and
> simultaneously instantiating extra entries to mitigate
> constant resizing thrashing, then copies the old contents into
> the early part of its buffer. The first part of this appears
> to instantiate the default constructor for your objects before
> it does the eventual copy that you want performed.

What makes you think that? It most certainly doesn't. (At
least VC++ doesn't, nor does any C++ implementation that I've
ever seen.)

--
James Kanze (GABI Software) email:
Conseils en informatique orientée objet/
Beratung in objektorientierter Datenverarbeitung
9 place Sémard, 78210 St.-Cyr-l'École, France, +33 (0)1 30 23 00 34

On Jun 16, 11:00 pm, Paul Hsieh <websn...@gmail.com> wrote:
> On Jun 16, 12:48 pm, "Bo Persson" <b...@gmb.dk> wrote:
> > Paul Hsieh wrote:
> > > On Jun 13, 8:15 pm, Keith Thompson <ks...@mib.org> wrote:
> > >> Paul Hsieh <websn...@gmail.com> writes:
> > >>> It also
> > >>> forces you to be more exact in function declarations. This I
> > >>> found to be the biggest actual source code impact, as it
> > >>> basically forces you to cast all mallocs.

> > >> Right (but it's usually better practice to use new and delete in
> > >> C++ anyway, or some STL type that manages memory for you).

> > > C++ is built on the RAII principle. Using new and delete invoke
> > > constructors which you might not want to happen. Furthermore, its
> > > easy to show that STL's vector templates have either ridiculously
> > > bad performance in comparison to hand managed realloc()'s precisely
> > > because of the RAII overhead or else compromise your design to the
> > > point that you might as well use realloc().

> > Constructors are invoked for types that have constructors. How do you
> > do that with realloc?

> You don't. That's precisely the point I am trying to make.

In other words, you write code that doesn't work. Or you
reimplement all of std::vector yourself.

> > > I.e., its not surprising that malloc/free has not been and
> > > will not be deprecated in the C++.

Yes. They're there for the cases where you have to interface
with C code.

> > Is it?

> Well, why don't you try and see what the reaction amongst real
> world programmers or compiler vendors is? More seriously,
> take a survey, look at real code and find out for yourself.

The reaction is that nobody uses malloc or free except when they
have to interface with legacy software.

> Often C++'s power can be used to wrap "unsafe/gross" calls to
> malloc and realloc anyways. I think that sort of flexibility
> was intentional anyways to make sure someone couldn't complain
> about capabilities taken away by C++. This is exactly what is
> done in Bstrlib, and I wouldn't be surprised if many STL's use
> realloc in the guts of their std::string class. It would be a
> little hard to take away malloc/ realloc in view of this.

Why don't you actually find out what is going on, instead of
just speculating about it. The standard requires that all
allocations in the STL which use a default allocator go through
operator new(). And at least the implementations I have access
to (Sun CC, g++ and VC++) do.

--
James Kanze (GABI Software) email:
Conseils en informatique orientée objet/
Beratung in objektorientierter Datenverarbeitung
9 place Sémard, 78210 St.-Cyr-l'École, France, +33 (0)1 30 23 00 34

On 16 Jun, 21:52, Paul Hsieh <websn...@gmail.com> wrote:
> On Jun 16, 12:10 pm, Ian Collins <ian-n...@hotmail.com> wrote:
> > Paul Hsieh wrote:
> > > On Jun 13, 8:15 pm, Keith Thompson <ks...@mib.org> wrote:
> > >> Paul Hsieh <websn...@gmail.com> writes:

<snip>

> > >> ([...] it's usually better practice to use new and delete in C++
> > >> anyway, or some STL type that manages memory for you).
>
> > > C++ is built on the RAII principle.
>
> > Is it?
>
> All struct or class declarations invoke whatever the default
> constructor it has. *Furthermore any class with a constructor must
> invoke a constructor at the time of declaration. *That's basically
> what RAII is -- a method of synchronizing allocation and
> initialization.
>
> > > Using new and delete invoke
> > > constructors which you might not want to happen. *Furthermore, its
> > > easy to show that STL's vector templates have either ridiculously bad
> > > performance in comparison to hand managed realloc()'s precisely
> > > because of the RAII overhead or else compromise your design to the
> > > point that you might as well use realloc().
>
> > Care to demonstrate?
>
> Sure. *Lets make a class of mail messages. *Note that its impossible
> to have an empty mail message (because there is always at least a
> header), hence a mail message can only be initialized based on some
> input text stream or string; there is no well defined concept of a
> default mail message constructor. *Further it makes very little sense
> to mutate a mail message by changing its contents after the fact. *So
> its a well motivated read-only class without an empty or default
> constructor.
>
> Now lets say you want to have a dynamic vector of mail messages (this
> is exactly what you would expect a deserialized mailbox essentially to
> be). *The implementation of STL vectors require that the class have a
> default constructor if the vector is modified (which it would be as a
> result of incrementally reading the mailbox).
>
> There are numerous work arounds to this such as creating a wrapper
> class which does have an empty constructor which hides a pointer to a
> mail message class that starts out NULL. *But individual new()s to
> each one is still going to take extra overhead (performance + memory)
> so you would prefer to point into a memory pool of your own which you
> maintain with malloc() or realloc() anyways, in which case you have
> not saved or improved anything by using these C++ constructs.
>

what's wrong with this? (I know its not exception safe)

class MailMsg
{
public:
MailMsg(istream&);

private:
unsigned char *data_block;
};

typedef vector<MailMsg*> MailMsgList;


void handle_mail(istream& mail_box)
{
MailMsgList current_mail;

/* load mail */
while (more_mail (mail_box))
{
MailMsg* next_msg = new MailMsg(mail_box);
current_mail.push_back(next_msg);
}
}


--
Nick Keighley

On 17 Jun., 10:47, James Kanze <james.ka...@gmail.com> wrote:
> On Jun 17, 2:14 am, Paul Hsieh <websn...@gmail.com> wrote:
[snip]
> > Uhh ... I was just hoping that C++'s std::vector did some
> > "magic" that made it as fast as I can do with late
> > initialization and realloc() without requiring I essentially
> > perform work equal or worse than doing it the C way in the
> > first place.
>
> It does. *In fact, because it has been optimized by some real
> experts, it typically does a lot better than you or I could do.
[snip]

I can second that. The last time I had to export some C-code into a C+
+ project, I basically compiled it without anything but trivial
problems (malloc returning void was the primary problem).
In the second phase, I removed a lot of custom code, replacing it with
std::string and std::vector. This made the performance somewhat better
(not much as the bottleneck was elsewhere), and the code a lot
cleaner.
I will not argue that there might be cases, where a custom string-
class or (more rarely) a custom std::vector could improve performance,
but in the dominating number of cases this will not be so, and in the
rest of these, the performance hit will probably not really be
something worth bothering about, so you will almost always start with
std::vector or std::string and only change strategy when measurements
show that these elements are the culprit. The exception is when you
have relatively small, fixed-size vectors. Boost has code for these.

Peter

On Jun 17, 1:04 pm, peter koch <peter.koch.lar...@gmail.com> wrote:
> On 17 Jun., 10:47, James Kanze <james.ka...@gmail.com> wrote:> On Jun 17, 2:14 am, Paul Hsieh <websn...@gmail.com> wrote:
> [snip]

> I will not argue that there might be cases, where a custom
> string- class or (more rarely) a custom std::vector could
> improve performance, but in the dominating number of cases
> this will not be so, and in the rest of these, the performance
> hit will probably not really be something worth bothering
> about, so you will almost always start with std::vector or
> std::string and only change strategy when measurements show
> that these elements are the culprit. The exception is when you
> have relatively small, fixed-size vectors. Boost has code for
> these.

In this regard, if the abstraction in question is central to
your application, you should wrap the use of the standard
classes in your own class, which should provide exactly the
interface needed, and no more. In that way, if it should be
necessary to replace them, you can do so without having to
implement the full interface, and without modifying any of the
client code.

This is really, more than anything else, why C++ has less
performance problems than C. It supports better encapsulation,
and better encapsulation allows you to correct the performance
problems, once the profiler has shown where they were, without
doing a major rewrite.

--
James Kanze (GABI Software) email:
Conseils en informatique orientée objet/
Beratung in objektorientierter Datenverarbeitung
9 place Sémard, 78210 St.-Cyr-l'École, France, +33 (0)1 30 23 00 34