«

Dear C++ Experts,

Over the last couple of months I have been writing my first program
using shared memory. It has been something of an "in-at-the-deep-end"
experience, to say the least. At present the shared memory contains a
few fixed-size structs, but I really need to be able to store more
complex variable-sized data in there. So the next task is to work out
how to store C++ objects, and if possible STL containers, in this
shared region.

So far I have got the following bits working:

- Creating the shared memory region, and attaching to it from the
various processes. It can of course end up at different addresses in
the different processes, and this is a key problem.

- Basic C malloc()/free() style allocation in the region, using offsets
rather than pointers so that it works wherever it is mapped.

- Implementations of operator new, new[], delete and delete[] that use
these C-style primitives.

I now have two remaining challenges. First, I'd like to write an
offset<T> class that I can use in places where I would normally use T*.
I think that this is possible; I'll need to define operator*, a
conversion from T* to offset<T>, and a few other things. Any
suggestions would be welcome, though I think I would be able to work it
out eventually. My first decision is whether the offsets are relative
to the start of the region, in which case a "per-process global" is
needed to record the region start address, or they're relative to where
they are stored, which avoids the global but makes the implementation
harder.

For me the more difficult challenge is the STL allocators. I have only
the vaguest idea about this. Presumably I can create an "allocator
object" that the containers will use, without too much trouble (though
an example would be good). But can I persuade the containers that
they'd like to store my offsets, instead of real pointers?

I'm hoping that someone in this group will be able to either point me
to some good examples, or maybe convince me that it's impossible!
Which is it to be?

BTW this is for Anyterm: http://chezphil.org/anyterm/ - could be
interesting if you ever need to do remote server admin.

Cheers, Phil.

Hmm. Why not put just the pointer(s) into the shared region, and
dynamically alloc the container(s) (or any other object) from the heap?



<> wrote
> Dear C++ Experts,
>
> Over the last couple of months I have been writing my first program
> using shared memory. It has been something of an "in-at-the-deep-end"
> experience, to say the least. At present the shared memory contains a
> few fixed-size structs, but I really need to be able to store more
> complex variable-sized data in there. So the next task is to work out
> how to store C++ objects, and if possible STL containers, in this
> shared region.
>
> So far I have got the following bits working:
>
> - Creating the shared memory region, and attaching to it from the
> various processes. It can of course end up at different addresses in
> the different processes, and this is a key problem.
>
> - Basic C malloc()/free() style allocation in the region, using offsets
> rather than pointers so that it works wherever it is mapped.
>
> - Implementations of operator new, new[], delete and delete[] that use
> these C-style primitives.
>
> I now have two remaining challenges. First, I'd like to write an
> offset<T> class that I can use in places where I would normally use T*.
> I think that this is possible; I'll need to define operator*, a
> conversion from T* to offset<T>, and a few other things. Any
> suggestions would be welcome, though I think I would be able to work it
> out eventually. My first decision is whether the offsets are relative
> to the start of the region, in which case a "per-process global" is
> needed to record the region start address, or they're relative to where
> they are stored, which avoids the global but makes the implementation
> harder.
>
> For me the more difficult challenge is the STL allocators. I have only
> the vaguest idea about this. Presumably I can create an "allocator
> object" that the containers will use, without too much trouble (though
> an example would be good). But can I persuade the containers that
> they'd like to store my offsets, instead of real pointers?
>
> I'm hoping that someone in this group will be able to either point me
> to some good examples, or maybe convince me that it's impossible!
> Which is it to be?
>
> BTW this is for Anyterm: http://chezphil.org/anyterm/ - could be
> interesting if you ever need to do remote server admin.
>
> Cheers, Phil.
>

wrote:
....
>
> I now have two remaining challenges. First, I'd like to write an
> offset<T> class that I can use in places where I would normally use T*.

See austria's "relative pointer"

http://austria.sourceforge.net/dox/h...8h-source.html

It's behaviour is compiler implementation dependant but I think it will
work with most compilers.

....
>
> I'm hoping that someone in this group will be able to either point me
> to some good examples, or maybe convince me that it's impossible!
> Which is it to be?

The problem you will find is that if you want to have multiple shared
regions in your code you need to know what the offset is for each one.
Hence a parameter to shared memory pointer dereferencing is the offset
of the region you're currently referecing. There is a technique you can
use to take this out as a parameter by using the "this" pointer to
figure out which map you're referencing, but performance may become a
significant issue (although there are a number of strategies for
optimization).

Once you have this problem licked, you have the issue of vtables for
objects with virtual methods. Unless you build in some support from the
compiler, you're SOL.

G

> Hmm. Why not put just the pointer(s) into the shared region, and
> dynamically alloc the container(s) (or any other object) from the
heap?

Because then they're not shared.

--Phil.

> The problem you will find is that if you want to have multiple shared
> regions in your code you need to know what the offset is for each
one.

I'm happy with just the one region, I think.

> you have the issue of vtables for objects with virtual methods.

And I can live without virtual methods as well. Really I just need a
few set<struct>, map<int,struct>, strings, etc.

So, will it all just work then??

--Phil.

I wrote:
> how to store C++ objects, and if possible STL containers, in [a]
> shared region
> ...
> offset<T> class that I can use in places where I would normally use
T*.
> ...
> can I persuade the containers that
> they'd like to store my offsets, instead of real pointers?

Well, it's a bit later and I have some progress, but I'm not hopeful
about being able to store the containers. I have an offset<T> class
that can convert implicitly to and from T* in the obvious way, and it
seems to work in my own code. And I've got an allocator class that
uses my shared memory that I can pass to containers.

All that remains is getting the containers to store offsets rather than
pointers. I changed the typedef for "pointer" in the allocator from T*
to offset<T>, but this doesn't work because the containers (e.g. map)
have explicit T* types in their implementations, rather than using the
allocator's pointer type. So my offsets are converted straight back
into real pointers.

My feeling is that this is insurmountable. Will I have to write my own
map class?

(Gianni, can the Austria relative pointer class be used in standard
containers? I've had a look at it but I don't think I understand it
all.)

--Phil.

<> wrote
> > Hmm. Why not put just the pointer(s) into the shared region, and
> > dynamically alloc the container(s) (or any other object) from the
> heap?
>
> Because then they're not shared.

But the pointer to the data is shared. For practical cases this gives the same result.

>>> Hmm. Why not put just the pointer(s) into the shared region, and
>>> dynamically alloc the container(s) (or any other object) from the
>>> heap?
>> Because then they're not shared.
> But the pointer to the data is shared. For practical cases this gives
the same result.

No it doesn't. If I've got two processes and a shared memory region,
if I have a pointer in the shared region that points outside the
region, then it points to different things in each process. E.g. if
one process does

shared_pointer = "foo";

and the other process does

printf("%s",shared_pointer);

then it will NOT print 'foo', but instead whatever that process happens
to have at that address.

--Phil.

You need an implementation of the Standard C++ library that makes use
of nonstandard pointer types defined in allocators. AFAIK, only
Dinkumware offers such a library. The C++ Standard encourages this sort
of extension but doesn't require it. See Microsoft VC++ and a number of
other compilers that use our library. Or you can license an add-on
version, for a variety of compilers, directly at our web site.

Without such a library, your problem is indeed insurmountable.

P.J. Plauger
Dinkumware, Ltd.
http://www.dinkumware.com

> You need an implementation of the Standard C++ library that makes use
> of nonstandard pointer types defined in allocators. AFAIK, only
> Dinkumware offers such a library.

Thanks. Unfortunately this is for Anyterm
(http://chezphil.org/anyterm/) which is distributed as source code
under a GPL license. So it really needs to compile with g++ and its
default libraries, else no-one is going to use it.

(I'm curious to understand what downside, if any, there is to storing
allocator<T>:ointer rather than explicit T* in the container
implementation. Is this just an easy-to-fix issue with the GNU
libraries, or are they doing it for a good reason?)

--Phil.