Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Python > Shared Memory Modules

Reply
Thread Tools

Shared Memory Modules

 
 
S Green
Guest
Posts: n/a
 
      11-25-2003
Does any one now if a shared memory module exists, written in python
for a windows platform. I now one exists for unix?


help most appreciated,

S Green
 
Reply With Quote
 
 
 
 
Peter Hansen
Guest
Posts: n/a
 
      11-25-2003
S Green wrote:
>
> Does any one now if a shared memory module exists, written in python
> for a windows platform. I now one exists for unix?


Your first message with this same question did in fact make it to
Usenet and the mailing list.

Generally speaking, you should allow several days for answers to questions,
rather than getting anxious just because ten people don't jump up and answer
it in the first few hours.

(And my guess is that one doesn't exist, as Windows shared memory would
probably be a completely unreliable bitch, but that's just my guess.

-Peter
 
Reply With Quote
 
 
 
 
Thomas Heller
Guest
Posts: n/a
 
      11-25-2003
http://www.velocityreviews.com/forums/(E-Mail Removed) (S Green) writes:

> Does any one now if a shared memory module exists, written in python
> for a windows platform. I now one exists for unix?
>
>
> help most appreciated,
>
> S Green


import mmap

 
Reply With Quote
 
Peter Hansen
Guest
Posts: n/a
 
      11-25-2003
Thomas Heller wrote:
>
> (E-Mail Removed) (S Green) writes:
>
> > Does any one now if a shared memory module exists, written in python
> > for a windows platform. I now one exists for unix?
> >
> >
> > help most appreciated,
> >
> > S Green

>
> import mmap


The docs suggest that mmap.mmap(x, x, mmap.MAP_SHARED) will work under
Unix but not Windows.

Does the Windows version really support shared memory, or is the
multiple-maps-per-file feature only valid within a single process?

-Peter
 
Reply With Quote
 
Thomas Heller
Guest
Posts: n/a
 
      11-25-2003
Peter Hansen <(E-Mail Removed)> writes:

> Thomas Heller wrote:
>>
>> (E-Mail Removed) (S Green) writes:
>>
>> > Does any one now if a shared memory module exists, written in python
>> > for a windows platform. I now one exists for unix?
>> >
>> >
>> > help most appreciated,
>> >
>> > S Green

>>
>> import mmap

>
> The docs suggest that mmap.mmap(x, x, mmap.MAP_SHARED) will work under
> Unix but not Windows.
>
> Does the Windows version really support shared memory, or is the
> multiple-maps-per-file feature only valid within a single process?


Yes, it does. You have to give this memory block a name, though:

sharedmem = mmap.mmap(0, 16384, "GlobalSharedMemory")

This allows to access 16384 bytes of memory, shared across processes,
not backed up by a file in the filesystem.

Thomas
 
Reply With Quote
 
Peter Hansen
Guest
Posts: n/a
 
      11-25-2003
Thomas Heller wrote:
>
> Peter Hansen <(E-Mail Removed)> writes:
>
> > Thomas Heller wrote:
> >>
> >> (E-Mail Removed) (S Green) writes:
> >>
> >> > Does any one now if a shared memory module exists, written in python
> >> > for a windows platform. I now one exists for unix?
> >> >
> >> >
> >> > help most appreciated,
> >> >
> >> > S Green
> >>
> >> import mmap

> >
> > The docs suggest that mmap.mmap(x, x, mmap.MAP_SHARED) will work under
> > Unix but not Windows.
> >
> > Does the Windows version really support shared memory, or is the
> > multiple-maps-per-file feature only valid within a single process?

>
> Yes, it does. You have to give this memory block a name, though:
>
> sharedmem = mmap.mmap(0, 16384, "GlobalSharedMemory")
>
> This allows to access 16384 bytes of memory, shared across processes,
> not backed up by a file in the filesystem.


Thanks, Thomas.

In my opinion the documentation on this is entirely unclear. I attach
it for reference, but I can't offer any suggestions for improvement (as
I don't know anything about shared memory) except that even after reading
it a second time, Thomas' answer above is very much news to me:

'''tagname, if specified and not None, is a string giving a tag name
for the mapping. Windows allows you to have many different mappings
against the same file. If you specify the name of an existing tag,
that tag is opened, otherwise a new tag of this name is created. If
this parameter is omitted or None, the mapping is created without a
name. Avoiding the use of the tag parameter will assist in keeping your
code portable between Unix and Windows. '''

(from http://www.python.org/doc/current/lib/module-mmap.html)

-Peter
 
Reply With Quote
 
Irmen de Jong
Guest
Posts: n/a
 
      11-25-2003
Thomas Heller wrote:

>>Does the Windows version really support shared memory, or is the
>>multiple-maps-per-file feature only valid within a single process?

>
>
> Yes, it does. You have to give this memory block a name, though:
>
> sharedmem = mmap.mmap(0, 16384, "GlobalSharedMemory")
>
> This allows to access 16384 bytes of memory, shared across processes,
> not backed up by a file in the filesystem.


That is très cool, it doesn't tell you this in the docs, does it?
The first argument is 'the file handle' of the file to be mapped,
and it doesn't say that 0 is valid and means 'no file at all'...

However, I've just tried it, and managed to crash Python in mmap.pyd
with an application exception... twice. But trying to reproduce it
now fails-- Python keeps running.
I did this:
>>> import mmap
>>> mem=mmap.mmap(0,3000000,'irmen')
>>> mem[0]='a'
>>> mem[2000000]='a'

and initially, it crashed......... Python 2.3.2 on win xp)

--Irmen

 
Reply With Quote
 
Thomas Heller
Guest
Posts: n/a
 
      11-25-2003
Irmen de Jong <(E-Mail Removed)> writes:

> Thomas Heller wrote:
>
>>> Does the Windows version really support shared memory, or is the
>>> multiple-maps-per-file feature only valid within a single process?

>> Yes, it does. You have to give this memory block a name, though:
>> sharedmem = mmap.mmap(0, 16384, "GlobalSharedMemory")
>> This allows to access 16384 bytes of memory, shared across processes,
>> not backed up by a file in the filesystem.

>
> That is très cool, it doesn't tell you this in the docs, does it?
> The first argument is 'the file handle' of the file to be mapped,
> and it doesn't say that 0 is valid and means 'no file at all'...
>
> However, I've just tried it, and managed to crash Python in mmap.pyd
> with an application exception... twice. But trying to reproduce it
> now fails-- Python keeps running.
> I did this:
> >>> import mmap
> >>> mem=mmap.mmap(0,3000000,'irmen')
> >>> mem[0]='a'
> >>> mem[2000000]='a'

> and initially, it crashed......... Python 2.3.2 on win xp)


This works for me even from the beginning (with 2.3 CVS version, XP Pro).

Understading which parameters to pass apparently requires

- reading the mmapmodule.c sources
- and reading about CreateFileMapping and MapViewOfFile in MSDN.

It would be great if someine could submit a patch for the docs <wink>.

I vaguely remember, but am not able to find it anymore: didn't AMK once
had an article about this somewhere?

Thomas
 
Reply With Quote
 
Scott David Daniels
Guest
Posts: n/a
 
      12-01-2003
Irmen de Jong wrote:
> However, I've just tried it, and managed to crash Python in mmap.pyd
> with an application exception... twice. But trying to reproduce it
> now fails-- Python keeps running.
> I did this:
> >>> import mmap
> >>> mem=mmap.mmap(0,3000000,'irmen')
> >>> mem[0]='a'
> >>> mem[2000000]='a'

> and initially, it crashed......... Python 2.3.2 on win xp)
> --Irmen


Probably not enough actual memory hanging around. Some systems
(and from this I'd guess XP) allocate virtual memory by reserving
address space, not actually allocating the RAM and/or backing
store for that memory. Python has no control over this, and you
have nothing good to do if the memory is over-allocated. When
you create the memory, you could walk across it writing into it
(forcing it to exist), but that would just force the failure to
happen earlier.


-Scott David Daniels
(E-Mail Removed)

 
Reply With Quote
 
Irmen de Jong
Guest
Posts: n/a
 
      12-01-2003
Scott David Daniels wrote:
> Irmen de Jong wrote:
>
>> However, I've just tried it, and managed to crash Python in mmap.pyd
>> with an application exception... twice. But trying to reproduce it
>> now fails-- Python keeps running.
>> I did this:
>> >>> import mmap
>> >>> mem=mmap.mmap(0,3000000,'irmen')
>> >>> mem[0]='a'
>> >>> mem[2000000]='a'

>> and initially, it crashed......... Python 2.3.2 on win xp)
>> --Irmen

>
>
> Probably not enough actual memory hanging around. Some systems
> (and from this I'd guess XP) allocate virtual memory by reserving
> address space, not actually allocating the RAM and/or backing
> store for that memory.


Over-commit is that called, right?
I'm sorry but that certainly wasn't the case here.
My machine has 512 Mb RAM and about 250 Mb of them
allocated when I tried it. (physical RAM that is)

Very weird, I cannot reproduce the initial crash I experienced.

--Irmen

 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Avoid memory corruption in shared memory used by several processes? Sune C Programming 14 08-26-2007 03:57 AM
Avoid memory corruption in shared memory used by several processes? Sune C Programming 5 07-13-2007 02:29 PM
Shared Public Variables and Shared Methods Joe Fallon ASP .Net 3 07-16-2004 07:11 AM
Cannot refer to an instance member of a class from within a shared method or shared member initializer without an explicit instance of the class. DJ Dev ASP .Net 3 02-08-2004 04:19 PM



Advertisments