Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Python > memory mapped tar file contents

Reply
Thread Tools

memory mapped tar file contents

 
 
Chris Brooks
Guest
Posts: n/a
 
      11-13-2008

Hi,

I would like to read directly from a tar file into memory so I can
manipulate a file (quickly) and write its changes out to another file. I
thought I could do something like:

#!/usr/bin/env python

import tarfile
import mmap

fil = tarfile.open( "out.tar.gz" , "r:gz" )
tarinf = fil.next()
myfils = {}
while tarinf != None:
tarinf = fil.next()
ref = fil.extractfile( tarinf )
myfils[ tarinf.name ] = mmap.mmap( ref.fileno() , 0 )

But the extractfile() function of TarInfo doesn't seem to give me a fileno,
so I can't pass this to mmap.

Thoughts on a way to accomplish this?

Chris
--
View this message in context: http://www.nabble.com/memory-mapped-...p20473925.html
Sent from the Python - python-list mailing list archive at Nabble.com.

 
Reply With Quote
 
 
 
 
Aaron Brady
Guest
Posts: n/a
 
      11-13-2008
On Nov 12, 8:51*pm, Chris Brooks <(E-Mail Removed)> wrote:
> Hi,
>
> I would like to read directly from a tar file into memory so I can
> manipulate a file (quickly) and write its changes out to another file. *I
> thought I could do something like:
>
> #!/usr/bin/env python
>
> import tarfile
> import mmap
>
> fil = tarfile.open( "out.tar.gz" , "r:gz" )
> tarinf = fil.next()
> myfils = {}
> while tarinf != None:
> * * tarinf = fil.next()
> * * ref = fil.extractfile( tarinf )
> * * myfils[ tarinf.name ] = mmap.mmap( ref.fileno() , 0 )
>
> But the extractfile() function of TarInfo doesn't seem to give me a fileno,
> so I can't pass this to mmap.
>
> Thoughts on a way to accomplish this?


It appears you have to read the contents of the file into the mmap.
You can create an anonymous map of size tarinfoobj.size, then set
mapobj[:]= fil.extractfile( tarinf ).read( ) . Untested.
 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
os.system('tar -c * | tar -C dst') ##Any other suggestions... list.repository@gmail.com Python 2 04-24-2007 10:29 PM
Segementation Problem while accessing contents from Memory Mapped File nkrisraj@gmail.com C Programming 3 01-10-2006 08:00 PM
Core dump while accessing contents from Memory Mapped File nkrisraj@gmail.com C Programming 8 01-10-2006 07:58 PM
Segementation Problem while accessing contents from Memory Mapped File nkrisraj@gmail.com C Programming 0 01-10-2006 04:53 PM
Version of TAR in tarfile module? TAR 1.14 or 1.15 port to Windows? Claudio Grondi Python 4 08-20-2005 08:01 PM



Advertisments