Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Perl > Perl Misc > zipping large file using Archive::Zip

Reply
Thread Tools

zipping large file using Archive::Zip

 
 
Mark
Guest
Posts: n/a
 
      11-09-2007
I have a 16 GB file that I zipped using perl's Archive::zip module.
When I try to unzip this file using WinZip, it shows the "uncompressed
size" = 4294967295.

Upon trying to extract this file via WinZip, the winzip program
deflates the file correctly to 16 GB, but then before it completes and
yields control to the user, WinZip does a file size check between
actual decompressed size and the file information (uncompressed size)
stored with the Zip, and, since those don't match, it automatically it
considers the decompressed file as invalid and removes it from the
disk.

This root cause of this problem is due to perl's ZLIB module writing
an incorrect header information on the zip file

Can someone please tell me why Perl's Archive::zip module writes
incorrect information in the zip file header ? Are there any settings
that can override such behaviour?

Thanks

 
Reply With Quote
 
 
 
 
Paul Marquess
Guest
Posts: n/a
 
      11-12-2007
From: Christian Winter [(E-Mail Removed)]

> Mark wrote:
> > I have a 16 GB file that I zipped using perl's Archive::zip module.
> > When I try to unzip this file using WinZip, it shows the "uncompressed
> > size" = 4294967295.
> >
> > Upon trying to extract this file via WinZip, the winzip program
> > deflates the file correctly to 16 GB, but then before it completes and
> > yields control to the user, WinZip does a file size check between
> > actual decompressed size and the file information (uncompressed size)
> > stored with the Zip, and, since those don't match, it automatically it
> > considers the decompressed file as invalid and removes it from the
> > disk.
> >
> > This root cause of this problem is due to perl's ZLIB module writing
> > an incorrect header information on the zip file
> >
> > Can someone please tell me why Perl's Archive::zip module writes
> > incorrect information in the zip file header ? Are there any settings
> > that can override such behaviour?

>
> The only way around I know of is to use a different decompression
> tool. Archive::Zip makes use of Compress::Zlib, which in turn
> calls the native zlib library. zlib, however, hasn't adopted the
> somewhat proprietary deflate64 extension which PKWare introduced
> in the pkzip sdk to make size values > 2^32 possible. In the standard
> zip format, "compressed size" and "uncompressed size" are only
> 4 bytes, therefore the value 4294967295.
>
> Unless deflate64 gets incorporated into zlib, I don't see much
> chance that Archive::Zip will be able to produce "correct" zips
> of this size. So the short solutions would be to either split
> up things into separate files, each smaller than 4 GB uncompressed,
> or use a different compression (e.g. tar/bzip2) - at least if
> dropping WinZip is a no-go.


That's not correct - you are confusing Deflate64 with Zip64. Here is the
definition of Deflate64 from the Zip definition (from
http://www.pkware.com/documents/casestudies/APPNOTE.TXT)


Enhanced Deflating - Method 9
-----------------------------

The Enhanced Deflating algorithm is similar to Deflate but
uses a sliding dictionary of up to 64K. Deflate64(tm) is supported
by the Deflate extractor.

To support compressed files > 2^32, your Zip implementation needs to support
Zip64. That has nothing to do with the underlying zlib implementation.

I don't think that Archive::Zip supports Zip64, but IO::Compress::Zip does.

Paul

 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
File zipping programme - FREE Chaudhry Nijjhar Computer Support 7 03-18-2007 11:42 AM
uploading files to file system/zipping/downloading problems OriginalBrownster Python 4 08-21-2006 04:52 PM
Zipping without a file Tyler Reed Java 5 09-16-2004 04:10 PM
Zipping large group of files? Kenny Computer Support 7 06-02-2004 10:37 PM
Zipping many files -- each file into its own zip file with same name k* Computer Support 14 07-15-2003 12:26 PM



Advertisments