Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Perl > Perl Misc > How to uncompress HTTP GZIP data

Reply
Thread Tools

How to uncompress HTTP GZIP data

 
 
Mark Smith
Guest
Posts: n/a
 
      12-19-2005
Hi,

Following is the response header from IIS:

HTTP/1.1 200 OK
Cache-Control: no-cache
Connection: close
Date: Mon, 19 Dec 2005 21:00:17 GMT
Pragma: no-cache
Content-Type: text/html
Expires: Wed, 01 Jan 1997 12:00:00 GMT
Server: Microsoft-IIS/6.0
P3P: CP="BUS CUR CONo FIN IVDo ONL OUR PHY SAMo TELo"
Content-Encoding: gzip
Vary: Accept-Encoding
Transfer-Encoding: chunked

The http data was captured from all of the TCP packets in sequence, and
saved to a .Z file. The file will not uncompress using GUNZIP utility.
I opened the .z file in a hex editor, and was able to see that the
header does not begin with the magic header for gzip files. I fixed
that, but it still doesn't work.

HOW COULD I UNCOMPRESS THE CAPTURED HTTP DATA?

thanks much for all your help!

 
Reply With Quote
 
 
 
 
Gunnar Hjalmarsson
Guest
Posts: n/a
 
      12-19-2005
Mark Smith wrote:
> Following is the response header from IIS:
>
> HTTP/1.1 200 OK
> Cache-Control: no-cache
> Connection: close
> Date: Mon, 19 Dec 2005 21:00:17 GMT
> Pragma: no-cache
> Content-Type: text/html
> Expires: Wed, 01 Jan 1997 12:00:00 GMT
> Server: Microsoft-IIS/6.0
> P3P: CP="BUS CUR CONo FIN IVDo ONL OUR PHY SAMo TELo"
> Content-Encoding: gzip
> Vary: Accept-Encoding
> Transfer-Encoding: chunked
>
> The http data was captured from all of the TCP packets in sequence, and
> saved to a .Z file. The file will not uncompress using GUNZIP utility.
> I opened the .z file in a hex editor, and was able to see that the
> header does not begin with the magic header for gzip files. I fixed
> that, but it still doesn't work.
>
> HOW COULD I UNCOMPRESS THE CAPTURED HTTP DATA?


You may find this thread useful:
http://groups.google.com/group/comp....cfa9d737888425

--
Gunnar Hjalmarsson
Email: http://www.gunnar.cc/cgi-bin/contact.pl
 
Reply With Quote
 
 
 
 
Paul Marquess
Guest
Posts: n/a
 
      12-20-2005

"Mark Smith" <(E-Mail Removed)> wrote in message
news:(E-Mail Removed) oups.com...
> Hi,
>
> Following is the response header from IIS:
>
> HTTP/1.1 200 OK
> Cache-Control: no-cache
> Connection: close
> Date: Mon, 19 Dec 2005 21:00:17 GMT
> Pragma: no-cache
> Content-Type: text/html
> Expires: Wed, 01 Jan 1997 12:00:00 GMT
> Server: Microsoft-IIS/6.0
> P3P: CP="BUS CUR CONo FIN IVDo ONL OUR PHY SAMo TELo"
> Content-Encoding: gzip
> Vary: Accept-Encoding
> Transfer-Encoding: chunked
>
> The http data was captured from all of the TCP packets in sequence, and
> saved to a .Z file. The file will not uncompress using GUNZIP utility.
> I opened the .z file in a hex editor, and was able to see that the
> header does not begin with the magic header for gzip files. I fixed
> that, but it still doesn't work.
>
> HOW COULD I UNCOMPRESS THE CAPTURED HTTP DATA?
>
> thanks much for all your help!


The HTTP header states that as well as having a content encoding of "gzip"
applied, it also uses chunked transfer encoding. If that is the case then
you will have to un-chunk the data before you can uncompress it.

Post a hex dump of the start of file you have saved so we can see if it
actually is chunked.

Paul


 
Reply With Quote
 
Tad McClellan
Guest
Posts: n/a
 
      12-20-2005
Mark Smith <(E-Mail Removed)> wrote:

> Following is the response header from IIS:
>
> HTTP/1.1 200 OK
> Cache-Control: no-cache
> Connection: close
> Date: Mon, 19 Dec 2005 21:00:17 GMT
> Pragma: no-cache
> Content-Type: text/html
> Expires: Wed, 01 Jan 1997 12:00:00 GMT
> Server: Microsoft-IIS/6.0
> P3P: CP="BUS CUR CONo FIN IVDo ONL OUR PHY SAMo TELo"
> Content-Encoding: gzip
> Vary: Accept-Encoding
> Transfer-Encoding: chunked
>
> The http data was captured from all of the TCP packets in sequence, and
> saved to a .Z file. The file will not uncompress using GUNZIP utility.
> I opened the .z file in a hex editor, and was able to see that the
> header does not begin with the magic header for gzip files. I fixed
> that, but it still doesn't work.
>
> HOW COULD I UNCOMPRESS THE CAPTURED HTTP DATA?



It is not necessary to shout at us like that.


> thanks much for all your help!



Did you have a Perl question?


--
Tad McClellan SGML consulting
http://www.velocityreviews.com/forums/(E-Mail Removed) Perl programming
Fort Worth, Texas
 
Reply With Quote
 
Mark Smith
Guest
Posts: n/a
 
      12-20-2005
Hi Paul

Here are first few bytes of captured data. From what I could tell, it
is chunked. And you can also see the 'magic header' of gzip (1F8B)
starting at location 6....

610D0A1F8B08000000000004000D0A3763300D0AEC5C7973E2 C6B6FF7B5CF5BE

thanks very much!

data was from IIS 6

 
Reply With Quote
 
Mark Smith
Guest
Posts: n/a
 
      12-20-2005
Sorry sir, I wasn't shouting. Just a bad typing style to make the
question stand out.

 
Reply With Quote
 
Paul Marquess
Guest
Posts: n/a
 
      12-20-2005
Mark Smith wrote:

> Hi Paul
>
> Here are first few bytes of captured data. From what I could tell, it
> is chunked. And you can also see the 'magic header' of gzip (1F8B)
> starting at location 6....
>
> 610D0A1F8B08000000000004000D0A3763300D0AEC5C7973E2 C6B6FF7B5CF5BE


Yep, that's chunked and gzipped.

Try filtering it through something like this to unchunk it, then see if
gunzip can deal with it.

undef $/;
$_ = <>;

while (length $_)
{
last if /^\r\n0+\r\n\r\n/;

die "error dechunking\n"
unless s/^(?:\r\n)?([0-9A-F]+)\r\n//i ;

my $chunkSize = hex $1 ;

print substr($_, 0, $chunkSize) ;
substr($_, 0, $chunkSize) = '';
}


Paul

 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Re: How include a large array? Edward A. Falk C Programming 1 04-04-2013 08:07 PM
trying to gzip uncompress a StringIO bob@snee.com Python 3 05-23-2007 09:41 AM
Compress data on server and uncompress on client? =?Utf-8?B?SGFycnkgS2Vjaw==?= ASP .Net 4 05-23-2006 04:09 PM
can Inflater be used to uncompress GZIP data? chattycow Java 11 05-22-2006 08:13 PM
Cannot read or uncompress mail files jtk6204@gmail.com Firefox 1 01-09-2006 12:57 AM



Advertisments