Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Java > How to check if byte array is compressed or not?

Reply
Thread Tools

How to check if byte array is compressed or not?

 
 
prakasan@gmail.com
Guest
Posts: n/a
 
      08-03-2005
hi All,

I have some data stored in the database as blob. I can read this blob
into byte array using jdbc. Now I need to read the byte array using
InputStream. The problem is I want to know if the data in the database
is stored in comprressed form. If the byte array is compressed I have
to use InflaterInputStream. other wise I can use ObjectInputStream.

Before using which stream to use with the byte array I want to know if
the byte array is compressed or not?

Any idea how this can be done??

thanks in advance,
Prakasan

 
Reply With Quote
 
 
 
 
Ingo R. Homann
Guest
Posts: n/a
 
      08-03-2005
Hi,

http://www.velocityreviews.com/forums/(E-Mail Removed) wrote:
> hi All,
>
> I have some data stored in the database as blob. I can read this blob
> into byte array using jdbc. Now I need to read the byte array using
> InputStream. The problem is I want to know if the data in the database
> is stored in comprressed form. If the byte array is compressed I have
> to use InflaterInputStream. other wise I can use ObjectInputStream.
>
> Before using which stream to use with the byte array I want to know if
> the byte array is compressed or not?
>
> Any idea how this can be done??


I am not familar with the InflaterInputStream, and you did not say much
about the kind of compression. In fact, *every* byte array can be viewed
as 'compressed' if you specify an adequate compression algorithm. So,
without knowing the algorithm, your problem cannot be solved.

Having said this, look at some of your byte arrays in detail. Normally,
because of what I said above, most compression algorithms add some kind
of header to mark the byte array as 'compressed by this algorithm'.

Hth,
Ingo

PS: Note that even if a byte array starts with "Compressed with GZIP
Version 3.45", this might be a coincidental sequence of bytes!
So, If you *really* want to be sure, it would be a good idea to add a
header in any case (even, if the byte array is not compressed).

 
Reply With Quote
 
 
 
 
Andrew Thompson
Guest
Posts: n/a
 
      08-03-2005
On 3 Aug 2005 00:52:13 -0700, (E-Mail Removed) wrote:

> hi All,


Hello again.. Please refrain from multi-posting.
<http://www.physci.org/codes/javafaq.jsp#xpost>

Also, please be clear that you are posting to Usenet, not Google.
<http://www.physci.org/codes/javafaq.jsp#usenet>

--
Andrew Thompson
physci.org 1point1c.org javasaver.com lensescapes.com athompson.info
Beats A Hard Kick In The Face
 
Reply With Quote
 
Antti S. Brax
Guest
Posts: n/a
 
      08-03-2005
(E-Mail Removed) wrote in comp.lang.java.programmer:
> (E-Mail Removed) wrote:
>> I have some data stored in the database as blob. I can read this blob
>> into byte array using jdbc. Now I need to read the byte array using
>> InputStream. The problem is I want to know if the data in the database
>> is stored in comprressed form. If the byte array is compressed I have
>> to use InflaterInputStream. other wise I can use ObjectInputStream.
>>
>> Before using which stream to use with the byte array I want to know if
>> the byte array is compressed or not?

<snip>
> PS: Note that even if a byte array starts with "Compressed with GZIP
> Version 3.45", this might be a coincidental sequence of bytes!
> So, If you *really* want to be sure, it would be a good idea to add a
> header in any case (even, if the byte array is not compressed).


My first suggestion would be to add a table column which
indicates the compression algorithm (if any) used in the
data. If you can't change the database schema, then consider
adding a header. But then be aware that the data will become
useless for programs that don't understand your magic header.

--
Antti S. Brax Rullalautailu pitää lapset poissa ladulta
http://www.iki.fi/asb/ http://www.cs.helsinki.fi/u/abrax/hlb/

[1385 messages expunged from folder "Spam"]
 
Reply With Quote
 
John Currier
Guest
Posts: n/a
 
      08-03-2005
(E-Mail Removed) wrote:
> I have some data stored in the database as blob. I can read this blob
> into byte array using jdbc. Now I need to read the byte array using
> InputStream. The problem is I want to know if the data in the database
> is stored in comprressed form. If the byte array is compressed I have
> to use InflaterInputStream. other wise I can use ObjectInputStream.
>
> Before using which stream to use with the byte array I want to know if
> the byte array is compressed or not?


A simplistic approach is to just try to decompress it. If that fails
then it's not compressed (or is corrupted). This simplistic approach
assumes that the majority of the stored data is compressed. The
approach can be slightly optimized by checking the header yourself
before attempting a decompression. I've used this technique in
interceptors for decompressing CORBA traffic.

The only real risk is of successfully decompressing something that
wasn't compressed.

Antti's approach, however, is cleaner.

John
http://schemaspy.sourceforge.net

 
Reply With Quote
 
Harald
Guest
Posts: n/a
 
      08-03-2005
(E-Mail Removed) writes:

> hi All,
>
> I have some data stored in the database as blob. I can read this blob
> into byte array using jdbc. Now I need to read the byte array using
> InputStream. The problem is I want to know if the data in the database
> is stored in comprressed form. If the byte array is compressed I have
> to use InflaterInputStream. other wise I can use ObjectInputStream.
>
> Before using which stream to use with the byte array I want to know if
> the byte array is compressed or not?


The entropy of compressed data tends to be higher than that of
uncompressed data. But this is just a statistical observation.-)

Strange setup you have there where you don't know what kind of data
format you are dealing with.

Harald.
--
---------------------+---------------------------------------------
Harald Kirsch (@home)|
Java Text Crunching: http://www.ebi.ac.uk/Rebholz-srv/whatizit/software
 
Reply With Quote
 
Roedy Green
Guest
Posts: n/a
 
      08-03-2005
On 3 Aug 2005 00:52:13 -0700, (E-Mail Removed) wrote or quoted :

>
>Any idea how this can be done??


but a boolean on the front of the stream to tell you is the easiest
way.

I am beginning to realize the wisdom of putting a field on the front
of any stream giving the version. Otherwise it becomes impossible to
deal with old format files later on.

Other that that, look at the two files with hex viewer to see if
the9re is a recognisable signature.

see http://mindprod.com/jgloss/hex.html

--
Bush crime family lost/embezzled $3 trillion from Pentagon.
Complicit Bush-friendly media keeps mum. Rumsfeld confesses on video.
http://www.infowars.com/articles/us/...s_rumsfeld.htm

Canadian Mind Products, Roedy Green.
See http://mindprod.com/iraq.html photos of Bush's war crimes
 
Reply With Quote
 
espenskogen espenskogen is offline
Junior Member
Join Date: Jan 2011
Posts: 1
 
      01-31-2011
Quote:
Originally Posted by (E-Mail Removed)
hi All,

I have some data stored in the database as blob. I can read this blob
into byte array using jdbc. Now I need to read the byte array using
InputStream. The problem is I want to know if the data in the database
is stored in comprressed form. If the byte array is compressed I have
to use InflaterInputStream. other wise I can use ObjectInputStream.

Before using which stream to use with the byte array I want to know if
the byte array is compressed or not?

Any idea how this can be done??

thanks in advance,
Prakasan
Easiest way is to inspect the first bytes of the byte array. ZLib compression starts with 0x78AD - whilst Zip files start with 0x04034b50

E
 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
convert form byte[4] to Int32 while retaining the binary value of the byte array jeff@foundrymusic.com C++ 20 09-07-2009 08:54 PM
Re: how to extend a byte[] array with a null byte? Tom McGlynn Java 4 04-18-2008 11:49 PM
Read compressed files and deal with byte swapping. tianyf@gmail.com C Programming 7 10-01-2007 10:11 AM
Converting a Primative byte array to a Byte array object Kirby Java 3 10-08-2004 03:01 AM
Appending byte[] to another byte[] array Bharat Bhushan Java 15 08-05-2003 07:52 PM



Advertisments