«

I'd like to hear this group's reaction on a javascript compression
script I've been working on. It uses the LZW algorithm and base85
encoding to squeeze large scripts down to size.

Quick test...

used this: http://ajax.googleapis.com/ajax/libs.../jquery.min.js
original size: 72173
compressed: 44782

You can test it here:
http://pressjs.googlecode.com/svn/trunk/build/test.html

Browse the source:
http://code.google.com/p/pressjs/sou...#svn/trunk/src

I'd love to hear what you guys think, esp. any way we could optimize
it for speed or size, or if you catch any bugs / memory leaks /
namespace pollution / stupid programming fails / etc. Thanks!

On May 22, 11:10*pm, nick <nick...@fastmail.fm> wrote:
> I'd like to hear this group's reaction on a javascript compression
> script I've been working on. It uses the LZW algorithm and base85
> encoding to squeeze large scripts down to size.
>
> Quick test...
>
> used this:http://ajax.googleapis.com/ajax/libs.../jquery.min.js
> original size: 72173
> compressed: 44782
>
> You can test it here:http://pressjs.googlecode.com/svn/trunk/build/test.html
>
> Browse the source:http://code.google.com/p/pressjs/sou...#svn/trunk/src
>
> I'd love to hear what you guys think, esp. any way we could optimize
> it for speed or size, or if you catch any bugs / memory leaks /
> namespace pollution / stupid programming fails / etc. Thanks!


I'm sorry to say that your attempt to 'compress' code has failed. Did
you ever take into consideration that gzip (used to served compressed
files) also use LZW (and in a more efficient way than you are)?

A quick test I did with an input file of 56.3KB:
Direct compression using 7-Zip into a .gz archive = 12KB
Compression using pressjs and then compressed into a .gz archive:
20.9KB

And the same using a minified version of the same script
Direct compression using 7-Zip into a .gz archive = 4.51KB
Compression using pressjs and then compressed into a .gz archive:
7.68KB

Not to mention the added overhead of having to decompress the file
after the UA has downloaded the file.

The only scenario where this method would be beneficial is where gzip
is not used on the server, bad caching directives are used causing the
file to be downloaded in full each time, and the extra time used
downloading is higher than the extra time needed to decompress.
Hopefully that isn't a too-common scenario.

But hey, it was probably fun to create

On May 23, 7:43*am, Sean Kinsey <okin...@gmail.com> wrote:

> I'm sorry to say that your attempt to 'compress' code has failed. Did
> you ever take into consideration that gzip (used to served compressed
> files) also use LZW (and in a more efficient way than you are)?

Yeah, I thought about that but I figured the point of javascript
compressors was that they would be used in environments where gzip
compression on the server is not an option (many shared hosts, which
many people seem content to use, for some reason don't use gzip).

> A quick test I did with an input file of 56.3KB:
> Direct compression using 7-Zip into a .gz archive = 12KB
> Compression using pressjs and then compressed into a .gz archive:
> 20.9KB

> And the same using a minified version of the same script
> Direct compression using 7-Zip into a .gz archive = 4.51KB
> Compression using pressjs and then compressed into a .gz archive:
> 7.68KB

I wonder if encoding to base64 would yield better compression ratios
afterwards? Maybe still not as good as using gzip on the uncompressed
file though.

I just did a similar test with Dean Edwards' "packer" with the "Base62
encode" and "Shrink variables" options on and it manages to get a
similar gzip-compressed size to the gzip-compressed size of the
original... If I can achieve a similar gzip-compressed size after
pressing, I think this should be at least as useful as packer (not
sure what this group's opinion of packer is, though).

> Not to mention the added overhead of having to decompress the file
> after the UA has downloaded the file.

True, although the size overhead is only about 1200 bytes (and
shrinking), and the processing overhead is negligible.

> The only scenario where this method would be beneficial is where gzip
> is not used on the server, bad caching directives are used causing the
> file to be downloaded in full each time, and the extra time used
> downloading is higher than the extra time needed to decompress.
> Hopefully that isn't a too-common scenario.

It's more common than you might think (shared hosting).

> But hey, it was probably fun to create

It was Thanks for the comments.

On May 23, 9:53*am, Johannes Baagoe <baa...@baagoe.com> wrote:
> nick :
>
> >http://pressjs.googlecode.com/svn/trunk/build/test.html
>
> "Où qu'il réside, même aux îles Caïmans, tout Français inscrit au rôle
> paiera son dû dès Noël" (length 92) "compresses" to 118 characters.

Well, you obviously used the wrong text.

"banana cabana banana cabana banana cabana banana cabana banana
cabana" (length 69) compresses to 44 characters!

> >http://code.google.com/p/pressjs/sou...#svn/trunk/src
>
> Fromhttp://code.google.com/p/pressjs/source/browse/trunk/src/compdict.js:
>
> * // Populate table with all possible character codes.
> * for(var i = 0; i < 256; ++i) {
> * * var str = String.fromCharCode(i);
> * * this.hashtable[str] = this.nextcode++;
> * } *
>
> What about character codes >= 256?

I'm pretty sure those characters aren't allowed in a javascript
document? I'm not really sure what's going on there though, I was
puzzled by that bit as well. See my next paragraph.

> My general impression is that you are complicating things for no reason.
> Why use constructors, prototypes and fancy "//#" pseudo-cpp directives?
> Just one file which defines the two functions that compress and expand
> would be much easier both to write and to review.

Yeah, that stuff is all part of another GPL program I ripped off to
make this compressor, which in turn is a pretty much direct port of
some c++ code, so it has a very class-like design. I've been going
through and making it more object-based, and trying to learn the
algorithm at the same time. Eventually I'd like to replace all of that
code, but for now I just wanted to see if this whole idea was viable.

Well, the cpp directives were my idea. I like to be able to separate
the files into logical units, and ifdef comes in handy when building
multiple similar-but-different targets (like stand-alone vs embedded
decompressor).

I'm definitely considering merging instream and outstream
functionalities into the compressor / decompressor, but I think I like
the dictionaries in separate files for now.

> (I assume that you are doing this for fun, for the challenge of writing
> a compressor in javascript. If it is in order to reduce bandwidth
> in real applications on the Web, enabling gzip on the server is much
> more efficient.)

Yeah, I'm mostly doing it to see if it can be done. Next I want to
experiment with a different compression algorithm or one of the
variations on LZW. Server-side gzip is obviously the better
alternative if it's available; however that's not always the case (see
my response to Sean) and so we have things like "packer" and maybe
this thing.

nick wrote:
> On May 23, 7:43 am, Sean Kinsey <okin...@gmail.com> wrote:
>
>> I'm sorry to say that your attempt to 'compress' code has failed. Did
>> you ever take into consideration that gzip (used to served compressed
>> files) also use LZW (and in a more efficient way than you are)?
>
> Yeah, I thought about that but I figured the point of javascript
> compressors was that they would be used in environments where gzip
> compression on the server is not an option (many shared hosts, which
> many people seem content to use, for some reason don't use gzip).

Mine doesn't; still I wouldn't use something like this. The largest
benefactors of GZIP are dial-up users and nodems have built-in
compression.

>
>> A quick test I did with an input file of 56.3KB:
>> Direct compression using 7-Zip into a .gz archive = 12KB
>> Compression using pressjs and then compressed into a .gz archive:
>> 20.9KB
>
>> And the same using a minified version of the same script
>> Direct compression using 7-Zip into a .gz archive = 4.51KB
>> Compression using pressjs and then compressed into a .gz archive:
>> 7.68KB
>
> I wonder if encoding to base64 would yield better compression ratios
> afterwards?

I seriously doubt it.

> Maybe still not as good as using gzip on the uncompressed
> file though.

Almost certainly not.

>
> I just did a similar test with Dean Edwards' "packer" with the "Base62
> encode" and "Shrink variables" options on and it manages to get a
> similar gzip-compressed size to the gzip-compressed size of the
> original... If I can achieve a similar gzip-compressed size after
> pressing, I think this should be at least as useful as packer (not
> sure what this group's opinion of packer is, though).

Packer is a complete waste of time.

>
>> Not to mention the added overhead of having to decompress the file
>> after the UA has downloaded the file.
>
> True, although the size overhead is only about 1200 bytes (and
> shrinking), and the processing overhead is negligible.

Define negligible.

>
>> The only scenario where this method would be beneficial is where gzip
>> is not used on the server, bad caching directives are used causing the
>> file to be downloaded in full each time, and the extra time used
>> downloading is higher than the extra time needed to decompress.
>> Hopefully that isn't a too-common scenario.
>
> It's more common than you might think (shared hosting).

Shared hosting doesn't automatically fit that bill. Mine doesn't have
GZIP, but I don't use bad "caching directives". And again, modem-based
compression makes all of these "packers" a waste of time.

On May 23, 2:34*pm, David Mark <dmark.cins...@gmail.com> wrote:
> nick wrote:
> > On May 23, 7:43 am, Sean Kinsey <okin...@gmail.com> wrote:

> >> I'm sorry to say that your attempt to 'compress' code has failed. Did
> >> you ever take into consideration that gzip (used to served compressed
> >> files) also use LZW (and in a more efficient way than you are)?

> > Yeah, I thought about that but I figured the point of javascript
> > compressors was that they would be used in environments where gzip
> > compression on the server is not an option (many shared hosts, which
> > many people seem content to use, for some reason don't use gzip).

> Mine doesn't; still I wouldn't use something like this. *The largest
> benefactors of GZIP are dial-up users and nodems have built-in
> compression. *

I had never heard of it before, but I found a good article on it here:

http://ixbtlabs.com/articles/compressv44vsv42bis/

It looks like text compresses particularly well in their tests.

Is this kind of thing usually enabled by default, or do modem users
have to jump through a bunch of hoops to set it up? I also found a lot
of instructions for enabling modem compression.

> > I wonder if encoding to base64 would yield better compression ratios
> > afterwards?

> I seriously doubt it.

Only one way to find out.

> > Maybe still not as good as using gzip on the uncompressed
> > file though.
>
> Almost certainly not.

Might be close. Packer-packed scripts can be slightly smaller than
their non-packed equivalents when gzipped.

> > I just did a similar test with Dean Edwards' "packer" with the "Base62
> > encode" and "Shrink variables" options on and it manages to get a
> > similar gzip-compressed size to the gzip-compressed size of the
> > original... If I can achieve a similar gzip-compressed size after
> > pressing, I think this should be at least as useful as packer (not
> > sure what this group's opinion of packer is, though).

> Packer is a complete waste of time.

Heh, thought you might say that. Packer has failed to work properly
with at least one script I've written... not sure who's fault that
was, but I've never felt comfortable using it.

> > [...] the processing overhead is negligible.

> Define negligible.

I don't notice any time going by at all, and I'm using an old laptop
from 2003 with one gig of ram downclocked to 1.07 GHz so it doesn't
catch on fire. I guess that's not a very scientific test though.

GG did something weird with the quotes, sorry about that.

nick wrote:
> On May 23, 2:34 pm, David Mark <dmark.cins...@gmail.com> wrote:
>> nick wrote:
>>> On May 23, 7:43 am, Sean Kinsey <okin...@gmail.com> wrote:
>
>>>> I'm sorry to say that your attempt to 'compress' code has failed. Did
>>>> you ever take into consideration that gzip (used to served compressed
>>>> files) also use LZW (and in a more efficient way than you are)?
>
>>> Yeah, I thought about that but I figured the point of javascript
>>> compressors was that they would be used in environments where gzip
>>> compression on the server is not an option (many shared hosts, which
>>> many people seem content to use, for some reason don't use gzip).
>
>> Mine doesn't; still I wouldn't use something like this. The largest
>> benefactors of GZIP are dial-up users and nodems have built-in
>> compression.
>
> I had never heard of it before, but I found a good article on it here:
>
> http://ixbtlabs.com/articles/compressv44vsv42bis/
>
> It looks like text compresses particularly well in their tests.

Yes, extremely well.

>
> Is this kind of thing usually enabled by default, or do modem users
> have to jump through a bunch of hoops to set it up? I also found a lot
> of instructions for enabling modem compression.

It typically works right out of the box. Has for decades. Skip the
articles about modem init strings. They haven't been a concern for the
average user in decades.

>
>>> I wonder if encoding to base64 would yield better compression ratios
>>> afterwards?
>
>> I seriously doubt it.
>
> Only one way to find out.
>
>>> Maybe still not as good as using gzip on the uncompressed
>>> file though.
>> Almost certainly not.
>
> Might be close. Packer-packed scripts can be slightly smaller than
> their non-packed equivalents when gzipped.

But then you have to download Packer and wait for it to decompress the
content. It's a waste of time.

>
>>> I just did a similar test with Dean Edwards' "packer" with the "Base62
>>> encode" and "Shrink variables" options on and it manages to get a
>>> similar gzip-compressed size to the gzip-compressed size of the
>>> original... If I can achieve a similar gzip-compressed size after
>>> pressing, I think this should be at least as useful as packer (not
>>> sure what this group's opinion of packer is, though).
>
>> Packer is a complete waste of time.
>
> Heh, thought you might say that. Packer has failed to work properly
> with at least one script I've written... not sure who's fault that
> was, but I've never felt comfortable using it.

Even if it worked flawlessly, it would still be a waste of time. The
fact that it introduces an additional point of failure is just a
"bonus".

>
>>> [...] the processing overhead is negligible.
>
>> Define negligible.
>
> I don't notice any time going by at all, and I'm using an old laptop
> from 2003 with one gig of ram downclocked to 1.07 GHz so it doesn't
> catch on fire. I guess that's not a very scientific test though.

No.

nick wrote:
> GG did something weird with the quotes, sorry about that.

That's alright. Thunderbird did something very weird with the post as
well. It threw an error (too many IP connections to the server or some
such BS), put it in the Sent folder, but didn't send it. Just so
happens I noticed and pasted it into GG. That's probably what caused
whatever weirdness you are referring to.

On May 23, 3:07*pm, David Mark <dmark.cins...@gmail.com> wrote:
> nick wrote:
> > Is [modem compression] usually enabled by default, or do modem users
> > have to jump through a bunch of hoops to set it up? I also found a lot
> > of instructions for enabling modem compression.

> It typically works right out of the box. *Has for decades. *Skip the
> articles about modem init strings. *They haven't been a concern for the
> average user in decades.

I was seeing stuff like this:

http://technet.microsoft.com/en-us/l...22(WS.10).aspx

It doesn't mention what the default setting might be, of course.

> Even if [packer] worked flawlessly, it would still be a waste of time. *
> The fact that it introduces an additional point of failure is just a
> "bonus". *

Hopefully press.js will remain bonus-free, and prove to be an
interesting time-waster.

> >>> [...] the processing overhead is negligible.

> >> Define negligible.

> > I don't notice any time going by at all [...]

> No.

"Show me where it's slow!"