Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Ruby > Streams of bits

Reply
Thread Tools

Streams of bits

 
 
Richard Fairhurst
Guest
Posts: n/a
 
      05-17-2007
Hi,

I'm writing a bit of Ruby to output SWF files.

SWF's opcodes and arguments are variable-length streams of bits. They're
packed in direct succession - i.e. not usually padded to byte
boundaries.

So, for example, you might have

00111 5-bit record
0110101 7-bit record
0000110 7-bit record
0001100 7-bit record
1111101 7-bit record

which would be packed as

0b00111011,0b01010000,0b11000011,0b00111110,0b1000 0000

(the final byte here is null-padded)

I'm trying to write these opcode by opcode, and get a bytestream out the
end of it. Currently I'm just appending each opcode to a long string
(m+='00111'), and when it comes to writing it out, splitting this every
eight characters and converting back to a single character. But this is
awfully slow.

Can anyone suggest a faster way?

(Apologies if this shows up twice, I've been arguing with Google Groups
today. )

cheers
Richard
http://www.velocityreviews.com/forums/(E-Mail Removed)

--
Posted via http://www.ruby-forum.com/.

 
Reply With Quote
 
 
 
 
Daniel Martin
Guest
Posts: n/a
 
      05-17-2007
Richard Fairhurst <(E-Mail Removed)> writes:

> I'm trying to write these opcode by opcode, and get a bytestream out the
> end of it. Currently I'm just appending each opcode to a long string
> (m+='00111'), and when it comes to writing it out, splitting this every
> eight characters and converting back to a single character. But this is
> awfully slow.
>
> Can anyone suggest a faster way?


How are you going from your string of 0s and 1s to bytes?

Might I suggest that the fastest way to do that part of the job is
this?

outstring = [m].pack("B*")

That'll pack things the way you seem to want them, and it'll
appropriately null-pad the last byte.

If you want to crunch 0s and 1s into bytes as you're building up your
opcode string, one possibility is to add this into whatever loop it is
that is appending opcodes:

while m.length > 40 do
outstring += [m.slice!(0...40)].pack("B*")
end

That pulls bytes off m five bytes at a time. The goal is to try to
strike a balance between letting m get too large (which makes
manipulating it in memory slightly slower) and letting pack - written
in C - do its job efficiently. (pack is going to be more efficient
the larger the input)

You'll still need to at the very end do
outstring += [m].pack("B*")
to get the remainder.

You can experiment with how much to pull off of m at a time to see
what value makes your particular program fastest. For your purposes,
you may well find that the fastest solution is to not do any
0s-and-1s-to-bytes operations in your loop, and simply use pack at the
end.

--
s=%q( Daniel Martin -- (E-Mail Removed)
puts "s=%q(#{s})",s.to_a.last )
puts "s=%q(#{s})",s.to_a.last

 
Reply With Quote
 
 
 
 
Brian Candler
Guest
Posts: n/a
 
      05-17-2007
On Thu, May 17, 2007 at 09:52:51PM +0900, Richard Fairhurst wrote:
> I'm writing a bit of Ruby to output SWF files.
>
> SWF's opcodes and arguments are variable-length streams of bits. They're
> packed in direct succession - i.e. not usually padded to byte
> boundaries.
>
> So, for example, you might have
>
> 00111 5-bit record
> 0110101 7-bit record
> 0000110 7-bit record
> 0001100 7-bit record
> 1111101 7-bit record
>
> which would be packed as
>
> 0b00111011,0b01010000,0b11000011,0b00111110,0b1000 0000
>
> (the final byte here is null-padded)
>
> I'm trying to write these opcode by opcode, and get a bytestream out the
> end of it. Currently I'm just appending each opcode to a long string
> (m+='00111'), and when it comes to writing it out, splitting this every
> eight characters and converting back to a single character. But this is
> awfully slow.
>
> Can anyone suggest a faster way?


Hmm, sounds like Huffman coding... see Ruby Quiz just gone

If speed is critical it might be worth writing a C extension to do it.

 
Reply With Quote
 
Phrogz
Guest
Posts: n/a
 
      05-17-2007
On May 17, 6:52 am, Richard Fairhurst <(E-Mail Removed)> wrote:
> I'm writing a bit of Ruby to output SWF files.


Nice!

> SWF's opcodes and arguments are variable-length streams of bits. They're
> packed in direct succession - i.e. not usually padded to byte
> boundaries.


Not that I've used either, but see:
BitStruct: http://raa.ruby-lang.org/project/bit-struct/
BitStructEx: http://bit-struct-ex.rubyforge.org/wiki/wiki.pl

 
Reply With Quote
 
Richard Fairhurst
Guest
Posts: n/a
 
      05-18-2007
Daniel Martin wrote:
> That pulls bytes off m five bytes at a time. The goal is to try to
> strike a balance between letting m get too large (which makes
> manipulating it in memory slightly slower) and letting pack - written
> in C - do its job efficiently. (pack is going to be more efficient
> the larger the input)


Thanks (and to everyone else who replied) for a really good bunch of
suggestions.

Packing ten bytes at a time seems to be optimal and has shaved a whole
load of the execution time.

Thanks again,
Richard

--
Posted via http://www.ruby-forum.com/.

 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
shifting bits, shift 32 bits on 32 bit int GGG C++ 10 07-06-2006 06:09 AM
what about unsigned and signed 8 bits number, 16 bits, etc?? sarmin kho Python 2 06-15-2004 06:40 PM
8 bits/ch vs 16 bits/ch in PS Terry Digital Photography 5 01-21-2004 06:59 PM
8-Bits vs 12 or 16 bits/pixel; When does more than 8 bits count ? Al Dykes Digital Photography 3 12-29-2003 07:08 PM
win XP 32 bits on a 64 bits processor.. Abbyss Computer Support 3 11-13-2003 12:39 AM



Advertisments