Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Ruby > Marshal's handling of floats

Reply
Thread Tools

Marshal's handling of floats

 
 
Brian Palmer
Guest
Posts: n/a
 
      07-09-2006
I was thinking about writing a patch to modify how Marshal handles
floats, right now it dumps them using sprintf(3) and stores the
resulting string in the Marshal stream. I'd like to see it handle
floats the same way that Array#pack does:

[400.53].pack('g').length == 4
[400.53].pack('G').length == 8

while

Marshal.dump(400.53).length - 3 == 22
(and is slower, to boot)

I want to make sure, though, that this would be an acceptable patch.
I can't think why it would be OK for Array#pack to work this way and
not Marshal, but is there any particular reason why it can't be done?
Obviously it would break backwards compatability with older Marshal
dumps, but I don't think they're often used for long-term storage,
are they?

-- Brian Palmer


 
Reply With Quote
 
 
 
 
ara.t.howard@noaa.gov
Guest
Posts: n/a
 
      07-09-2006
On Sun, 9 Jul 2006, Brian Palmer wrote:

> I was thinking about writing a patch to modify how Marshal handles floats,
> right now it dumps them using sprintf(3) and stores the resulting string in
> the Marshal stream. I'd like to see it handle floats the same way that
> Array#pack does:
>
> [400.53].pack('g').length == 4
> [400.53].pack('G').length == 8
>
> while
>
> Marshal.dump(400.53).length - 3 == 22
> (and is slower, to boot)
>
> I want to make sure, though, that this would be an acceptable patch. I can't
> think why it would be OK for Array#pack to work this way and not Marshal, but
> is there any particular reason why it can't be done? Obviously it would break
> backwards compatability with older Marshal dumps, but I don't think they're
> often used for long-term storage, are they?
>
> -- Brian Palmer


i've never tried to use marshaled data across a big and little endian machine
- but this would break it. consider drb: if you had a mac and a linux box
talking on the wire you might see

harp:~ > ruby -e' puts [1.44417819733316e-41].pack("g").reverse.unpack("g")[0].to_i '
42

which could be confusing. then again maybe i'm overlooking something.

cheers.

-a
--
suffering increases your inner strength. also, the wishing for suffering
makes the suffering disappear.
- h.h. the 14th dali lama

 
Reply With Quote
 
 
 
 
Brian Palmer
Guest
Posts: n/a
 
      07-09-2006
Hi Ara,

The 'g' and 'G' flags for Array#pack/String#unpack are in network
byte order, so they're in a platform-independent format, as far as I
know. I actually tested this by Packing a couple thousand floats on
my Mac, sending them in UDP packets and unpacking them on my AMD64
desktop, they all came across correctly.

-- Brian

On Jul 8, 2006, at 11:40 PM, http://www.velocityreviews.com/forums/(E-Mail Removed) wrote:

> On Sun, 9 Jul 2006, Brian Palmer wrote:
>
>> I was thinking about writing a patch to modify how Marshal handles
>> floats, right now it dumps them using sprintf(3) and stores the
>> resulting string in the Marshal stream. I'd like to see it handle
>> floats the same way that Array#pack does:
>>
>> [400.53].pack('g').length == 4
>> [400.53].pack('G').length == 8
>>
>> while
>>
>> Marshal.dump(400.53).length - 3 == 22
>> (and is slower, to boot)
>>
>> I want to make sure, though, that this would be an acceptable
>> patch. I can't think why it would be OK for Array#pack to work
>> this way and not Marshal, but is there any particular reason why
>> it can't be done? Obviously it would break backwards compatability
>> with older Marshal dumps, but I don't think they're often used for
>> long-term storage, are they?
>>
>> -- Brian Palmer

>
> i've never tried to use marshaled data across a big and little
> endian machine
> - but this would break it. consider drb: if you had a mac and a
> linux box
> talking on the wire you might see
>
> harp:~ > ruby -e' puts [1.44417819733316e-41].pack
> ("g").reverse.unpack("g")[0].to_i '
> 42
>
> which could be confusing. then again maybe i'm overlooking something.
>
> cheers.
>
> -a
> --
> suffering increases your inner strength. also, the wishing for
> suffering
> makes the suffering disappear.
> - h.h. the 14th dali lama
>



 
Reply With Quote
 
Robert Klemme
Guest
Posts: n/a
 
      07-10-2006
Brian Palmer <(E-Mail Removed)> wrote:
> Hi Ara,
>
> The 'g' and 'G' flags for Array#pack/String#unpack are in network
> byte order, so they're in a platform-independent format, as far as I
> know. I actually tested this by Packing a couple thousand floats on
> my Mac, sending them in UDP packets and unpacking them on my AMD64
> desktop, they all came across correctly.


Yeah, but I still see serious drawbacks of your proposal:

- it will break existing code (YAML data)

- it's not human readable which is one of the key advantages (YAML files
seem to be frequently written by hand as a nice user interface to config
data)

- it's not portable and it might not be compliant with YAML spec

In sum all these are serious showstoppers for a general change of YAML
behavior.

Another solution - if you want to make use of this: AFAIK you can customize
YAMLfication of classes. If that was true, you can pretty easily perform
the conversion during to conversion to and from YAML. That would be the way
to go IMHO.

Kind regards

robert

 
Reply With Quote
 
Joel VanderWerf
Guest
Posts: n/a
 
      07-10-2006
Robert Klemme wrote:
> Brian Palmer <(E-Mail Removed)> wrote:
>> Hi Ara,
>>
>> The 'g' and 'G' flags for Array#pack/String#unpack are in network
>> byte order, so they're in a platform-independent format, as far as I
>> know. I actually tested this by Packing a couple thousand floats on
>> my Mac, sending them in UDP packets and unpacking them on my AMD64
>> desktop, they all came across correctly.

>
> Yeah, but I still see serious drawbacks of your proposal:
>
> - it will break existing code (YAML data)
>
> - it's not human readable which is one of the key advantages (YAML files
> seem to be frequently written by hand as a nice user interface to config
> data)
>
> - it's not portable and it might not be compliant with YAML spec
>
> In sum all these are serious showstoppers for a general change of YAML
> behavior.


I thought the OP wanted to change the behavior of Marshal, not YAML. Or
are you saying that the YAML rep'n would also have to change so that
both formats have the same lossiness?

--
vjoel : Joel VanderWerf : path berkeley edu : 510 665 3407

 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
signal handling and (structured) exception handling Peter C++ 34 10-17-2009 10:03 AM
python list handling and Lisp list handling Mark Tarver Python 22 04-26-2009 09:36 PM
Is faster handling hexadecimal values than handling chars? IƱaki Baz Castillo Ruby 1 04-15-2008 09:04 AM
check for divisor equal to zero if when handling floats? Giff C++ 4 05-15-2007 09:32 AM
Floats to chars and chars to floats Kosio C Programming 44 09-23-2005 09:49 AM



Advertisments