Velocity Reviews > Ruby > Precision and ==

# Precision and ==

LAMBEAU Bernard
Guest
Posts: n/a

 04-07-2010
[Note: parts of this message were removed to make it a legal post.]

We all know that finite representation comes with some difficulties...
but...

x = 45.0*(Math:I / 180.0)
y = 0.785398163397448
puts "#{x.class} : #{x}"
puts "#{y.class} : #{y}"
puts 45.0*(Math:I / 180.0) == 0.785398163397448

prints

Float : 0.785398163397448
Float : 0.785398163397448
false

Does to_s show something less precise than the internal representation of
Floats?

thx
Bernard

Jesús Gabriel y Galán
Guest
Posts: n/a

 04-07-2010
On Wed, Apr 7, 2010 at 1:02 PM, LAMBEAU Bernard <(E-Mail Removed)> wrote:
> We all know that finite representation comes with some difficulties...
> but...
>
> x = 45.0*(Math:I / 180.0)
> y = 0.785398163397448
> puts "#{x.class} : #{x}"
> puts "#{y.class} : #{y}"
> puts 45.0*(Math:I / 180.0) == 0.785398163397448
>
> prints
>
> Float : 0.785398163397448
> Float : 0.785398163397448
> false
>
> Does to_s show something less precise than the internal representation of
> Floats?

Yes, it rounds to 15 decimal positions:

irb(main):001:0> x = 45.0*(Math:I / 180.0)
=> 0.785398163397448
irb(main):002:0> y = 0.785398163397448
=> 0.785398163397448
irb(main):003:0> puts "#{x.class} : #{"%.20f" % x}"
Float : 0.78539816339744827900
=> nil
irb(main):004:0> puts "#{y.class} : #{"%.20f" % y}"
Float : 0.78539816339744794593

Jesus.

 Posting Rules You may not post new threads You may not post replies You may not post attachments You may not edit your posts BB code is On Smilies are On [IMG] code is On HTML code is OffTrackbacks are On Pingbacks are On Refbacks are Off Forum Rules

 Similar Threads Thread Thread Starter Forum Replies Last Post Timothy Daniels Computer Support 0 10-27-2003 07:05 PM Rod Speed Computer Support 0 10-27-2003 06:42 PM Vincent Lefevre XML 4 08-21-2003 12:59 PM john smith C++ 1 08-05-2003 04:22 PM Erik2000 C++ 3 06-27-2003 10:29 AM