Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Ruby > Precision and ==

Reply
Thread Tools

Precision and ==

 
 
LAMBEAU Bernard
Guest
Posts: n/a
 
      04-07-2010
[Note: parts of this message were removed to make it a legal post.]

We all know that finite representation comes with some difficulties...
but...

x = 45.0*(Math:I / 180.0)
y = 0.785398163397448
puts "#{x.class} : #{x}"
puts "#{y.class} : #{y}"
puts 45.0*(Math:I / 180.0) == 0.785398163397448

prints

Float : 0.785398163397448
Float : 0.785398163397448
false

Does to_s show something less precise than the internal representation of
Floats?

thx
Bernard

 
Reply With Quote
 
 
 
 
Jesús Gabriel y Galán
Guest
Posts: n/a
 
      04-07-2010
On Wed, Apr 7, 2010 at 1:02 PM, LAMBEAU Bernard <(E-Mail Removed)> wrote:
> We all know that finite representation comes with some difficulties...
> but...
>
> x = 45.0*(Math:I / 180.0)
> y = 0.785398163397448
> puts "#{x.class} : #{x}"
> puts "#{y.class} : #{y}"
> puts 45.0*(Math:I / 180.0) == 0.785398163397448
>
> prints
>
> Float : 0.785398163397448
> Float : 0.785398163397448
> false
>
> Does to_s show something less precise than the internal representation of
> Floats?


Yes, it rounds to 15 decimal positions:

irb(main):001:0> x = 45.0*(Math:I / 180.0)
=> 0.785398163397448
irb(main):002:0> y = 0.785398163397448
=> 0.785398163397448
irb(main):003:0> puts "#{x.class} : #{"%.20f" % x}"
Float : 0.78539816339744827900
=> nil
irb(main):004:0> puts "#{y.class} : #{"%.20f" % y}"
Float : 0.78539816339744794593

Jesus.

 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Re: Formatting and Partitioning with Precision Timothy Daniels Computer Support 0 10-27-2003 07:05 PM
Re: Formatting and Partitioning with Precision Rod Speed Computer Support 0 10-27-2003 06:42 PM
XPath/XSLT and extended precision Vincent Lefevre XML 4 08-21-2003 12:59 PM
setting and resetting stream precision john smith C++ 1 08-05-2003 04:22 PM
Float and double, precision and scale?? Erik2000 C++ 3 06-27-2003 10:29 AM



Advertisments