Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Ruby > Integer/Float oddity

Reply
Thread Tools

Integer/Float oddity

 
 
Lutz
Guest
Posts: n/a
 
      04-24-2005

wrote:
> Glenn Parker wrote:
> > wrote:
> > > not really an oddity, more a lack of understanding on my part

> regarding
> > > ruby's internal type conversion...
> > >
> > > (100*(1.15-1))
> > > => 15.0
> > >
> > > (100*(1.15-1)).to_int
> > > => 14
> > >
> > > (100*(1.15-1)).to_int.round
> > > => 15

> >
> > FWIW, I get 14. Perhaps you meant (100*(1.15-1)).round.to_int ?
> >

>
> oops, yep thats what i meant. thanks for all the replies to my

query;
> i figured it had something to do with the internal representation of

fp
> numbers.
>
> this whole thing started as a conversation i was having with my dad
> about the relative merits of various programming languages. he is a
> die-hard xbasic user, and pointed me to the following post on an xb
> list:
>
> ---------------------------------------------------------------------
>
> [snip]
>
> Message: 7
> Date: Wed, 30 Mar 2005 12:59:42 -0000
> From: "Bruno Schaefer" <bup.schaefer@...>
> Subject: XBasic is one of the best!
>
>
> I'm back now to work with XBasic, because there are strange things
> with other languages.
>
> If you try to calculate the following simple expression
>
> INT(100*(1.15-1))
>
> XBasic produces the correct result: 15.
>
> This is a calculation, which is solved by children easy.
>
> But try this with other languages
> (e.g. newLISP, Visual Basic, Liberty Basic, IBasic, Yabasic, Python)
> you will get the result
>
> 14 !!!!?
>
> What a nonsense ! Does somebody understand this ?
>
> This is one the reason for me to use XBasic!
>
> ---------------------------------------------------------------------
> i'm assuming that since this behaviour is exhibited in ruby and the
> rest of the above "other languages" and NOT xbasic that it is really
> due to an idiosyncracy of xbasic that the "right" answer comes out.
>
> so the question becomes, what is different about xbasic's internal
> numeric representation that allows it to give a correct answer to

this
> seemingly simple number manipulation?
>
> horati0


 
Reply With Quote
 
 
 
 
Lutz
Guest
Posts: n/a
 
      04-24-2005
Actually newLISP produces the correct answer:

(mul 100 (sub 1.15 1)) => 15

You probably used * and - for integer arithmetik only, instead of mul
and sub for floats.

Lutz

www.newlisp.org

when contacting me via email put the word newlisp in the subject line.

 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
One More Wireless-to-Wired Oddity J Wireless Networking 1 01-02-2006 06:37 PM
pix policy nat small oddity Walter Roberson Cisco 0 07-12-2005 02:07 PM
PIX 501 PDM Oddity Hank Zoeller Cisco 2 05-20-2005 02:48 AM
803 password recovery oddity Cas Cisco 5 05-13-2005 09:41 AM
split commands oddity rxl124@hehe.com Perl 3 01-29-2004 07:59 AM



Advertisments
 



1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57