Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Javascript > FAQ Topic - Why does simple decimal arithmetic give strange results?

Reply
Thread Tools

FAQ Topic - Why does simple decimal arithmetic give strange results?

 
 
FAQ server
Guest
Posts: n/a
 
      08-15-2007
-----------------------------------------------------------------------
FAQ Topic - Why does simple decimal arithmetic give strange
results?
-----------------------------------------------------------------------

For example, 5*1.015 does not give exactly 5.075 and 0.06+0.01 does
not give exactly 0.07 in javascript.

Javascript numbers are represented in binary as IEEE-754 (IEC 559)
Doubles, with a resolution of 53 bits, giving an accuracy of
15-16 decimal digits; integers up to about 9e15 are precise, but
few decimal fractions are. Given this, arithmetic is as exact
as possible, but no more. Operations on integers are exact if
the true result and all intermediates are integers within that
range.

In particular, non-integer results should not normally be
compared for equality; and non-integer computed results
commonly need rounding; see 4.6.

http://msdn.microsoft.com/library/de...6dbf483b41.asp


===
Postings such as this are automatically sent once a day. Their
goal is to answer repeated questions, and to offer the content to
the community for continuous evaluation/improvement. The complete
comp.lang.javascript FAQ is at http://jibbering.com/faq/index.html.
The FAQ workers are a group of volunteers.

 
Reply With Quote
 
 
 
 
Dr J R Stockton
Guest
Posts: n/a
 
      08-16-2007
In comp.lang.javascript message <46c38575$0$90264$(E-Mail Removed).
dk>, Wed, 15 Aug 2007 23:00:01, FAQ server <(E-Mail Removed)>
posted:
>
>For example, 5*1.015 does not give exactly 5.075 and 0.06+0.01 does
>not give exactly 0.07 in javascript.
>


True; but "exactly 0.07" is ambiguous. How best can we state, briefly
BOTH that (0.06+0.01).toString() -> "0.06999999999999999"
AND that (0.06+0.01) - 0.07 != 0.0 // ??

The recent short list of FAQ changes has an error.

--
(c) John Stockton, Surrey, UK. ?@merlyn.demon.co.uk Turnpike v6.05 MIME.
Web <URL:http://www.merlyn.demon.co.uk/> - FAQish topics, acronyms, & links.
Proper <= 4-line sig. separator as above, a line exactly "-- " (SonOfRFC1036)
Do not Mail News to me. Before a reply, quote with ">" or "> " (SonOfRFC1036)
 
Reply With Quote
 
 
 
 
John Hosking
Guest
Posts: n/a
 
      08-17-2007
Dr J R Stockton wrote:
>
> The recent short list of FAQ changes has an error.


Well, you're about as helpful as a government employee.

--
John Hosking
 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
FAQ Topic - How do I convert a Number into a String with exactly 2 decimal places? FAQ server Javascript 52 02-07-2007 04:57 PM
findcontrol("PlaceHolderPrice") why why why why why why why why why why why Mr. SweatyFinger ASP .Net 2 12-02-2006 03:46 PM
FAQ Topic - How do I convert a Number into a String with exactly 2 decimal places? FAQ server Javascript 6 11-03-2006 11:14 PM
FAQ Topic - Why does parseInt('09') give an error? FAQ server Javascript 0 10-21-2006 11:00 PM
FAQ Topic - Why does parseInt('09') give an error? FAQ server Javascript 24 09-20-2006 08:15 AM



Advertisments