Velocity Reviews > Calculations on LARGE numbers

# Calculations on LARGE numbers

Frinton
Guest
Posts: n/a

 03-13-2006
Hi,

I am trying to do some calculations on large numbers (ie
7,768,489,957,892,578,474,792,094 / 12,280) and no matter what I do it
doesn't get it quite right. Its always somewhere between 10 and and 5000 out

I have a suspition is could be down to one of the number functions I am
using along the way but im not sure.

Any help or guidance would be greatly appriciated.

Fred

Richard Mueller
Guest
Posts: n/a

 03-13-2006
Fred wrote:

> I am trying to do some calculations on large numbers (ie
> 7,768,489,957,892,578,474,792,094 / 12,280) and no matter what I do it
> doesn't get it quite right. Its always somewhere between 10 and and 5000
> out
>
> I have a suspition is could be down to one of the number functions I am
> using along the way but im not sure.
>
> Any help or guidance would be greatly appriciated.

VB and VBScript integers are Long datatypes (32-bit) and range
from -2,147,483,648 to 2,147,483,647, which is -2^31 to 2^31-1. Larger
values are handled as Float datatype which is 64-bit. Your large number is
stored as an approximation.

--
Richard
Hilltop Lab - http://www.rlmueller.net

Egbert Nierop \(MVP for IIS\)
Guest
Posts: n/a

 03-13-2006

"Frinton" <(E-Mail Removed)> wrote in message
news:44158bca@212.67.96.135...
> Hi,
>
> I am trying to do some calculations on large numbers (ie
> 7,768,489,957,892,578,474,792,094 / 12,280) and no matter what I do it
> doesn't get it quite right. Its always somewhere between 10 and and 5000
> out

The best is to convert it to currency.
dim v
v=CCur(10)
v=v * ccur(103030303030.1034)

> I have a suspition is could be down to one of the number functions I am
> using along the way but im not sure.

Evertjan.
Guest
Posts: n/a

 03-13-2006
Frinton wrote on 13 mrt 2006 in microsoft.public.inetserver.asp.general:

> I am trying to do some calculations on large numbers (ie
> 7,768,489,957,892,578,474,792,094 / 12,280) and no matter what I do it
> doesn't get it quite right. Its always somewhere between 10 and and
> 5000 out
>
>
> I have a suspition is could be down to one of the number functions I
> am using along the way but im not sure.
>
> Any help or guidance would be greatly appriciated.

<http://en.wikipedia.org/wiki/Long_division>

--
Evertjan.
The Netherlands.

Frinton
Guest
Posts: n/a

 03-13-2006

Is there any work around to this?

Im storing the values in SQL, is BigInt the largest exact number possible?

Thanks

Fred

"Richard Mueller" <(E-Mail Removed)> wrote in message
news:(E-Mail Removed)...
> Fred wrote:
>
>> I am trying to do some calculations on large numbers (ie
>> 7,768,489,957,892,578,474,792,094 / 12,280) and no matter what I do it
>> doesn't get it quite right. Its always somewhere between 10 and and 5000
>> out
>>
>> I have a suspition is could be down to one of the number functions I am
>> using along the way but im not sure.
>>
>> Any help or guidance would be greatly appriciated.

>
> VB and VBScript integers are Long datatypes (32-bit) and range
> from -2,147,483,648 to 2,147,483,647, which is -2^31 to 2^31-1. Larger
> values are handled as Float datatype which is 64-bit. Your large number is
> stored as an approximation.
>
> --
> Richard
> Microsoft MVP Scripting and ADSI
> Hilltop Lab - http://www.rlmueller.net
>

Frinton
Guest
Posts: n/a

 03-13-2006

Im getting an overflow when I try this

Fred

"Egbert Nierop (MVP for IIS)" <(E-Mail Removed)> wrote in
message news:(E-Mail Removed)...
>
> "Frinton" <(E-Mail Removed)> wrote in message
> news:44158bca@212.67.96.135...
>> Hi,
>>
>> I am trying to do some calculations on large numbers (ie
>> 7,768,489,957,892,578,474,792,094 / 12,280) and no matter what I do it
>> doesn't get it quite right. Its always somewhere between 10 and and 5000
>> out

>
> The best is to convert it to currency.
> dim v
> v=CCur(10)
> v=v * ccur(103030303030.1034)
>
>> I have a suspition is could be down to one of the number functions I am
>> using along the way but im not sure.

>

Frinton
Guest
Posts: n/a

 03-13-2006

Its not the calculation that is causing me the problem its the storing of
values

Fred

"Evertjan." <(E-Mail Removed)> wrote in message
news:Xns9785B2358D898eejj99@194.109.133.242...
> Frinton wrote on 13 mrt 2006 in microsoft.public.inetserver.asp.general:
>
>> I am trying to do some calculations on large numbers (ie
>> 7,768,489,957,892,578,474,792,094 / 12,280) and no matter what I do it
>> doesn't get it quite right. Its always somewhere between 10 and and
>> 5000 out
>>
>>
>> I have a suspition is could be down to one of the number functions I
>> am using along the way but im not sure.
>>
>> Any help or guidance would be greatly appriciated.

>
> <http://en.wikipedia.org/wiki/Long_division>
>
>
> --
> Evertjan.
> The Netherlands.

Tim Slattery
Guest
Posts: n/a

 03-13-2006
"Frinton" <(E-Mail Removed)> wrote:

>
>Its not the calculation that is causing me the problem its the storing of
>values

In VB or VBScript there's probably no way out of this. Java (not
JavaScript) has a BigInteger class, and there are classes available
for C++ that can handle integers of arbitrary precision.

Arithmetic operations in these classes would not be fast, but they
would preserve and use the full precision.

--
Tim Slattery
MS MVP(DTS)
http://www.velocityreviews.com/forums/(E-Mail Removed)

Jerold Schulman
Guest
Posts: n/a

 03-13-2006
On Mon, 13 Mar 2006 15:12:09 -0000, "Frinton" <(E-Mail Removed)> wrote:

>Hi,
>
>I am trying to do some calculations on large numbers (ie
>7,768,489,957,892,578,474,792,094 / 12,280) and no matter what I do it
>doesn't get it quite right. Its always somewhere between 10 and and 5000 out
>
>
>I have a suspition is could be down to one of the number functions I am
>using along the way but im not sure.
>
>Any help or guidance would be greatly appriciated.
>
>
>Fred
>

See tip 4533 » How do I perform accurate and/or complex math in a batch? 11-Dec-01
in the 'Tips & Tricks' at http://www.jsifaq.com

When I type the following at a CMD.EXE prompt:
for /f %i in ('domath //nologo "Round(7768489957892578474792094/12280)"') do @echo %i

I get 6.32613188753467E+20

Jerold Schulman
Windows Server MVP
JSI, Inc.
http://www.jsiinc.com
http://www.jsifaq.com

Frinton
Guest
Posts: n/a

 03-13-2006

"Jerold Schulman" <(E-Mail Removed)> wrote in message
news:(E-Mail Removed)...
> On Mon, 13 Mar 2006 15:12:09 -0000, "Frinton" <(E-Mail Removed)>
> wrote:
>
>>Hi,
>>
>>I am trying to do some calculations on large numbers (ie
>>7,768,489,957,892,578,474,792,094 / 12,280) and no matter what I do it
>>doesn't get it quite right. Its always somewhere between 10 and and 5000
>>out
>>
>>
>>I have a suspition is could be down to one of the number functions I am
>>using along the way but im not sure.
>>
>>Any help or guidance would be greatly appriciated.
>>
>>
>>Fred
>>

> See tip 4533 » How do I perform accurate and/or complex math in a batch?
> 11-Dec-01
> in the 'Tips & Tricks' at http://www.jsifaq.com
>
> When I type the following at a CMD.EXE prompt:
> for /f %i in ('domath //nologo "Round(7768489957892578474792094/12280)"')
> do @echo %i
>
> I get 6.32613188753467E+20
>
>
>
>
> Jerold Schulman
> Windows Server MVP
> JSI, Inc.
> http://www.jsiinc.com
> http://www.jsifaq.com