Velocity Reviews > RE: prePEP: Decimal data type

# RE: prePEP: Decimal data type

Tim Peters
Guest
Posts: n/a

 11-05-2003
[Batista, Facundo]
> Nice picture, . But in Money you *want* to get rounded in the
> decimal places.
>
> Example of decimal with precision = 100:
>
> 1122334455667788 --> 1122334455667788
> 1122334455.667788 --> 1122334455.667788
>
> But in Money, you may want to get rounded to two decimals:
>
> 1122334455.667788 --> 1122334455.67

What are these example of? Literals? If you don't want 40 digits after the
decimal point in a literal, don't write 40 digits after the decimal point in
a literal. Or, if you have to, use the spec's quantize() function to round
to the number of fractional decimal digits you want. After that, + and -
take care of themselves (they neither increase nor decrease the number of
decimal digits after the radix point, and IBM's arithmetic *remembers*
that -- e.g., 3.12 - 1.12 is 2.00 in that scheme, not 2 or 2.0 or 2.000).
You do need to explicitly round (if desired) after * and /. Because of the
high value attached to 100% conformance to imposed standards in accounting
apps, I think it's a good thing that you're forced to round explicitly when
the result of a calculation can't be represented exactly in the mandated
format -- those are exactly the places where your app will fail to meet the
requirements, so it's good to be aware of them.

> ...
> So, unbounded precision and fixed point are the same? (I always
> assumed that)

It depends on who you ask, but most people would probably say "no". "Fixed
point" has no fixed meaning -- it's been applied to a plethora of schemes
over the decades, and each meaning has its own gang of frightened defenders
<0.9 wink>. My FixedPoint class provided an historically peculiar meaning
(fixed # of digits after the decimal point, unbounded # of digits before
it), but one I found useful at the time. Most meanings of "fixed point"
have in mind a fixed number of digits both before and after the decimal
point, and rarely the same number of digits before as after. Variants in
all areas are rampant.

For example, for multiplication Java's BigDecimal class gives a number of
digits after the decimal point in the product equal to the sum of the number
of digits after the decimal points in both inputs; in the same case,
FixedPoint rounds the infinitely-precise result to the larger of the number
of digits after the decimal point across both inputs.

Both schemes are crazy for some apps. For example, if there's 8.875% tax on
a \$1.01 item,

1.08875 * 1.01 = 1.0996375 # BigDecimal
= 1.09964 # FixedPoint

are both probably useless as-is. Under the IBM spec, you multiply and then
call quantize with a second argument of (for example) 0.01. Assuming
banker's rounding, that gives the 1.10 you're probably seeking. Or maybe
that's not what you want -- maybe your municipality wants the excess
truncated. They're both equally easy (and equally difficult <wink>) under
the IBM spec. What doesn't work is hoping that some class will do rounding
correctly in all cases by magic.

Alex Martelli
Guest
Posts: n/a

 11-06-2003
Tim Peters wrote:
...
> You do need to explicitly round (if desired) after * and /. Because of
> the high value attached to 100% conformance to imposed standards in
> accounting apps, I think it's a good thing that you're forced to round
> explicitly when the result of a calculation can't be represented exactly
> in the mandated format -- those are exactly the places where your app will
> fail to meet the requirements, so it's good to be aware of them.

European Union directives (adopted as laws by member states) mandate
the rounding procedure to be used in computations involving Euros (round
to closest Eurocent, always round up on half-Eurocent results); they very
explicitly mention that this may give a 1-Eurocent discrepancy compared
to "exact" arithmetic, and give examples; they establish that such a 1-cent
discrepancy that comes from following exactly the prescribed rules is NOT
legal cause for any lawsuit whatsoever; they earnestly recommend that all
computing equipment and programs follow these same rules to avoid the huge
headaches that would result in trying to reconcile accounts otherwise.

Thus, for most accounting programs intended to run within the EU (not just
in Euros: these provisions also apply to the other non-Euro currencies, as
far as EU law is concerned), I do NOT think it would be a good thing for
the programmer to have to remember to round explicitly -- the legal mandate
is about rounding rules and it's quite easy to avoid the "fail to meet the
requirements", as they seem designed to be easy to meet.

Whether Decimal itself allows an optional rounding-policy (including of
course "no rounding" as a possibility) is one issue (I guess that might
violate some ANSI or IEEE standard...?) but I most surely do want to be
able to use such policies in whatever arithmetic type underlies Money --
so I hope Decimal is at least designed so that _subclasses_ can easily
provide such customized rounding (e.g., feature-testing for a __round__
specialmethod, not defined in the baseclass Decimal if need be, but
offering the needed hook for subclasses to add the functionality).

Alex

Emile van Sebille
Guest
Posts: n/a

 11-06-2003
Alex Martelli:
> Thus, for most accounting programs intended to run within the EU

(not just
> in Euros: these provisions also apply to the other non-Euro

currencies, as
> far as EU law is concerned), I do NOT think it would be a good thing

for
> the programmer to have to remember to round explicitly -- the legal

mandate
> is about rounding rules and it's quite easy to avoid the "fail to

meet the
> requirements", as they seem designed to be easy to meet.
>

These rules must apply to the subset of accounting situations that
involve exchange of funds. However, pricing and costing of some items
regularly require a number of post-decimals beyond the '3 then round'
as you say is mandated. An encforced rounding policy provided by a
Money subclass would require the programmer to invent work arounds for
these situations. EIBTI (Cool! I just learned what that means.)

--
Emile van Sebille
http://www.velocityreviews.com/forums/(E-Mail Removed)

Aahz
Guest
Posts: n/a

 11-06-2003
In article <Xwnqb.422473\$(E-Mail Removed)>,
Alex Martelli <(E-Mail Removed)> wrote:
>Tim Peters wrote:
>>
>> You do need to explicitly round (if desired) after * and /. Because of
>> the high value attached to 100% conformance to imposed standards in
>> accounting apps, I think it's a good thing that you're forced to round
>> explicitly when the result of a calculation can't be represented exactly
>> in the mandated format -- those are exactly the places where your app will
>> fail to meet the requirements, so it's good to be aware of them.

>
>European Union directives (adopted as laws by member states)
>mandate the rounding procedure to be used in computations involving
>Euros (round to closest Eurocent, always round up on half-Eurocent
>results); they very explicitly mention that this may give a 1-Eurocent
>discrepancy compared to "exact" arithmetic, and give examples; they
>establish that such a 1-cent discrepancy that comes from following
>exactly the prescribed rules is NOT legal cause for any lawsuit
>whatsoever; they earnestly recommend that all computing equipment and
>programs follow these same rules to avoid the huge headaches that would
>result in trying to reconcile accounts otherwise.

Yeah, but *what* do you round? If you purchase five items in a store,
does tax get applied to each item individually or to the total? There
are lots of similar cases, so rounding must still be at least partially
explicit.

>Whether Decimal itself allows an optional rounding-policy (including
>of course "no rounding" as a possibility) is one issue (I guess that
>might violate some ANSI or IEEE standard...?) but I most surely do want
>to be able to use such policies in whatever arithmetic type underlies
>Money -- so I hope Decimal is at least designed so that _subclasses_
>can easily provide such customized rounding (e.g., feature-testing
>for a __round__ specialmethod, not defined in the baseclass Decimal
>if need be, but offering the needed hook for subclasses to add the
>functionality).

Once again, I suggest that you read Cowlishaw.
--
Aahz ((E-Mail Removed)) <*> http://www.pythoncraft.com/

"It is easier to optimize correct code than to correct optimized code."
--Bill Harlan

Bengt Richter
Guest
Posts: n/a

 11-06-2003
On Thu, 06 Nov 2003 08:22:15 GMT, Alex Martelli <(E-Mail Removed)> wrote:

>Tim Peters wrote:
> ...
>> You do need to explicitly round (if desired) after * and /. Because of
>> the high value attached to 100% conformance to imposed standards in
>> accounting apps, I think it's a good thing that you're forced to round
>> explicitly when the result of a calculation can't be represented exactly
>> in the mandated format -- those are exactly the places where your app will
>> fail to meet the requirements, so it's good to be aware of them.

>
>European Union directives (adopted as laws by member states) mandate
>the rounding procedure to be used in computations involving Euros (round
>to closest Eurocent, always round up on half-Eurocent results); they very
>explicitly mention that this may give a 1-Eurocent discrepancy compared
>to "exact" arithmetic, and give examples; they establish that such a 1-cent
>discrepancy that comes from following exactly the prescribed rules is NOT
>legal cause for any lawsuit whatsoever; they earnestly recommend that all
>computing equipment and programs follow these same rules to avoid the huge
>headaches that would result in trying to reconcile accounts otherwise.
>
>Thus, for most accounting programs intended to run within the EU (not just
>in Euros: these provisions also apply to the other non-Euro currencies, as
>far as EU law is concerned), I do NOT think it would be a good thing for
>the programmer to have to remember to round explicitly -- the legal mandate
>is about rounding rules and it's quite easy to avoid the "fail to meet the
>requirements", as they seem designed to be easy to meet.

I wonder how they deal with compound interest. E.g., may a contract for a loan
express interest as mathematically continuous between payments? In that case
does rounding from mathematically exact values occur only at payment times?
Is the rounding only applied to the payment amount, so that the princal is
decreased by the exact quantized payment, but the balance retains exactness?
Or is requantization of all values mandated at each transaction? Is each step in
a sum a separate transaction? What about the rules for pre-tax-calculation subtotals
vs accumulating actual receipt stated tax amounts? I remember as a kid bugging
store clerks to sell me two candies separately to stay under the minimum taxable
price threshold twice rather than exceed it once and pay the extra penny

ISTM these things are 'way beyond the concern of a math package per se, yet a math package
that is intended to support programming of a business/rule/law-abiding application
would have to make it easy to express in program source what the unambiguous intent is,
and not have the math be playing unexpected tricks with the values.

IMO to have rounding/precision applied sub-expression by subexpression is a recipe
for unpleasant surprises. It would be like applying snap-to-grid rules in a drawing
package at every internal intermediate math subexpression evaluation step of the
implementation. BTW, does this mean expressions can't be run through optimizers that
might e.g., hoist a subexpression out of a loop and thereby change how many times
a rounding effect happens? What about advice to eager programmers who want to do the same
optimization by hand? Must they be advised? ISTM programmers will not want to face
these issues in internal implementation, and will instead turn up precision to
close-enough-to-exact, and then just round when they print or interact with databases etc.,
ajusting ad hoc to correct infractions of rules revealed in testing. Chances are that's
what they would wind up doing anyway, using step-wise-rounding-enforcing math.

>
>Whether Decimal itself allows an optional rounding-policy (including of
>course "no rounding" as a possibility) is one issue (I guess that might
>violate some ANSI or IEEE standard...?) but I most surely do want to be
>able to use such policies in whatever arithmetic type underlies Money --
>so I hope Decimal is at least designed so that _subclasses_ can easily
>provide such customized rounding (e.g., feature-testing for a __round__
>specialmethod, not defined in the baseclass Decimal if need be, but
>offering the needed hook for subclasses to add the functionality).
>

Agreed -- that something is needed to make defining and coding special rules on top
of the math base easy. I was thinking subclassing too. OTOH, casting about
for the meaning of "context" in this PEP context (, I think you could also
develop an implementation convention around a separate Context class, that
would provide instances as customized attribute-name-space contexts, using
__[gs]etattr__ overrides and/or properties, maybe with a metaclass to make parameterized
customization easy to write. Using this sort of thing, you could cause property/attribute
retrieval to return rule-enforcing wrapper objects with money quantities inside,
and any statements and math expressions written in terms of those would
be defined by the wrappers, and assignments would be defined by the context object
whose attribute space was being assigned to.

So long as people are willing write stuff like, e.g., e.total and e.tax
for total and tax values they want operated upon subject to, e.g., custom
Euro rules, IWT that could work.

But I think it's hard to anticipate what will work out slick, so some alternative
implementation sketches for a realistic use case would help clarify. Better
to concentrate on powerful orthogonal primitives and test them against a reference
problem than provide too much of what amounts to speculative app macros, IMO.

Regards,
Bengt Richter

Guest
Posts: n/a

 11-08-2003
On Thu, 6 Nov 2003 10:43:52 -0500, "Tim Peters" <(E-Mail Removed)>
wrote:
>I'd be astonished if the rules were consistent enough so that a type
>implementing a fixed number of decimal digits "after the decimal point"
>would be of real use for non-experts ... OK, here from:
>
>
> The conversion of prices into euros will require the use of a six-
> significant-digit conversion rate (six digits disregarding initial
>zeros)
> which should not be rounded or truncated. This rate will be irrevocably
> fixed on 1 January 1999 and defined in the form of one euro expressed in
> national currencies.
>
>So that part mandates a *floating* point input (not meaning binary floating
>point, but "6 digits disregarding initial zeros" is the essence of floating
>point -- the total number of digits isn't fixed, nor is the # of digits
>after the decimal point fixed, just the # of *significant* digits).
>
> To convert from national currencies to the euro, one has to divide by
> the conversion rate. To convert from the euro to the national currency,
> one has to multiply by the conversion rate. The use of inverse rates is
> forbidden.
>
>Neutral.
>
> To convert from one national currency to another, amounts must be first
> converted into euros and then into the second national currency. The
> euro amount must be rounded to three decimal places. The national
> currency should then be rounded to two decimals.
>
>So no single fixed-point discipline can suffice: in one direction they want
>rounding to 3 digits after the decimal point, in the other to 2 digits, and
>one of the inputs is a floating-point value with no fixed number of digits
>after the decimal point.
>
>This is all very easily done with IBM's proposed arithmetic, but it requires
>the programmer to specify explicitly how many places after the decimal point
>they want after each * and / operation. Since that value changes in
>arbitrarily (from the POV of arithmetic semantics) mandated ways, no class
>can guess what's required from step to step.
>

I agree with you on this, and this is an excellent example of the
situation.

I believe a general use built in decimal type which addresses the
problems with floating point would be great. It should have good
general numeric tools to manipulate the numbers and digits, including
rounding up or down but maybe not rounding even. A function to count
the number of digits before and after the decimal place might be good.
I just looked for that one and didn't see it in the math.py or
cmath.py modules. I was thinking of using it to round the decimal
portion of a number to one less digit to compensate for the floating
point internal base two representation.

As for accounting specific features, I really think a good separate
accounting math module would be better choice. It could have a rich
set of functions for calculating interest rates and Time-Value-Money
calculations such as mortgage payments, and loan Amortization lists
along with round_even and include most functions that one would find
on a business calculator or in a spreadsheet for that purpose. There
could also be euro/country specific functions if they are needed. A
module such as this may be able to support localization better than
trying to build it in.

I don't know (yet) how python handles things internally since I'm
still new with it, but I lean towards not letting the core get too
bloated or specialized, especially if it will slow it down in any way.
When it comes to numbers, speed is important.

By the way, I haven't seen anything yet on how fast the decimal type
will be compared to floats and ints? From reading the specs on it I
get the impression its slower than ints but has compression so uses
less memory, but I was wandering how much slower? Or maybe it isn't
slower if it is handled as an integer with a base ten integer
exponent?

(E-Mail Removed)