COBOL Manual for TNS and TNS/R Programs

Procedure Division
HP COBOL Manual for TNS and TNS/R Programs—522555-006
8-52
Arithmetic Precision
The number of decimal places in the intermediate result (DR) is chosen first, then the
number of integer places in the intermediate result is determined from that and the
characteristics of the two operands.
HP COBOL guarantees that the mathematical significance of results will be at least as
good as application of these rules implies.
When an arithmetic expression involves a division operation, the intermediate results
are evaluated in these steps:
1. The actual division
2. The adjustment of that result for use in further computations
Therefore, in each of these instances of the arithmetic expression “A1/A2+A3*A4,”
COMPUTE AX = A1/A2 + A3 * A4
IF A1/A2 + A3 * A4 LESS THAN AX GO TO ...
the division is performed before further evaluation of either of the preceding
statements. If the division operation is not the last operation in the expression and the
divisor is not 2, 4, 5, 8, or a power of 10, the division is carried out to 36 digits. As
many fractional digits as possible are kept. This number can be truncated depending
on subsequent operations. In some cases adjustment is not possible, so the compiler
uses binary floating-point arithmetic (and issues warning 85). Since the precision of
floating-point arithmetic is 16 digits and numbers cannot always be represented
exactly, the result might be slightly larger or smaller than the exact number. You might
have to use individual operations or revise the expression to guarantee accurate
results.
To obtain the maximum accuracy in an arithmetic expression that involves a division,
use parentheses or revise the expression as necessary to assure that the division is
the last operation performed. For example, you can rewrite the expression “a / b * c" as
"(a * c) / b".
When a conditional expression compares a variable and an expression, the number of
decimal places carried for the variable is used for the number of decimal places carried
for the expression.
Operation
Decimal Places (Worst-Case Precision)
DR is the greater/greatest of … IR is the lesser of …
OP1 + OP2
OP1 - OP2
D1 and D2 (The greater of I1 and I2) + 1) and
(36 - DR)
OP1
* OP2 D1 and D2 (I1 + I2) and (36 - DR)
OP1 / OP2 D1 - D2, the D of the composite of
all the result fields, and 1
(I1 + D2) and (36 - DR)
OP1
** OP2 D1 or D2 (The greater of I1 or I2) and (36 - DR)
Note. If (I1+D2+DR) > 36, the compiler uses floating-point arithmetic (and issues warning 85).