DataLoader/MX Reference Manual (G06.24+)

Recovery Strategies
DataLoader/MX Reference Manual—525872-002
7-7
Batch Totals
Purging the Restart File
Before you purge the restart file, consider the way DataLoader/MX handles input and
output. It is possible for DataLoader/MX to successfully end without processing all its
input data.
Suppose that a process is writing to DataLoader/MX by using the process name of the
DataLoader/MX process. When that process terminates, DataLoader/MX receives an
EOF on its $RECEIVE file, whether that process terminates normally or abnormally.
DataLoader/MX then ends normally even though it has not read all input data, which
makes a rerun necessary. If you have purged the restart file, you are not able to rerun
DataLoader/MX.
When you set up your load scenario using restarting from a known state, plan carefully
how you will determine if DataLoader/MX has completed its job and when you can
safely purge the restart file.
Batch Totals
When performing a load job, you commonly accumulate batch totals of such things as
record counts, total dollar amounts, total units sold, and so on. These batch totals can
be used to confirm the integrity of the load operation. How can such batch totals be
accumulated for situations in which the batch is not applied to the database in one
continuous operation?
DataLoader/MX provides a user exit, SKIPPING, for restarting from a known state.
When DataLoader/MX performs a restart and skips records that have already been
applied to the database, it calls SKIPPING for each record, with an indication of why it
is skipping the record. You will need to include code in this exit to update the batch
totals for the skipped records exactly as if DataLoader/MX had applied the record.
When DataLoader/MX has skipped the necessary number of records, it begins to
process records in the normal way by calling the appropriate user exits and adds to the
batch totals as the restarted DataLoader/MX applies the record. When DataLoader/MX
terminates successfully, these batch totals are output in the TERMINATING user exit.
When you are restarting from an unknown state, DataLoader/MX cannot alert the user
exit code as to which records have already been processed. If you use batch totals
only to check that the input records match what the source of the data specifies the
totals should be, it does not matter which records were processed earlier. Your user
exit code will receive all the records again, so you would compute the batch totals the
same way as on the initial run, without regard to whether the database change that
was made using a given record succeeded or failed.
But if you intend to use batch totals to summarize the change that the job made to the
database or to summarize how much of the input data failed to update the database,
that simple approach will not give valid totals. You might need to use the approach of
including a LOAD_ID column in the database, as described under Restarting From An
Unknown State on page 7-4.