Dataloader/MP Reference Manual

Recovery Strategies
DataLoader/MP Reference Manual—424148-003
7-7
Batch Totals
Suppose a process is writing to DataLoader/MP using the process name of the
DataLoader/MP process. When that process terminates, DataLoader/MP will receive
an end of file on its $RECEIVE file whether that process terminated normally or
abnormally. DataLoader/MP then ends normally even though it has not read all of the
input data, which will make a rerun necessary. If you have purged the restart file, you
will not be able to rerun DataLoader/MP.
When you set up your load scenario using restarting from a known state, plan carefully
how you will determine if DataLoader/MP has completed its job and when you can
safely purge the restart file.
Batch Totals
When performing a load job, you commonly accumulate batch totals of such things as
record counts, total dollar amounts, total units sold, and so on. These batch totals can
be used to confirm the integrity of the load operation. How can such batch totals be
accumulated for situations in which the batch is not applied to the database in one
continuous operation?
DataLoader/MP provides a user exit, SKIPPING, for restarting from a known state.
When DataLoader/MP performs a restart and skips records that have already been
applied to the database, it calls SKIPPING for each record with an indication of why it
is skipping the record. You will need to include code in this exit to update the batch
totals for the skipped records exactly as if DataLoader/MP had applied the record.
When DataLoader/MP has skipped the necessary number of records, it begins to
process records in the normal way by calling the appropriate user exits and adds to the
batch totals as the restarted DataLoader/MP applies the record. When DataLoader/MP
terminates successfully, these batch totals are output in the TERMINATING user exit.
When you are restarting from an unknown state, DataLoader/MP cannot alert the user
exit code as to which records have already been processed. If you use batch totals
only to check that the input records match what the source of the data specifies the
totals should be, it makes no difference which records were processed earlier. Your
user exit code will receive all the records again, so you would compute the batch totals
the same way as on the initial run, without regard to whether the database change that
was made using a given record succeeded or failed.
But if you intend to use batch totals to summarize the change that the job made to the
database or to summarize how much of the input data failed to update the database,
that simple approach will not give valid totals. You may need to use the approach of
including a LOAD_ID column in the database, as described under Restarting From An
Unknown State on page 7-4.
Multi-Process Considerations
What is a failure in a situation where an application consists of multiple processes? If
one process fails should the others continue running? DataLoader/MP uses the
strategy with the simplest restart state. If recovery is not simple and obvious, the