DataLoader/MX Reference Manual (G06.24+)

Recovery Strategies
DataLoader/MX Reference Manual—525872-002
7-3
Simple Recovery Approaches
FTP to a disk from which you can load, rather than loading it directly from the FTP
process.
If you cannot reread the input data stream that was being processed when the failure
occurred, you may not be able to craft a reliable failure recovery scenario.
Your goal should be to construct a failure recovery plan that enables you to run the
failed loading batch job again. The loading batch job should contain logic, if necessary,
to recognize and recover from the failure. In some cases, you might have to do some
simple operations, such as setting a parameter, purging data, or copying a table to
prepare for this rerun. Avoid creating a situation in which you must create a completely
different set of complex batch jobs to recover from the failure.
Simple Recovery Approaches
In some cases, simple techniques can be used to recover a table from a failed load. If
the table started out empty, remove any data left by the failed load and restart
DataLoader/MX. If the table is small, you can make a copy before the load job runs
and use the copy for reruns of the job if the load fails. Note that both of these scenarios
involve restarting from a known state.
Potential Problems
When you load a table, changes such as INSERTs, UPDATEs, or DELETEs must be
either totally done or totally not done. They must never be partially done. Although you
can create a load process that performs these actions, in the real world, you must
account for problems such as power failures that could interrupt the load process,
leaving a file in an incomplete state. Furthermore, suppose that the load updates a
table with an index but is prevented from updating the index file or files that go with that
file. The HP NonStop Transaction Management Facility (TMF) can handle most of
these types of failures. If you choose not to use TMF protection, you must create your
own method to detect and correct failures.
Double failures are a different problem where the previous contingencies do not apply.
In this case, TMF restores the integrity of the table if the table is TMF protected. If the
table is not TMF protected, use the technique you designed to handle double failures
when you designed your database. Note that double failures must be addressed at the
system level and not at the application level.
The Two Main Approaches
To recover from a failed load of a table:
•
Restarting from an unknown state. In this method, you leave the modifications
made by the failed load in the table but devise a way for the rerun load to
recognize, on a row-by-row basis, whether the action it is preparing to take has
already been done by a previously failed run. See Parallel Considerations on
page 7-8 for a description of self-balancing parallel configurations.