DataLoader/MX Reference Manual (H06.03+, J06.03+)

Recovery Strategies
DataLoader/MX Reference Manual—543544-001
7-2
Design Time Considerations
In this discussion, loading refers to loading new information into the database in the
third way. Some of the new information takes the form of new rows, some takes the
form of updated rows, and some takes the form of deleted rows.
DataLoader/MX can be used to load a SQL/MX or SQL/MP database in many ways. It
can be as simple as a single DataLoader/MX process putting data directly into a single
table. It can be as complex as dozens, or hundreds, of DataLoader/MX processes
arranged in multiple stages, with these processes taking input from multiple sources,
processing it, and using it to load many different tables, some directly and some
through import, LOAD, or APPEND. Clearly, it is impossible to describe a specific
recovery plan for each possible scenario. The following are a number of different
general approaches you can use to develop an overall recovery plan for your loading
scenario.
Design Time Considerations
You should consider how to load your database at the time you design it, along with
other considerations, such as query performance and online transaction processing
performance. Along with those considerations, consider how you will recover if the load
process fails. You must ensure that DataLoader/MX will be able to read the data again,
possibly in the same order as it read it on its first run.
The database design process is a trade-off between a design that can be loaded
quickly and recover from failures during loads and a design that is optimized for other
functions. For example, putting these new data in a new partition or at the end of an
existing partition is good for load performance and recovery but can result in poor
application performance because placing these new data on a single disk eliminates
the possibility of using parallelism in the normal use of the database. No fixed set of
priorities exist, because different types of applications have different considerations.
Tools
The tools used to implement loading are:
•
DataLoader/MX: This tool uses single threaded or parallel processing to load
databases.
•
import: This tool inserts or appends data in a partition of a SQL/MX table. Note
that import does not delete existing data.
Some Basic Rules
In every case where you must perform recovery of the load process, recovery requires
that the DataLoader/MX process read the input data again, and in some cases, it must
be in the exact same order, so you must determine if this is possible. Whatever the
data source, you must make sure that the data, or a copy of it, remains unmodified
until the load is successfully finished. If necessary, you must add steps to your load
scenario to make this occur.