DataLoader/MX Reference Manual (G06.24+)

DataLoader/MX Examples
DataLoader/MX Reference Manual—525872-002
6-8
Multiple Source Parallel Loading
The following commands and their descriptions describe one way to implement the
scenario in Figure 6-3.
Command lines 1 through 3 execute a version of the DataLoader/MX process called
dbl0 (created with the build.sh script). In dbl0, the default user exit BUILDKEY is
replaced by one that, given an input record, returns its key. This strategy is necessary
because the DataLoader/MX process uses this key to determine which downstream
DataLoader/MX process receives the record.
1$ run dbl0 -E=err0a -I=infile1 "-O=kr(keyrange)" -S=100000 &
2$ run dbl0 -E=err0b -I=infile2 "-O=kr(keyrange)" -S=100000 &
3$ run dbl0 -E=err0c -I=infile3 "-O=kr(keyrange)" -S=100000 &
The preceding commands use these parameters:
•
-E specifies that any execution time errors are recorded in the appropriate error
listing file.
•
-I directs the DataLoader/MX process to read the input from the appropriate input
file.
•
-O specifies that the output is written to the file kr (which is interpreted as a
keyrange file).
•
-S directs the DataLoader/MX process to generate a status message every
100,000 records.
The file kr is a text file containing:
"" $dbl1
"D" $dbl2
"G" $dbl3
...
"Y" $dbln
Because the DataLoader/MX process is instructed to interpret this output file as a
keyrange file, the output records are not written to kr itself. Instead, the contents of kr,
together with the key of each record, are used to determine where the records are
written. Records with keys greater than or equal to "" (NULL) but less than "D" are
written to $dbl1. Records with keys greater than or equal to "D" but less than "G" are
written to $dbl2, and so on.
Command lines 4 through 7 start the downstream DataLoader/MX processes, which
are customized versions of DataLoader/MX with user exits that convert the raw input
records.
•
-E gives each of these downstream DataLoader/MX processes different files (err1,
err2, ...) in which to record their execution time errors.
Note. gpri and cpu parameters have been omitted from the example. The initial
DataLoader/MX process should run at high priority, and the downstream DataLoader/MX
process and import pairs should run in the CPU where the primary disk process for the
corresponding partition is running.