DataLoader/MX Reference Manual (G06.24+)

DataLoader/MX Examples
DataLoader/MX Reference Manual—525872-002
6-7
Multiple Source Parallel Loading
The steps are:
1. The initial DataLoader/MX process for data source 1 reads a block of records from
the input stream, looks at each record’s key, and sends it to the correct
downstream DataLoader/MX process.
2. The DataLoader/MX process for keyrange 1 reads a block of records from its
$RECEIVE file, does any necessary data conversions, and waits for a read from its
import process.
3. The import process for keyrange 1 reads a block of records from its
DataLoader/MX process and then does a load of the partition with keyrange 1.
4. At the same time, the initial DataLoader/MX process for data source 2 reads a
block of records from the input stream, looks at each record’s key, and sends it to
the correct downstream DataLoader/MX process.
5. The DataLoader/MX process for keyrange 2 reads a block of records, does any
necessary data conversions, and waits for a read from its import process.
6. The import process for keyrange 2 reads a block of records from its
DataLoader/MX process and then does a load of the partition with keyrange 2.
There should be as many initial DATALOADER processes as there are input streams
and as many DataLoader/MX process and import reading and writing processing
pairs as there are partitions.
Figure 6-3 is exactly like Figure 6-1, except there is one initial DataLoader/MX process
for each input stream.
Figure 6-3. Multiple Source Parallel Loading
Data
Source N
Data
Source 2
Data
Source 1
Part 1
Part 2
Part N
import
Keyrange 2
import
Keyrange N
import
Keyrange 1
dbl0 - Initial
DataLoader/MX
1
dbl0 - Initial
DataLoader/MX
2
dbl0 - Initial
DataLoader/MX
N
$dbln
DataLoader/MX
Keyrange N
$dbl2
DataLoader/MX
Keyrange 2
$dbl1
DataLoader/MX
Keyrange 1
VST012.vsd