Dataloader/MP Reference Manual

DataLoader/MP Examples
DataLoader/MP Reference Manual—424148-003
6-7
Multiple Source Parallel Loading
format required by SQLCI (SQLUTIL) LOAD and performs any other processing that
might be necessary. You can use any number of routines to do this conversion.
The steps that DataLoader/MP performs are as follows:
1. The initial DataLoader/MP process for data source 1 reads a block of records from
the input stream, looks at each record’s key, and sends it to the correct
downstream DataLoader/MP process.
2. The DataLoader/MP process for keyrange 1 reads a block of records from its
$RECEIVE file, does any necessary data conversions, and waits for a read from its
SQLCI LOAD process.
3. The SQLCI load process for keyrange 1 reads a block of records from its
DataLoader/MP process then does a PARTONLY load of the partition with
keyrange 1.
4. At the same time, the initial DataLoader/MP process for data source 2 reads a
block of records from the input stream, looks at each record’s key, and sends it to
the correct downstream DataLoader/MP process.
5. The DataLoader/MP process for keyrange 2 reads a block of records, does any
necessary data conversions, and waits for a read from its SQLCI LOAD process.
6. The SQLCI load process for keyrange 2 reads a block of records from its
DataLoader/MP process then does a PARTONLY load of the partition with
keyrange 2.
There should be as many initial DATALOLAD processes as there are input streams
and as many DataLoader/MP process and SQLCI reading and writing processing pairs
as there are partitions.
Figure 6-3 is exactly like Figure 6-1, except there is one initial DataLoader/MP process
for each input stream.