Dataloader/MP Reference Manual

DataLoader/MP Components
DataLoader/MP Reference Manual—424148-003
2-3
DataLoader/MP File System
build extremely large data file sets, consisting of many files. Using the INDIRECT
option for input and output, you can go beyond the size limitation for single files.
•
Full files. DataLoader/MP follows a specific strategy for file-full situations. When it
encounters an error 43 or 45 on a file, DataLoader/MP writes a message to its
home terminal and asks for the name of another file to use. This file can be on
another disk. Because there is no time-out, you can take as much time as
necessary to purge files to create room, or run a DSAP analysis or backup and
then purge files. You can also choose not to specify another file, which will make
DataLoader/MP terminate. If you do specify a file and that file also fills up,
DataLoader/MP will prompt again for a subsequent file or files.
•
Note that if you enter the name of a file that does not exist, DataLoader/MP creates
it for you, but creates it as an edit file. If you do not want your output file to be an
edit file type, create the file of the type you want at another TACL prompt, then give
this file’s name to DataLoader/MP.
For information about treating a set of output files as a unit, see the description of
the INDIRECT interpretation on page 4-7.
•
Unstructured files. Unstructured files are considered by the DataLoader/MP file
system to be simply a string of bytes. Any structure inside an unstructured file must
be provided by the application. Unstructured files are important in the loading
process because they can use bulk I/O, the ability to read or write in any multiple of
2 KB up to 56 KB chunks. The use of bulk I/O reduces the cost per byte of I/O,
which is best for I/O intensive applications like loading. However, bulk I/O has the
following restrictions:
°
Secondary partitions are not supported. Bulk I/O treats the primary partition as
the only partition. If an unstructured file with secondary partitions is given to
DataLoader/MP, it will terminate with an error giving the name of the offending
file and the number of secondary partitions it has. This restriction does not limit
the ability of DataLoader/MP to process a large amount of unstructured data
sequentially, because any number of (unpartitioned) unstructured files can be
concatenated using the INDIRECT interpretation.
°
The file must be opened in exclusive mode. (Input files may be opened in non-
exclusive mode.)
•
HP NonStop Open System Services (OSS) files. NM DataLoader/MP can read and
write OSS files. NM DataLoader/MP determines that a file is an OSS file by the
presence of a "/" in the file name. OSS files are treated the same as Guardian
unstructured files.
An OSS file cannot be used in cases where DataLoader/MP requires a Guardian
Edit file. A file used with a BROADCAST, CSV, DUMP, INDIRECT, or KEYRANGE
interpretation must be a Guardian edit file, but the file names listed in the file may
be any mixture of Guardian and OSS file names.
If an OSS file name is being opened for output and the file does not exist,
DataLoader/MP will create the file and write to it as an unstructured file. It will not