FastSort Manual Abstract This manual describes FastSort, the HP sort-merge utility for HP NonStop™ systems. The FastSort Manual is intended for users who sort interactively, programmatically, and from HP NonStop SQL/MP. Product Version FastSort D32 Supported Release Version Updates (RVUs) This publication supports G06.21 and all subsequent G-series RVUs until otherwise indicated by its replacement publication. To use increased Enscribe limits, the minimum RVUs are H06.28 and J06.17 with specific SPRs.
Document History Part Number Product Version Published 060035 FastSort C30 July 1992 118812 FastSort D40 December 1995 124077 FastSort D32 February 1996 429834-001 FastSort D32 July 2001 429834-002 FastSort D32 September 2003 429834-003 FastSort D32 April 2014
Legal Notices Copyright 1992, 2014 Hewlett-Packard Development Company, L.P. Confidential computer software. Valid license from HP required for possession, use or copying. Consistent with FAR 12.211 and 12.212, Commercial Computer Software, Computer Software Documentation, and Technical Data for Commercial Items are licensed to the U.S. Government under vendor's standard commercial license. The information contained herein is subject to change without notice.
FastSort Manual Glossary Index Examples Figures Tables Legal Notices What’s New in This Manual vii Manual Information vii New and Changed Information vii About This Manual ix SPR Requirements for Increased Enscribe Limits for the H06.28/J06.17 Release ix Audience x Related Manuals x Notation Conventions xii 1.
3. Using FastSort Commands Contents Specifying Input Files in the FROM Command 2-4 Specifying Input Records at the Input Prompt 2-5 Sorting on Key Fields 2-6 Running a Record Sort 2-7 Running a Key Sort 2-8 Running a Permutation Sort 2-9 Running a Key and Permutation Sort 2-10 Controlling Extended Memory 2-11 Understanding Statistics 2-13 Understanding Error Messages 2-14 Understanding Completion Codes 2-15 3.
5. Using FastSort System Procedures Contents Sending Records to Be Merged 4-5 Returning Output Records to a Process 4-6 Sending and Receiving Records 4-7 Estimating the Size of an Output File 4-8 Sorting From C Programs 4-9 Sorting From COBOL85 Programs 4-13 Sorting From TAL Programs 4-15 5.
7. Using SORT and SUBSORT DEFINEs Contents Understanding Statistics From Parallel Sorting 6-14 Identifying the Causes of Errors 6-14 Parallel Sorting From C Programs 6-15 Parallel Sorting From COBOL85 Programs 6-19 Parallel Sorting From TAL Programs 6-22 7.
A. FastSort Syntax Summary Contents Using Scratch Files 9-1 Selecting a Scratch Volume for Parallel Sorts Using a Partitioned Scratch File 9-8 Using Swap Files 9-10 Using VLM 9-10 Calculating Data Stack Space 9-12 Managing Sort Failures 9-13 Automating FastSort Tasks 9-15 Automating DEFINEs 9-16 9-7 A. FastSort Syntax Summary Interactive Commands A-1 FastSort Procedures A-3 B.
Figures Contents Example 6-1. Example 6-2. Example 6-3. C Example of a Parallel Sort Run 6-15 COBOL85 Example of a Parallel Sort Run 6-20 TAL Example of a Parallel Sort Run 6-23 Figures Figure i. Figure 1-1. Figure 1-2. Figure 4-1. Figure 4-2. Figure 4-3. Figure 4-4. Figure 5-1. Figure 5-2. Figure 6-1. Figure 8-1. Figure 9-1. Figure 9-2.
What’s New in This Manual Manual Information FastSort Manual Abstract This manual describes FastSort, the HP sort-merge utility for HP NonStop™ systems. The FastSort Manual is intended for users who sort interactively, programmatically, and from HP NonStop SQL/MP. Product Version FastSort D32 Supported Release Version Updates (RVUs) This publication supports G06.21 and all subsequent G-series RVUs until otherwise indicated by its replacement publication.
Changes to the 429834-003 manual: What’s New in This Manual Added a section Using 32 KB Buffers on page 5-5. Updated block size in the section Improving Performance With Record Blocking and Nowait I/O on page 5-6. Updated Using Supported File Types on page C-1. Updated Key-Sequenced Files on page C-3. Previous publication was updated to reflect new product names: Since product names are changing over time, this publication might contain both HP and Compaq product names.
About This Manual This manual is a combination user’s guide and reference manual that describes FastSort, the sort-merge utility for NonStop systems. SPR Requirements for Increased Enscribe Limits for the H06.28/J06.17 Release As of H06.28 and J06.17 RVUs, format 2 legacy key-sequenced 2 (LKS2) files with increased limits, format 2 standard queue files with increased limits, and enhanced key-sequenced (EKS) files with increased limits are introduced.
Audience About This Manual Audience This manual is intended for all FastSort users, including: Users who issue FastSort interactive commands from a terminal or through a command file Programmers who call FastSort system procedures from an application program (including COBOL85 programmers who use the SORT and MERGE statements) NonStop SQL/MP users, if SQL/MP is installed on your system and you initiate queries that sort entries or load data A reader of this manual should be familiar with the NonS
Related Manuals About This Manual Figure i.
Notation Conventions About This Manual Notation Conventions Hypertext Links Blue underline is used to indicate a hypertext link within text. By clicking a passage of text with a blue underline, you are taken to the location described. For example: This requirement is described under Backup DAM Volumes and Physical Disk Drives on page 3-2. General Syntax Notation This list summarizes the notation conventions for syntax presentation in this manual. UPPERCASE LETTERS.
General Syntax Notation About This Manual { } Braces. A group of items enclosed in braces is a list from which you are required to choose one item. The items in the list can be arranged either vertically, with aligned braces on each side of the list, or horizontally, enclosed in a pair of braces and separated by vertical lines. For example: LISTOPENS PROCESS { $appl-mgr-name } { $process-name } ALLOWSU { ON | OFF } | Vertical Line.
Notation for Messages About This Manual a blank line. This spacing distinguishes items in a continuation line from items in a vertical list of selections. For example: ALTER [ / OUT file-spec / ] LINE [ , attribute-spec ]… !i and !o. In procedure calls, the !i notation follows an input parameter (one that passes data to the called procedure); the !o notation follows an output parameter (one that returns data to the calling program).
Notation for Messages About This Manual lowercase italic letters. Lowercase italic letters indicate variable items whose values are displayed or returned. For example: p-register process-name [ ] Brackets. Brackets enclose items that are sometimes, but not always, displayed. For example: Event number = number [ Subject = first-subject-value ] A group of items enclosed in brackets is a list of all possible items that can be displayed, of which one or none might actually be displayed.
Change Bar Notation About This Manual Change Bar Notation Change bars are used to indicate substantive differences between this manual and its preceding version. Change bars are vertical rules placed in the right margin of changed portions of text, figures, tables, examples, and so on. Change bars highlight new or revised information. For example: The message types specified in the REPORT clause are different in the COBOL85 environment and the Common Run-Time Environment (CRE).
1 Introduction to FastSort FastSort is the sort-merge tool for HP NonStop systems. FastSort can sort or merge records in one of two modes: A serial operation uses one SORTPROG process in one processor with up to 32 individual scratch files or a single partitioned scratch file. A parallel operation distributes the sort or merge workload across multiple SORTPROG processes, processors, and scratch files.
FastSort Features Introduction to FastSort Figure 1-1. FastSort Operations Input From Files or Processes Output to a File or Process Sort 1. Unsorted Records 2. Sorted Records Sorted Records • • • Sorted Records Merge Sorted and Merged Records Sort and Merge Sorted and Merged Records Sort and Merge Sorted and Merged Records 3. Unsorted Records • • • Unsorted Records Sorted Records 4. • • • Sorted Records Unsorted Records • • • Unsorted Records VST101.
FastSort Components Introduction to FastSort Sequence options Can use an alternate collating sequence for alphanumeric characters Can remove records with duplicate key values Input and output file options Accepts up to 32 input Guardian files (except blocked tape files, SQL objects, or processes) Can merge up to 32 input streams from an application process Supports partitioned input and output files, including distributed Enscribe databases (but not NonStop SQL/MP objects)
FastSort Components Introduction to FastSort SORTPROG process (the actual sort-merge process) Record generator (RECGEN) process, if SQL/MP is installed on your system When you want to sort or merge records interactively or through a command file (IN file), you use interactive FastSort. When you want to sort or merge records in an application, you invoke programmatic FastSort through system procedure calls.
Interactive FastSort Introduction to FastSort Figure 1-2. FastSort Components FastSort Programmatic Interface FastSort Procedure Calls $SYSTEM. SYSnn. OSIMAGE FastSort System Library Procedures User Process SORTBUILDPARM SORTMERGESTART SORTMERGESEND SORTMERGERECEIVE SORTERROR SORTERRORDETAIL SORTERRORSUM SORTMERGESTATISTICS SORTMERGEFINISH $SYSTEM. SYSnn. SORTPROG SORTPROG Process FastSort Interactive Interface $SYSTEM. SYSnn.
Interactive FastSort Introduction to FastSort Table 1-1. FastSort Interactive Commands (page 1 of 2) Command Description Record Sequence Specification ASCENDING Describes the location and attributes of one or more key fields that determine an ascending sequence for output records. COLLATE Specifies a file that contains an alternate collating sequence for comparing key fields. COLLATEOUT Stores an alternate collating sequence table in an unstructured file.
Programmatic FastSort Introduction to FastSort Table 1-1. FastSort Interactive Commands (page 2 of 2) Command Description Process Control EXIT Ends the interactive FastSort session (same as Ctrl-Y). RUN Starts a sort or merge run and optionally specifies the SORTPROG process start parameters, the allocation of required disk space, and whether duplicate records should be removed.
SORTPROG Process Introduction to FastSort SORTPROG Process The SORTPROG process performs all sort or merge operations. It runs separately from an application process or the interactive SORT process. To configure and start a SORTPROG process, you either: Issue FastSort interactive commands Call FastSort system procedures The SORTPROG process does not run as a process pair. If a processor failure occurs when the SORTPROG process is running, you must restart the sort or merge run from the beginning.
Scratch Files Introduction to FastSort Scratch Files FastSort sorts files smaller than 100 kilobytes in memory. For larger input files, FastSort uses up to 32 scratch files to temporarily store groups of records called runs. You can create a scratch file before you run FastSort, or you can have SORTPROG create one for you. If you manually create a scratch file, SORTPROG leaves the file intact after the sort or merge run.
Class SORT and SUBSORT DEFINEs Introduction to FastSort Class SORT and SUBSORT DEFINEs You can use class SORT or SUBSORT DEFINEs to configure a sort or subsort. You specify the DEFINE(s) before running a SORTPROG process, and the information is applied to the sort process when it is run. You can use class SORT or SUBSORT DEFINEs to specify information such as the disk volume for the scratch file, the processors to use, and so on.
Products That Use FastSort Introduction to FastSort When you use the SQLCI CREATE INDEX statement, SQL uses FastSort to load data into the target table. When you use the SQLCI LOAD utility to load data into a keysequenced table and you do not specify the SORTED option, SQL uses FastSort to sort the data.
Products That Use FastSort Introduction to FastSort FastSort Manual—429834-003 1-12
2 Sorting Interactively You can use SORT, the FastSort interactive process, to sort or merge records without writing an application program. FastSort accepts interactive commands from: A TACL process A command (IN) file Running FastSort To start an interactive SORT process, enter SORT at a TACL prompt: 10> SORT This command executes an implicit TACL RUN command that starts the SORT process.
Entering Commands and Data in a Command File Sorting Interactively command-file is an EDIT file (file code 101) that contains FastSort interactive commands. A command file can also contain the input records for a sort or merge run. list-file is a disk file, I/O device, SPOOL DEFINE, or a process that receives the output from the sort or merge run. The output file also includes statistics and any error or warning messages.
Running With Input From a Command File Sorting Interactively Running With Input From a Command File In the following example, a command file named COMFILE contains FastSort commands and input records. To execute COMFILE and send the output to the list file named LISTFILE, you would enter: 10> SORT / IN COMFILE, OUT LISTFILE, NOWAIT / The SORT process reads the commands and data from COMFILE and initiates a SORTPROG process to sort the data.
Specifying Input Records Sorting Interactively FastSort - T9620D30 - (31OCT94) COPYRIGHT TANDEM COMPUTERS INCORPORATED 1991 - 1994 1 ! Send sorted records to the file named TOFILE. 2 ! Sort in descending order from column 1 to 10.
Specifying Input Records at the Input Prompt Sorting Interactively FastSort sorts and merges the records in the input files, then displays the sorted and merged records and the statistics for the sort run on your terminal, as shown below: apple banana grape grapefruit lemon orange watermelon 7 RECORDS 00:04 ELAPSED TIME 00:00 I/O WAIT TIME 16 COMPARES 0 SCRATCH DISK 0 SCRATCH SEEKS Errors detected: 0 Warnings detected: 0 132 166 0 0 MAX RECORD SIZE BUFFER PAGES INITIAL RUNS MERGE ORDER Specifying Input
Sorting on Key Fields Sorting Interactively Sorting on Key Fields FastSort returns four types of output records: Sort Operation Output Records Record Sort The entire file of input records reordered according to the values of one or more key fields Key Sort The values of the concatenated key fields in sorted order Permutation Sort The input record sequence numbers in the order the records would be in if they were sorted according to the specified key fields Key and Permutation Sort A sequence numb
Running a Record Sort Sorting Interactively 11 12 Hoptree PTELEA Catclaw Acacia ACACIA TRIFOLIATA GREGGII shrub shrub 27 12 Running a Record Sort A record sort reorders input records according to the values of one or more key fields.
Running a Key Sort Sorting Interactively 03 12 Busy Lizzy Daffodil IMPATIENS NARCISSUS flower flower 30 60 Running a Key Sort The output records from a key sort consist of the values of the concatenated key fields in sorted order.
Running a Permutation Sort Sorting Interactively Running a Permutation Sort The output records from a permutation sort consist of the input record sequence numbers, in the order the records would be in if they were sorted according to the specified key fields.
Running a Key and Permutation Sort Sorting Interactively Running a Key and Permutation Sort Each output record from a combined key and permutation sort consists of a sequence number followed by the key-field values.
Controlling Extended Memory Sorting Interactively Controlling Extended Memory By default, FastSort tries to use enough extended memory to make at most one merge pass, depending on the size of the output file.
Controlling Extended Memory Sorting Interactively Table 2-1. Extended Memory Used by FastSort File Size, in Bytes Parameter No Merge Passes MINSPACE 100 KB 512 KB AUTOMATIC 100 KB 512 KB MINTIME One Merge Pass Extended Memory, in Bytes > 100 KB bf MIN ------------- 1.3 0.5MB 2 MAX f 1.3 512KB > 200 KB bf MIN ------------- 1.3 0.7MB 2 200 KB SEGMENT n n n MIN MAX b f n = 256 > 256 = = = = = 100 KB 512 KB MIN n 2048 0.
Understanding Statistics Sorting Interactively Understanding Statistics After a sort or merge run, FastSort returns statistics to the list file. The list file is either the OUT file specified in the implicit TACL RUN command for the SORT process or your home terminal if you do not specify an OUT file.
Understanding Error Messages Sorting Interactively For a parallel sort run, FastSort returns some statistics that apply only to the distributor-collector process.
Understanding Completion Codes Sorting Interactively Understanding Completion Codes In addition to error messages, FastSort might return a completion code after a sort or merge run. Completion codes are summarized following: Code Explanation 1 Syntax errors occurred but are treated as warnings only. FastSort continues to accept input and returns this message: Syntax errors/warning detected 2 FastSort execution errors occurred. These errors include no input file.
Understanding Completion Codes Sorting Interactively FastSort Manual—429834-003 2-16
3 Using FastSort Commands FastSort interactive commands are summarized below. This section describes these commands in alphabetic order. Command Description ASCENDING Describes the location and attributes of one or more key fields that determine an ascending sequence for output records. CLEAR Deletes current command parameters for all commands or a specific command. COLLATE Specifies a file that contains an alternate collating sequence for comparing key fields.
ASCENDING Command Using FastSort Commands For more information about using these commands, see Section 2, Sorting Interactively. Appendix A, FastSort Syntax Summary contains a quick reference to the command syntax. ASCENDING Command Use the ASCENDING command to sort or merge records so that the values of each key field specified in the command are in smallest-to-largest order. ASCENDING and DESCENDING commands can apply to the same sort run.
ASCENDING Command Using FastSort Commands STRING UPPER INTEGER REAL UNSIGNED SIGNED LEADING EMBEDDED or SLE SIGNED LEADING SEPARATE or SLS SIGNED TRAILING EMBEDDED or STE SIGNED TRAILING SEPARATE or STS STRING specifies that the key field contains unsigned alphanumeric data. STRING is the default data type. UPPER specifies that the key field contains unsigned alphanumeric data. FastSort treats all lowercase ASCII characters as uppercase characters.
ASCENDING Command Using FastSort Commands SIGNED TRAILING SEPARATE | STS specifies that the key field contains signed ASCII numeric data with the sign character (+ or –) stored in the last byte of the field. The key length cannot be greater than 32 bytes. Key Fields The order in which you enter ASCENDING and DESCENDING commands affects sort output. The first command entered has the highest priority.
CLEAR Command Using FastSort Commands CLEAR Command Use the CLEAR command to delete command parameters entered for the current sort or merge run or saved from a previous run. You can use CLEAR to delete parameters for all commands or for individual commands. CLEAR { { { { { { { { { { ALL ASC[ENDING] COLLATE CPUS DESC[ENDING] FROM [ filename ] KEYS NOTCPUS SUBSORT TO } } } } } } } } } } ALL deletes all current command parameters.
COLLATE Command Using FastSort Commands SUBSORT deletes current parameters of all SUBSORT commands. TO deletes all parameters for the current output file. Examples CLEAR DESC CLEAR FROM FILETEN CLEAR TO COLLATE Command Use the COLLATE command to specify an alternate collating sequence during a sort or merge run. This enables you to define the comparison of alphanumeric key fields or string-type data.
COLLATE Command Using FastSort Commands ranges, and comments preceded by exclamation points. The following rules apply to specifying ranges in the file: All the ranges together must include character assignments for exactly 256 byte positions. You can assign more than one character to the same position. You can also assign the same character to more than one position. The number of lines in the file is irrelevant, but you cannot put a range of characters on more than one line.
COLLATE Command Using FastSort Commands To have SORTPROG treat several characters as equal in comparisons, you can assign them all to the same character position, like this: ";", ALSO ":", ALSO 128, ALSO 129, If ALSO assigns a range of characters, the number of characters in that range must equal the number of characters in the preceding range: "A" THRU "Z", ALSO "a" THRU "z", Ranges beginning with ALSO do not assign characters to additional byte positions.
COLLATEOUT Command Using FastSort Commands The other file, named ALTSEQ, contains these lines: 0 THRU 64, "A" THRU "Z", ALSO "a" THRU "z", 91 THRU 96, 123 THRU 255 A terminal session shows the results of sorting five records in two situations: Without the COLLATE command, using the ASCII collating sequence With the COLLATE command, using the collating sequence in ALTSEQ
CPUS Command Using FastSort Commands filename is the name of the unstructured file to which COLLATEOUT writes the 256-byte alternate collating sequence table. If filename already exists, FastSort purges it and creates a new file with that name. Example COLLATEOUT ALTSEQ CPUS Command Use the CPUS command to specify a group of processors (CPUs) that FastSort can use to run subsort processes for parallel sorting. CPUS [ ALL ] [ cpu-list ] ALL specifies that FastSort can use any processor.
DESCENDING Command Using FastSort Commands DESCENDING Command Use the DESCENDING command to sort or merge records so that the values of each key field specified in the command are in largest-to-smallest order. ASCENDING and DESCENDING commands can apply to the same sort run, which causes the values of some key fields to be in smallest-to-largest order.
DESCENDING Command Using FastSort Commands SIGNED LEADING EMBEDDED or SLE SIGNED LEADING SEPARATE or SLS SIGNED TRAILING EMBEDDED or STE SIGNED TRAILING SEPARATE or STS STRING specifies that the key field contains unsigned alphanumeric data. STRING is the default data type. UPPER specifies that the key field contains unsigned alphanumeric data. FastSort treats all lowercase ASCII characters as uppercase characters. INTEGER means the key field contains two’s complement signed binary data.
EXIT Command Using FastSort Commands Key Fields The order in which you enter DESCENDING and ASCENDING commands determines their relative significance. The first command has the highest priority. SORTPROG starts sorting the records according to the key fields specified in the first DESCENDING or ASCENDING command, then uses the key fields specified in the second DESCENDING or ASCENDING command, and so on.
FC Command Using FastSort Commands FC Command Use the FC (Fix) command to display the last FastSort command and then to repeat or edit the command. FC When you enter the FC command, FastSort displays the last command you typed followed by the FC prompt, a period (.), on the next line.
FROM Command Using FastSort Commands If you omit the in-file parameter, you can enter only one FROM command for a sort run, which means the input file is the command stream. If you use a command file (IN file) for input, put the input records after the RUN command, one record on each line. If you enter records at the terminal, type the input records after you enter the RUN command, one record at each ? prompt. EXCL[USION] mode specifies the exclusion mode with which FastSort opens a file.
FROM Command Using FastSort Commands For an unstructured disk file, SORTPROG calculates an approximate number of records in the file. The approximate number of records for an EDIT file is the end-of-file length multiplied by 2 and divided by the record length. The approximate number of records for other unstructured files is the end-of-file location divided by the record length. The default record length for all unstructured disk files is 132 bytes.
FROM Command Using FastSort Commands Exclusion Mode and File Access To enable another process to read the file at the same time as FastSort, specify PROTECTED in the FROM command and have the other process open the file in SHARED mode. Record Entry When you omit the FROM command or use a FROM command without an in-file parameter, you can supply the records from a terminal or from the command file (IN file) that starts the FastSort process. Caution.
HELP Command Using FastSort Commands HELP Command Use the HELP command to get help for FastSort commands. When you request information about a specific command, HELP displays the syntax of that command. If you do not specify a command, HELP displays a list of FastSort commands and a description of each command.
RUN Command Using FastSort Commands The NOTCPUS command is also useful to exclude processors you already specified in a CPUS command. This example excludes processors from a list specified in a previous run:
RUN Command Using FastSort Commands volume. For more information about scratch files and scratch volumes, see Managing Sort Workspace on page 9-1. AUTOMATIC directs FastSort to limit elapsed time by using at most 50 percent (90 percent in parallel sorting) of the physical memory not locked down by the operating system. For files equal to or smaller than 100 KB, FastSort uses 256 pages for an extended memory segment and makes no merge pass.
RUN Command Using FastSort Commands MINSPACE limits the size of the extended memory segment to 256 pages (512 KB). FastSort makes no merge pass or only one merge pass if the file size is equal to or less than 100 KB. The file size is the maximum number of records in all input files times the maximum record length for the output file (see the FROM Command on page 3-14). For more information, see Controlling Extended Memory on page 2-11 for details about how file size affects the number of merge passes.
RUN Command Using FastSort Commands FastSort determines which records have duplicate keys according to that collating sequence. DEFINE define-name is an optional 12-word array that specifies the name of a SORT DEFINE to use for the sort or merge run. For more information, see Section 7, Using SORT and SUBSORT DEFINEs. SEGMENT size specifies the size in pages of an extended memory segment for FastSort to use.
SAVE Command Using FastSort Commands bytes of stack space. If you specify SCRATCHON, you cannot specify NOSCRATCHON. Enclose NOSCRATCHON volume names in parentheses and separate the names with commas. FastSort recognizes the wild-card characters * and ? for NOSCRATCHON volume names. See the description of SCRATCHON under Setting DEFINE Attributes on page 7-2 for examples of how to use these characters. SCRATCHON (scratch-vol [, scratch-vol]...
SAVE Command Using FastSort Commands ALL saves all current command parameters. ASC[ENDING] saves all current key-field specifications defined by ASCENDING commands. COLLATE saves the alternate collating sequence table. CPUS saves the CPUS command currently in effect. DESC[ENDING] saves all current key-field specifications defined by DESCENDING commands. FROM [ filename ] saves the current parameters for the input file named filename.
SHOW Command Using FastSort Commands SHOW Command Use the SHOW command to display command parameters currently in effect, whether you entered them for the next sort or merge run or saved them from a previous run. SHOW does not display information for the COLLATE or COLLATEOUT command. SHOW { { { { { { { { { ALL } ASC[ENDING] } CPUS } DESC[ENDING] } FROM [ filename ] } KEYS } NOTCPUS } SUBSORT } TO } ALL displays all current command parameters.
SUBSORT Command Using FastSort Commands TO displays all parameters for the current output file. Examples SHOW KEYS SHOW FROM SUBSORT Command Use the SUBSORT command to set up the parameters for a SORTPROG subsort process for a parallel sort or merge run. A subsort process runs under a SORTPROG distributor-collector process set up by a RUN command. Note. Although you can specify up to 16 subsort processes, HP recommends that you specify no more than 8.
SUBSORT Command Using FastSort Commands CPU processor specifies the processor number for the subsort process. Because each subsort process is a separate SORTPROG process, you can run each process in a different processor. The default is the same processor in which the primary disk process for the scratch volume runs. MEM memory exists only for compatibility with earlier sort-merge code. MEM specifies the number of memory pages allocated for the SORTPROG process. The size is always 64 pages.
TO Command Using FastSort Commands SUBSORT $RAT.SORT.
TO Command Using FastSort Commands PROTECTED specifies that only FastSort has read and write access to the output file. EXCLUSIVE specifies that only FastSort has write access to the output file. Other processes can have read access to the file.
TO Command Using FastSort Commands NOPURGE directs FastSort not to purge the output file if the file seems too small to contain all the output records. This parameter ensures that FastSort preserves the original partitioning and extents of the file. FastSort still purges the data from an existing output file, even though it does not purge the file. When you specify NOPURGE, FastSort changes the record length to the default value of 132 bytes.
TO Command Using FastSort Commands 4. If none of the above conditions exists, SORTPROG creates an entry-sequenced file. You can use a process as an output file. If out-file is a blocked tape file, SORTPROG writes only one record for each block. You can use the File Utility Program (FUP) to block the records and load the tape file. For information about FUP, see the FUP Reference Manual. SORTPROG does not write output records to EDIT files.
TO Command Using FastSort Commands Output Options If you use both PERMUTATION and KEYS, the output for each record is a 32-bit (4byte) record number followed by the concatenated key-field values as shown in below: Byte 0 1 2 3 4 5 ... seq no | key 1 ...| key 2 ... The first 11 characters are sequence numbers, and the remaining characters are the defined keys. These options increase efficiency when you need only part of the data in the records.
4 Sorting Programmatically The FastSort programmatic interface consists of the FastSort system procedures. You can use the FastSort system procedures to sort and merge records from an application program. You can call FastSort system procedures from an application written in any language that can call TAL procedures. An application calls FastSort system procedures to start, control, and end a SORTPROG process.
Starting a Sort or Merge Run Sorting Programmatically Table 4-1. FastSort System Procedures (page 2 of 2) Procedure Name Description SORTERRORSUM Provides SORTERROR and SORTERRORDETAIL information and identifies the cause of the most recent error. SORTMERGESTATISTICS Reports information about a sort or merge run and ends the run. SORTMERGEFINISH Ends the sort or merge run and stops the SORTPROG process.
Allocating Scratch Space Sorting Programmatically SORTBUILDPARM puts the parameters you specify in a sort control block, which is a global array used for storing the information. SORTMERGESTART uses the sort control block to pass the parameters to the SORTPROG process. Allocating Scratch Space You can have SORTPROG create initial and overflow scratch files for you. To do this, specify either no scratch file in SORTMERGESTART or a disk file that does not exist.
Sending Input Records From a Process Sorting Programmatically Figure 4-1. Sorting and Merging With Input and Output Files Application Process SORTPROG Process • • • • • • • • • • SORTMERGESTART • • • • • • • • • • SORTMERGESTATISTICS • SORTMERGEFINISH • • • • FastSort Parameters File 1 File 2 Input Unsorted Records Input Sorted Records Completion Statistics SORTPROG Output Sorted and Merged Records VST401.
Sending Records to Be Sorted Sorting Programmatically Sending Records to Be Sorted To use SORTMERGESEND for a sort run, you must specify the following values in the call to SORTMERGESTART: 1 for the num-sort-files parameter Blanks for the input-file-name parameter Sending Records to Be Merged To use SORTMERGESEND for a merge run, you must specify the following in the call to SORTMERGESTART: A number from 2 to 32 for the num-merge-files parameter From 2 to 32 names of all blanks for the input-
Returning Output Records to a Process Sorting Programmatically Figure 4-2. Sending Input Records From an Application Process Application Process • • • • SORTMERGESTART • SORTPROG Process • • • • FastSort Parameters Input Record • SORTMERGESEND • Input Record • SORTMERGESEND • Last Input Record • SORTMERGESEND • Record Length = -1 • SORTMERGESEND • Completion • Statistics • SORTMERGESTATISTICS • SORTMERGEFINISH • • • SORTPROG Output Sorted Records VST402.
Sending and Receiving Records Sorting Programmatically Figure 4-3. Returning Sorted Records to an Application Process Application Process • • • • SORTMERGESTART • • • • • • • • • SORTMERGERECEIVE • • SORTMERGERECEIVE • • SORTMERGERECEIVE • • • • SORTMERGERECEIVE • SORTMERGERECEIVE • SORTMERGESTATISTICS • SORTMERGEFINISH SORTPROG Process FastSort Parameters • • • • File 1 File 2 Input Input Unsorted Records Sorted Records SORTPROG Sorted Records Last Record Record Length = -1 VST403.
Estimating the Size of an Output File Sorting Programmatically Figure 4-4.
Sorting From C Programs Sorting Programmatically You can also set flags.<5> to 1 to direct FastSort to not purge an existing output file that seems too small. Sorting From C Programs Example 4-1 shows a C program that calls FastSort procedures to perform a serial sort run. Example 4-1.
Sorting From C Programs Sorting Programmatically Example 4-1. C Example of a Serial Sort Run (page 2 of 5) /*-------------------------------------------------------------*/ /* FastSort error and statistics variables.
Sorting From C Programs Sorting Programmatically Example 4-1. C Example of a Serial Sort Run (page 3 of 5) //*-------------------------------------------------------------*/ /* Call SORTBUILDPARM to initialialize SORTPROG control block. */ /* Request blocked, double-buffered interface.
Sorting From C Programs Sorting Programmatically Example 4-1. C Example of a Serial Sort Run (page 4 of 5) if (error) /* check for SORTMERGESEND error */ { errlen = SORTERRORSUM (&ctlblk[0], &error_buf[0], &error_code[0], &error_source[0]); error_handler ; return EXIT_FAILURE; } /*------------------------------------------------------------*/ /* Call SORTMERGERECEIVE to receive records from SORTPROG.
Sorting From COBOL85 Programs Sorting Programmatically Example 4-1. C Example of a Serial Sort Run (page 5 of 5) //*-------------------------------------------------------------*/ /* Call SORTMERGEFINISH to stop SORTPROG after the process */ /* successfully completes the current sort and merge run(s).
Sorting From COBOL85 Programs Sorting Programmatically Example 4-2. COBOL85 Example of a Serial Sort Run (page 1 of 2) *--------------------------------------------------------* FastSort Serial Sort Run Program *--------------------------------------------------------* This program sorts an input file specified by the TACL * DEFINE =INFILE and writes the sorted records to an output * file specified by the TACL DEFINE =OUTFILE. The program * uses a temporary scratch file on the user's default volume.
Sorting From TAL Programs Sorting Programmatically Example 4-2. COBOL85 Example of a Serial Sort Run (page 2 of 2) *--------------------------------------------------------* Input: Read input records and release to SORTPROG. *--------------------------------------------------------SORTIN-PROCEDURE SECTION. DISPLAY "Reading input records...". READ-INPUT. READ INPUT-FILE NEXT RECORD AT END GO TO SORTIN-EXIT. RELEASE SORT-RECORD FROM IN-RECORD. GO TO READ-INPUT. SORTIN-EXIT. EXIT.
Sorting From TAL Programs Sorting Programmatically Example 4-3. TAL Example of a Serial Sort Run (page 1 of 3) ?SYMBOLS, NOCODE, INSPECT, MAP, LMAP !-----------------------------------------------------------! ! FastSort Serial Sort Run ! !-----------------------------------------------------------! ! This program sends input records to a SORTPROG process ! ! using SORTMERGESEND and then receives the sorted output ! ! records using SORTMERGERECEIVE.
Sorting From TAL Programs Sorting Programmatically Example 4-3.
Sorting From TAL Programs Sorting Programmatically Example 4-3. TAL Example of a Serial Sort Run (page 3 of 3) length := 70; ! Set length of buffer. FOR index := 1 TO max^count DO BEGIN inbuf := - index; ! Set value to send to SORTPROG. error := SORTMERGESEND (ctlblk,, length,,,, rec^addr); IF error THEN ! Check for SORTMERGESEND error. BEGIN length := SORTERRORSUM (ctlblk, error^buf, error^code, error^source); ! Process the SORTMERGESEND error.
5 Using FastSort System Procedures This section describes the FastSort system library procedures. FastSort procedures communicate between a user-written application process and a SORTPROG process. The SORTPROG process runs independently of an application process and by default resides in the $SYSTEM.SYSnn.SORTPROG program file. For information about calling these procedures for serial sorting, see Section 4, Sorting Programmatically.
SORTBUILDPARM Procedure Using FastSort System Procedures Operation Description Additional Space Simple Less than 5 keys, no subsorts, 1 input file 2 pages Medium Greater than 5 keys, either subsorts or multiple input files 3 pages Complex Greater than 5 keys, subsorts, multiple input files 4 pages To allocate this additional space in an application, use one of the following methods: For a TAL application, use the DATAPAGES compiler directive during compilation.
SORTBUILDPARM Procedure Using FastSort System Procedures The call to SORTBUILDPARM must precede the call to SORTMERGESTART. SORTBUILDPARM stores your parameters in the sort control block, and SORTMERGESTART passes the parameters to the SORTPROG process.
SORTBUILDPARM Procedure Using FastSort System Procedures is the address of a buffer that SORTPROG can use to block input records from SORTMERGESEND or deblock output records for SORTMERGERECEIVE. This buffer can be in the user data space segment (for buffer length up to 8 KB) or in an extended data segment. If the buffer is in an extended data segment, the segment must be in use at the time of the call. You should not rely on the information in buffer, because this information can change without warning.
SORTBUILDPARM Procedure Using FastSort System Procedures use the restart flags, it must call SORTMERGESTATISTICS or an error must end the SORTPROG process. define-name input INT:ref:12 is an optional 12-word array that specifies the SORT DEFINE name to be used. For more information, see Section 7, Using SORT and SUBSORT DEFINEs. reserved1 and reserved2 are reserved for future parameters. If you specify a value for reserved1 or reserved2, FastSort returns an error.
SORTBUILDPARM Procedure Using FastSort System Procedures Guidelines Follow these guidelines when you call the SORTBUILDPARM procedure. Specifying a Group of Processors for Subsort Processes When you configure a parallel sort run, you can have the distributor-collector SORTPROG process select processors for subsort processes. SORTPROG considers processors you specify in the cpu-mask parameter. You can use the not-cpu-mask parameter, which overrides cpu-mask, to exclude one or more processors.
SORTBUILDPARM Procedure Using FastSort System Procedures Figure 5-1. Sending and Receiving Unblocked Records User Application Process SORTMERGESTART • • • SORTMERGESEND Message Containing One Unsorted Record • • • SORTMERGERECEIVE • • • SORTMERGEFINISH Message Containing One Sorted Record SORTPROG Process VST501.vsd Figure 5-2 on page 5-8 shows how FastSort transfers blocked records between your process and SORTPROG if you use nowait I/O.
SORTBUILDPARM Procedure Using FastSort System Procedures Figure 5-2. Sending and Receiving Blocked Records User Application process SORTBUILDPARM SORTMERGESTART • • • SORTMERGESEND • • • SORTMERGERECEIVE • • • SORTMERGEFINISH Message containing a block of unsorted records Define buffer One record at a time Double Buffer One record at a time Message containing a block of sorted records SORTPROG process VST502.
SORTERROR Procedure Using FastSort System Procedures SORTERROR Procedure Use SORTERROR to provide the message text for the last FastSort error code returned by a FastSort procedure. { length := } SORTERROR ( ctlblock { CALL } , buffer ) length ! i ! o returned value INT returns the number of characters in the error message. ctlblock input INT:ref:200 is the same global storage array you name in the call to SORTMERGESTART.
SORTERRORSUM Procedure Using FastSort System Procedures returns error codes and the index of an input file in a double-word integer. The high-order word contains the file-system or NEWPROCESS error code. The loworder word contains the FastSort error code in the low-order byte and an index identifying the input file that caused the error in the high-order byte. The index is one of those in the array of file names created by the in-file-name parameter of SORTMERGESTART.
SORTERRORSUM Procedure Using FastSort System Procedures ctlblock input INT:ref:200 is the same global storage array you name in the call to SORTMERGESTART. You should not rely on the information in ctlblock, because this information can change without warning. buffer output INT:ref:* is a 16-word integer array that receives the error message text. SORTPROG does not pad the text with blanks if the buffer is shorter than 16 words. Any bytes to the right of the text remain unchanged.
SORTMERGEFINISH Procedure Using FastSort System Procedures Value Cause of Error 3 Scratch file 4 The free-list file (an additional scratch file that SORTPROG allocates for internal memory management when sorting large amounts of data) 5 Process communication subsort-index output INT:ref:1 receives the relative number of a subsort process that caused the last error.
SORTMERGERECEIVE Procedure Using FastSort System Procedures ctlblock input INT:ref:200 is the same global storage array you name in the call to SORTMERGESTART. You should not rely on the information in ctlblock, because this information can change without warning. abort input INT:value specifies when the SORTPROG process should stop: 0 Specifies that the SORTPROG process stop after completion of the current sort or merge run. This is the default value.
SORTMERGERECEIVE Procedure Using FastSort System Procedures ctlblock input INT:ref:200 is the same global storage array you name in the call to SORTMERGESTART. You should not rely on the information in ctlblock, because this information can change without warning. record-loc output INT:ref:* is a memory location for receiving a record. The maximum record size from SORTMERGESTART determines the maximum length of this buffer. You must specify record-loc or record-loc-ext, but you cannot specify both.
SORTMERGESEND Procedure Using FastSort System Procedures The sequence number as a 32-bit (4-byte) integer (permutation sort) The key-field values strung together (key sort) The sequence number followed by the key-field values strung together (permutation and key sort) Receiving Output Records in Extended Memory You can receive output records from SORTPROG in an extended data segment (which must be in use at the time of the call).
SORTMERGESEND Procedure Using FastSort System Procedures status returned value INT returns a FastSort error code if an error occurred; if not, returns 0. ctlblock input INT:ref:200 is the same global storage array you name in the call to SORTMERGESTART. You should not rely on the information in ctlblock, because this information can change without warning. record-loc input INT:ref:* is the memory location of an input record. You must specify record-loc or record-loc-ext, but you cannot specify both.
SORTMERGESEND Procedure Using FastSort System Procedures spare-1 and spare-2 reserved are reserved for future parameters. If you specify a value for spare1 or spare2, FastSort returns an error. However, if you specify record-loc-ext, you must put the commas in the call to reserve places for these parameters.
SORTMERGESEND Procedure Using FastSort System Procedures record-loc-ext input INT(32) is the extended memory location of an input record. You must specify record-loc-ext or record-loc, but you cannot specify both parameters. record-loc-ext must be used instead of record-loc, for records of size greater than 4072 bytes. Guidelines Follow these guidelines when you call the SORTMERGESEND procedure.
SORTMERGESTART Procedure Using FastSort System Procedures Merging Records From Input Streams An input stream is a source of sorted records for merging. You can specify up to 32 input streams in the call to SORTMERGESTART, with the num-merge-files and in-file-name parameters. The in-file-name parameter must specify all blanks as the name for each input stream. The first call to SORTMERGESEND sends the first input record from stream 0.
SORTMERGESTART Procedure Using FastSort System Procedures FastSort through a program. A COBOL85 program can call SORTMERGESTART through the SORT or MERGE statement.
SORTMERGESTART Procedure Using FastSort System Procedures key-block input INT:ref:* is an integer array defining the key fields. Its size is one word plus three words for each key. The first word contains the total number of keys. The rest of the array contains three-word descriptions of the keys. The maximum number of keys is 63. For more information, see Key-Field Definitions in the Key-Block Array on page 5-30.” num-merge-files input INT:value is the number of input files for merging.
SORTMERGESTART Procedure Using FastSort System Procedures If you omit in-file-name or if it equals all blanks, your process must call SORTMERGESEND to send each input record to SORTPROG. SORTMERGESEND cannot send both sorted and unsorted records for the same sort run. For more information, see SORTMERGESEND Procedure on page 5-15. You can specify the same file in both in-file-name and out-file-name for a sort run but not for a merge run. Caution.
SORTMERGESTART Procedure Using FastSort System Procedures in-file-count input INT(32):ref:* is an array including one 32-bit entry for each input file. Each entry contains the maximum number of records in the corresponding input file. When input is from a source other than disk, SORTPROG uses in-file-count to estimate the space required for the scratch file.
SORTMERGESTART Procedure Using FastSort System Procedures 0 The output records are in the same format as the input records. This is a record sort, the default SORTPROG uses when you omit format. 1 The output records are 32-bit integers describing the order of the sorted records. This is a permutation sort. For example, if the 20th input record is first in order after sorting, 20 is the value of the first output record.
SORTMERGESTART Procedure Using FastSort System Procedures If you specify –1 or omit this parameter, FastSort uses one of the following default exclusion modes: Device Exclusion Mode Disk or magnetic tape file EXCLUSIVE Temporary disk file SHARED Terminal SHARED out-file-type input INT:value specifies the type of file SORTPROG creates for the output records.
SORTMERGESTART Procedure Using FastSort System Procedures receives a completion code of 0 if no error occurred or receives error codes if an error occurred. The high-order word has the file-system or NEWPROCESS error code. The low-order byte of the low-order word has the FastSort error code: Bits Parameter Word 0 High-Order Low-Order 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 File-System or NEWPROCESS[NOWAIT] Error Code FastSort Input File Index FastSort Error Code VST503.
SORTMERGESTART Procedure Using FastSort System Procedures If you do not specify subsort-count, then scratch-file-name is the initial scratch file for a serial sort or merge run. If you omit scratch-file-name or it equals all blanks, SORTPROG creates a scratch file on a suitable volume. You cannot omit or use blanks for a subsort scratch-file-name. For more information about scracth files, see Table 5-4 on page 5-32.
SORTMERGESTART Procedure Using FastSort System Procedures Table 5-2. Expanded NEWPROCESS Structure Word Entity Description 0* Priority Assigns the priority of the SORTPROG process. If priority equals –1, the default value, the SORTPROG process has the same priority as the calling process. 1* Memory Specifies the maximum number of data pages the SORTPROG process can use. SORTPROG always uses 64 for this value. 2* CPU Specifies the number of the processor (CPU) in which SORTPROG runs.
SORTMERGESTART Procedure Using FastSort System Procedures max-record-length output INT:ref:* You should specify max-record-length as a reference to a single 16-bit word used for OUTPUT. In max-record-length, SORTMERGESTART returns the size of the largest output record that FastSort writes to the output file or returns through SORTMERGERECEIVE. collate-sequence-table input STRING:ref:256 is a 256-byte array defining an alternate collating sequence for SORTPROG to use in the sort or merge run.
SORTMERGESTART Procedure Using FastSort System Procedures Other flags2 bits are not used and should be set to 0. subsort-count input INT:value specifies the number of subsort processes from 2 to 8. Higher numbers can cause run-time errors, depending on your system configuration and the system load. If subsort-count equals n, scratch-file-name and process-start become integer arrays of dimension n + 1. spare5 reserved INT:value is reserved for a future parameter.
SORTMERGESTART Procedure Using FastSort System Procedures Table 5-3.
SORTMERGESTART Procedure Using FastSort System Procedures Table 5-4. SORTMERGESTART flags Parameter Bits (page 1 of 2) Flag Meaning No purge of existing output file Structure for NEWPROCESS parameters MINSPACE mode MINTIME mode Scratch file size check Alternate collating sequence table flags Bit Value Description <5> 0 SORTPROG purges an existing output file that seems too small. This value is the default.
SORTMERGESTART Procedure Using FastSort System Procedures Table 5-4. SORTMERGESTART flags Parameter Bits (page 2 of 2) Flag Meaning Removal of records that have duplicate keys Saving scratch files Creating a new scratch file Creating a new output file Restart option flags Bit Value Description <11> 0 SORTPROG keeps all records that have duplicate keys. This value is the default. 1 SORTPROG removes every record whose keys are all duplicates of a previous record's keys.
SORTMERGESTART Procedure Using FastSort System Procedures Input Files Follow these guidelines for input files: SORTPROG accepts all types of input files except blocked tape files and processes. SORTPROG accepts up to 32 input files. The files can contain fixed-length or variable-length records. The sum of number-merge-files and number-sort-files must be at least 1 file. Although both parameters are optional, you must specify one of them.
SORTMERGESTART Procedure Using FastSort System Procedures You can use only one sort key field, and the data type for the field must be BINARY UNSIGNED. The sort key field must be the same as the file's primary key field. You must specify ascending in the key-block array. You can specify the dslack and islack parameters for an existing key-sequenced output file.
SORTMERGESTART Procedure Using FastSort System Procedures Before your process can call SORTMERGESTART with the restart flag set to 1, your process must call SORTMERGESTATISTICS or an error must end the SORTPROG process. Each call to SORTMERGESTART must specify the same sort control block. If you call the SORTBUILDPARM procedure, its restart flag (build-flags.<15>) must be set to 0 for the first call and to 1 for successive calls.
SORTMERGESTART Procedure Using FastSort System Procedures sortfiles,,,, len,,,,, restart); ! restart.<15> is 1 ! Go to the first statement that calls SORTMERGESEND. ... error := CALL SORTMERGEFINISH (sortblock); ... END; ! End of the routine Writing a User Error Procedure You can use the errproc parameter of SORTMERGESTART to specify a TAL procedure to call if an error ends the SORTPROG process.
SORTMERGESTATISTICS Procedure Using FastSort System Procedures ,sorterrproc ); ... SORTMERGESTATISTICS Procedure Use SORTMERGESTATISTICS to obtain information about a successful sort or merge run after SORTPROG completes the run. { status := } SORTMERGESTATISTICS ( ctlblock { CALL } ,length ,statistics ,[flag1 ] ,[spare1 ] status ! i ! i, o ! o ! i !reserved returned value returns a FastSort error code if an error occurred; if not, status returns 0.
SORTMERGESTATISTICS Procedure Using FastSort System Procedures flag1 input INT:value tells FastSort to use the 22-word array to return statistics if this parameter is present and set to 1. If flag1 is present but set to 0 or if flag1 is not present, FastSort uses the 21-word statistics array and converts BUFFER PAGES from an INT(32) to an INT value before it reaches the array. For BUFFER PAGES, FastSort returns the value -1 for values greater than 32,767.
SORTMERGESTATISTICS Procedure Using FastSort System Procedures Table 5-5. SORTMERGESTATISTICS statistics Structure Word Type Description 17 INT MERGE ORDER: maximum number of runs that can be merged at one time 18 INT INTERMEDIATE PASSES: number of merge cycles between initial run formation and final merge 19:20 INT(32) NUMBER OF DUPLICATES: number of duplicate records SORTPROG removed * If VLM is on, BUFFER PAGES is an INT(32) value and all subsequent words in this array move up one word.
6 Sorting in Parallel If the total input file size is larger than one megabyte, a parallel sort run can provide better performance in elapsed execution time than a serial sort run. A parallel sort operation improves performance because it: Distributes the workload to multiple processors Uses scratch files on multiple disks For a parallel sort run, you set up a distributor-collector process and from 2 to 8 subsort processes. Note.
Using Commands for Parallel Sorting Sorting in Parallel Figure 6-1. Parallel Sorting Processor 1 Subsort Process 1 Input File $DATA1 Processor 0 Input File $DATA2 Distributor Collector Process $WORK1 Scratch file Processor 2 Subsort Process 2 $WORK2 Scratch file Processor 3 Input File $DATA3 Subsort Subsort Process Process 3 3 $WORK3 Scratch file VST601.vsd This section gives guidelines for using FastSort commands, procedures, and parameters for parallel sorting.
Using Procedures for Parallel Sorting Sorting in Parallel 5. If you want to specify a group of processors for running subsort processes, list the processors SORTPROG can use in a CPUS command. Specify any processors SORTPROG cannot use in a NOTCPUS command. For more information, see Selecting Processors to Run Subsort Processes on page 6-7. 6. Set up the distributor-collector process in the RUN command that starts the parallel sort operation.
Using the Automatic Configuration Sorting in Parallel 5. SORTMERGEFINISH stops the distributor-collector process. Instead of specifying input files, you can use calls to SORTMERGESEND after the call to SORTMERGESTART. Instead of specifying an output file, you can use calls to SORTMERGERECEIVE after the last call to SORTMERGESEND, if any, or after the call to SORTMERGESTART. The TAL example in Example 6-3 on page 6-23 shows how to use procedure calls for a parallel sort run.
Using FastSort Commands Sorting in Parallel An extended memory segment for each subsort process of at most 50 percent of the processor’s physical memory not locked down by the operating system A scratch file size for each subsort process equal to the output file size divided by the number of subsort processes plus 6 bytes per record for overhead FastSort computes the sizes of the extended memory segments and scratch files for you.
Improving Performance Sorting in Parallel The SCRATCHFILES array contains an entry for a distributor-collector process scratch file and three entries for subsort initial scratch files.
Selecting Processors to Run Subsort Processes Sorting in Parallel If you use commands to configure the parallel sort run, you can specify the parameters for each subsort process in a SUBSORT command. Some parameters of the RUN command also affect the configuration of subsort processes, as explained under Configuring a Distributor-Collector Process on page 6-10.
How FastSort Selects Processors Sorting in Parallel For example, you have a system with eight processors, and you want to run four subsort processes to sort the records from a large file. To allow for peak capacity, do not load any of the processors over 60 percent. Processors number 2 and 5 are generally 50 to 60 percent busy. Processor 0 runs the distributor-collector process because it has the lightest load.
Specifying the Size of the Extended Memory Segment Sorting in Parallel Specifying the Size of the Extended Memory Segment If you do not specify an extended segment size for a subsort process, FastSort tries to use enough memory for the subsort to make only one merge pass. In the automatic configuration, the segment size is at most 50 percent of the processor’s physical memory not locked down by the operating system.
Using Multiple Copies of the SORTPROG Program Sorting in Parallel Using Multiple Copies of the SORTPROG Program By default, each subsort process uses the SORTPROG program in the $SYSTEM.SYSnn.SORTPROG file. To run a subsort process from another local disk volume, follow these steps: 1. Duplicate the SORTPROG program to a file on the target local disk volume and use the FUP LICENSE command to license it. 2.
Specifying a Scratch Block Size Sorting in Parallel To improve performance for a parallel sort run on your system, you can specify one or more of the following options in the RUN command or in the SORTMERGESTART procedure: The size of the I/O blocks for all scratch files The size of the extended memory segment for the distributor-collector process and for each subsort process A location for the swap file for the distributor-collector process’s extended memory segment A copy of the licensed SOR
Specifying a Location for the Swap File Sorting in Parallel The SEGMENT parameter or the segment word of the process-start parameter overrides AUTOMATIC, MINSPACE, or MINTIME. However, if you specify more than 90 percent of the processor’s physical memory not locked down by the operating system, FastSort returns an error.
Specifying an Execution Priority Sorting in Parallel Specifying an Execution Priority The default execution priority for the distributor-collector process is the operating system’s default priority for a process.
Understanding Statistics From Parallel Sorting Sorting in Parallel 5. Avoid intermediate merge passes for subsort processes. Use enough extended memory for each subsort process to make only one merge pass. For information about how much extended memory you need for each subsort process with different sizes of files, see Controlling Extended Memory on page 2-11. If you do not have enough memory available in each processor, add enough subsort processes to limit the number of merge passes to one.
Parallel Sorting From C Programs Sorting in Parallel Parallel Sorting From C Programs Example 6-1 shows a C program that calls FastSort system procedures to perform a parallel sort run. Example 6-1.
Parallel Sorting From C Programs Sorting in Parallel Example 6-1.
Parallel Sorting From C Programs Sorting in Parallel Example 6-1. C Example of a Parallel Sort Run (page 3 of 5) /*-------------------------------------------------------------*/ /* Initialize SORT key definitions array.
Parallel Sorting From C Programs Sorting in Parallel Example 6-1. C Example of a Parallel Sort Run (page 4 of 5) /*-------------------------------------------------------------*/ /* Call SORTMERGESTART to start the SORTPROG processes. */ /*-------------------------------------------------------------*/ error = SORTMERGESTART (&ctlblk[0], &key_array[0],,1, (short *) &infile[0],,,,, (short *) &outfile[0],,, flags,,, (short *) &scratch_files[0].filename[0],, (short *) &newprocess_parms[0].
Parallel Sorting From COBOL85 Programs Sorting in Parallel Example 6-1.
Sorting in Parallel Parallel Sorting From COBOL85 Programs Example 6-2. COBOL85 Example of a Parallel Sort Run (page 1 of 3) *-----------------------------------------------------------* FastSort Parallel Sort Run * This program calls the COBOL85 interface routines * COBOL85^SET^SORT^PARAM^VALUE and * COBOL85^SET^SORT^PARAM^TEXT to start a parallel sort run. *-----------------------------------------------------------?SYMBOLS, INSPECT ?LIBRARY $SYSTEM.SYSTEM.COBOLLIB ?LIBRARY $SYSTEM.SYSTEM.
Sorting in Parallel Parallel Sorting From COBOL85 Programs Example 6-2. COBOL85 Example of a Parallel Sort Run (page 2 of 3) *-----------------------------------------------------------* Specify the number of subsort processes. *-----------------------------------------------------------MOVE "SUBSORT-COUNT" TO VALUE-PARAM. ENTER "COBOL85^SET^SORT^PARAM^VALUE" USING SORT-FILE, VALUE-PARAM, NUMBER-OF-SUBSORTS GIVING RETURN-CODE. IF RETURN-CODE NOT = 0 PERFORM ERROR-RETURN END-IF.
Parallel Sorting From TAL Programs Sorting in Parallel Example 6-2. COBOL85 Example of a Parallel Sort Run (page 3 of 3) *-----------------------------------------------------------* Return the sorted records. *-----------------------------------------------------------OUTPUT-SECTION SECTION. OUTPUT-ROUTINE. DISPLAY "Output sort procedure entered...". SET MORE-OUTPUT TO TRUE.
Parallel Sorting From TAL Programs Sorting in Parallel Example 6-3. TAL Example of a Parallel Sort Run (page 1 of 3) ?SYMBOLS, NOCODE, INSPECT, MAP, LMAP, DATAPAGES 64 !-----------------------------------------------------------! ! FastSort Parallel Sort Run ! ! This program uses 3 subsorts to sort an input file. ! !-----------------------------------------------------------! ! Global Declarations. ! !-----------------------------------------------------------! INT .
Parallel Sorting From TAL Programs Sorting in Parallel Example 6-3. TAL Example of a Parallel Sort Run (page 2 of 3) !-----------------------------------------------------------! ?PAGE "MAIN Procedure" !-----------------------------------------------------------! PROC main^proc MAIN; BEGIN INT length; !-----------------------------------------------------------! ! Open the home terminal and call the ! ! INITIALIZER to read the startup message.
Parallel Sorting From TAL Programs Sorting in Parallel Example 6-3. TAL Example of a Parallel Sort Run (page 3 of 3) !-----------------------------------------------------------! ! Return statistics and check the sort completion. ! !-----------------------------------------------------------! error := SORTMERGESTATISTICS (ctlblk,length,stat); IF error THEN ! Check for SORTMERGESTATISTICS error.
Parallel Sorting From TAL Programs Sorting in Parallel FastSort Manual—429834-003 6-26
7 Using SORT and SUBSORT DEFINEs Before you start a sort or merge run, you can set or change operating system parameters that affect FastSort. While the FastSort default settings are often sufficient for small sort or merge runs, modifying the default settings can improve the performance of a large run. You modify default settings for a sort operation with CLASS SORT and SUBSORT DEFINEs. You can set or change a DEFINE either interactively or programmatically.
Setting DEFINE Attributes Using SORT and SUBSORT DEFINEs Priority Value Type Specified By 1 User-specified A SORT or SUBSORT DEFINE 2 User parameter A FastSort interactive parameter or system procedure call 3 Default The =_SORT_DEFAULTS DEFINE Use a single method to specify values for a subsort process. For subsort processes, FastSort treats all parameter values as a single entity. After FastSort determines the source of information for a subsort process, it ignores values from other sources.
Setting SORT DEFINE Attributes Using SORT and SUBSORT DEFINEs CPUS { processor [, processor ]... | ALL } specifies the processor (CPU) numbers that are available for subsort processes, The range is 0 through 15. A value of ALL means that all processors are available for subsorts.
Setting SORT DEFINE Attributes Using SORT and SUBSORT DEFINEs PROGRAM file-name specifies a local or remote program file name to run in place of the default program file, such as: SET DEFINE PROGRAM $data.fastsort.sortprog SCRATCH file-name specifies a disk file name or disk volume name for an initial scratch file. This attribute is required. For example: SET DEFINE SCRATCH $data.fastsort.scratch If the file already exists, it must be unstructured.
Setting SUBSORT DEFINE Attributes Using SORT and SUBSORT DEFINEs SUBSORTS ( DEFINE-name [, DEFINE-name ]... ) specifies a list of DEFINE names for subsort processes. Separate the DEFINE names with commas and enclose them in parentheses, such as: SET DEFINE SUBSORTS (=subsorta, =subsortb, =subsortc) FastSort checks DEFINE names for validity at run time. SWAP file-name specifies the name of a swap file to use in an extended memory data segment.
Setting SUBSORT DEFINE Attributes Using SORT and SUBSORT DEFINEs PRI priority specifies the priority for the subsort process. The range is 1 through 199. The default is the operating system priority for the parent process. SET DEFINE PRI 180 PROGRAM file-name specifies a local or remote program file name to run for the subsort process in place of the default program file, such as: SET DEFINE PROGRAM $data.another.
Creating and Using DEFINEs Interactively Using SORT and SUBSORT DEFINEs Creating and Using DEFINEs Interactively Use the TACL DEFINE commands listed below to interactively create and modify SORT and SUBSORT DEFINEs. The operating system places the DEFINEs in the process file segment (PFS) of your TACL process. Command Description ADD DEFINE Creates a DEFINE in the PFS of the current TACL process. ALTER DEFINE Changes the attribute settings of an existing DEFINE in the PFS.
Displaying a DEFINE Using SORT and SUBSORT DEFINEs You can also create the same SORT DEFINE using a single command as shown in the next example. The ampersand (&) is the continuation character for a TACL command that continues on the next physical line. ADD DEFINE =distributor_collector, CLASS SORT, SCRATCH $disk.fastsort.scratch, SWAP $disk.fastsort.
Creating a SUBSORT DEFINE Using SORT and SUBSORT DEFINEs Creating a SUBSORT DEFINE A SUBSORT DEFINE controls a subsort process in a parallel sort run. Specify between 2 and 8 SUBSORT DEFINEs for a SORT DEFINE. Note. FastSort supports up to 16 SUBSORT DEFINEs; however, to prevent run-time errors and performance problems, HP recommends that you specify no more than 8 SUBSORT DEFINEs. You create a SUBSORT DEFINE in the same manner as a SORT DEFINE.
Deleting a DEFINE Using SORT and SUBSORT DEFINEs INFO DEFINE =distributor_collector, DETAIL Define Name =DISTRIBUTOR_COLLECTOR DEFINE CLASS SORT SWAP $DISK.FASTSORT.SWAPFILE MODE MINSPACE SUBSORTS (=SUBSORTB,=SUBSORTC,=SUBSORTD) To delete an attribute from the working attribute set before you create a new DEFINE, use the RESET DEFINE command. The following example deletes all previously specified SUBSORT DEFINE names for the =DISTRIBUTOR_COLLECTOR DEFINE.
Creating and Modifying DEFINEs Programmatically Using SORT and SUBSORT DEFINEs Procedure Description DEFINEADD Creates a DEFINE for the user from the working attribute set. DEFINEDELETE Deletes a specific DEFINE for the user. DEFINEDELETEALL Deletes all DEFINEs for the user. DEFINEINFO Returns information about a DEFINE. DEFINEMODE Sets the DEFINE mode (DEFMODE variable) for the user process. DEFINENEXTNAME Returns the name of the DEFINE that follows the specified DEFINE.
Using DEFINEs With Programmatic FastSort Using SORT and SUBSORT DEFINEs run your application.) After you have set the necessary attributes, you use the DEFINEADD procedure to name the DEFINE and add it to your application’s PFS. The following TAL example shows the programmatic use of a SORT DEFINE named =SORT^DEFINE. The CONVERT^INT^TO^STRING and ERROR^RECOVERY procedures not shown in this example are user-written procedures.
Creating and Using the =_SORT_DEFAULTS DEFINE Using SORT and SUBSORT DEFINEs The following TAL example uses a SORT DEFINE named =SORT^DEFINE in the SORTBUILDPARM procedure. The operating system does not check the existence or validity of the SORT DEFINE until the sort operation begins. PROC sort^procedure; BEGIN INT .sort^define^name[0:11] := [ 12 * [" "]]; ... sort^define^name ':=' ["=sort^define "]; ... ! Set the other SORTBUILDPARM parameters.
Using SORT and SUBSORT DEFINEs Define Name CLASS Creating and Using the =_SORT_DEFAULTS DEFINE =_SORT_DEFAULTS SORT The following ADD DEFINE command creates the =_SORT_DEFAULTS DEFINE and sets the SCRATCH, SWAP, and CPU attributes: ADD DEFINE =_SORT_DEFAULTS, CLASS SORT, SCRATCH $disk.scratch.file SWAP $disk.swap.file CPU 8 & & & You can change the settings of the current attributes of the =_SORT_DEFAULTS DEFINE using the ALTER DEFINE command.
Examples of SORT and SUBSORT DEFINEs Using SORT and SUBSORT DEFINEs
Serial Sort Run Example Using SORT and SUBSORT DEFINEs SET DEFINE CPU 5 SET DEFINE MODE AUTOMATIC SET DEFINE PRI 170 Display the current attribute set: SHOW DEFINE * CLASS SCRATCH SCRATCHON NOSCRATCHON SWAP MODE CPU BLOCK PRI SEGMENT PROGRAM CPUS NOTCPUS SUBSORTS VLM SORT $DISK $DATA AUTOMATIC 5 170 Create the SORT DEFINE and display the current attribute set for all current DEFINEs. The current attribute set for the =SORT_RUN DEFINE is adopted from the working attribute set.
Parallel Sort Run Example Using SORT and SUBSORT DEFINEs Parallel Sort Run Example The following example shows a parallel sort run using a SORT DEFINE named =PARALLEL_SORT and SUBSORT DEFINEs named =SUBSORTA and =SUBSORTB. (The input file FRUIT contains only 8 records; however, an actual input file would be much larger to require a parallel sort operation.) SET DEFMODE ON ADD DEFINE =parallel_sort, CLASS SORT, SCRATCH $disk.fastsort.
Parallel Sort Run Example Using SORT and SUBSORT DEFINEs CPU 3 Define Name CLASS SCRATCH CPU =SUBSORTB SUBSORT $DISK 6 Define Name CLASS VOLUME SWAP =_DEFAULTS DEFAULTS $DISK.
8 Sorting From NonStop SQL/MP Under certain circumstances, SQL/MP invokes FastSort in a manner that is transparent to the user. SQL/MP invokes FastSort when you do any of the following: Specify ordering or grouping options in an SQL query statement Execute a query that results in a sort merge join operation Use a CREATE INDEX or LOAD statement to load data in parallel Because SQL/MP automatically invokes FastSort, this section describes how a sort operation is implemented.
Configuring Your SQL/MP Sort Environment Sorting From NonStop SQL/MP An external FastSort process (SORTPROG), if the SQL optimizer estimates that the data to sort might exceed 4 MB, or there are fewer than 32,768 rows or fewer than 63 columns A sort performed by a series of inserts into a temporary key-sequenced table, if both of the following conditions apply: The table contains more than 500 rows and more than 63 columns The total key length is less than 255 bytes Note.
Sorting From NonStop SQL/MP Setting Up a =_SORT_DEFAULTS DEFINE You can create or modify a =_SORT_DEFAULTS DEFINE directly from your SQLCI prompt.
Ordering and Grouping Query Results Sorting From NonStop SQL/MP Guidelines for =_SORT_DEFAULTS DEFINE Attributes To optimize sort performance for SQL/MP, HP recommends you follow these guidelines for SORT and SUBSORT attributes in your =_SORT_DEFAULTS DEFINE: Attribute Recommended value VLM OFF ON for nonparallel LOAD operations PRI 180 for high priority users and queries 160 for most users and queries 80 for routine load operations and queries PROGRAM A local file CPU A processor that is less th
Optimizing SQL Clause Combinations Sorting From NonStop SQL/MP UNION (without the ALL option) and the specified columns do not match a prefix of the index columns. Note. NonStop SQL/MP does not invoke FastSort if the optimizer chooses a query plan that reads the base table by primary key value. If you specify more than one SQL ordering or grouping clause in a query, you can often structure the query to avoid duplicate sorts.
Using a Sort Merge Join Sorting From NonStop SQL/MP The GROUP BY list is a subset of the SELECT DISTINCT list, as in the following query: SELECT DISTINCT COUNT(*), BOSTON, BOSTON-DALLAS, DALLAS FROM SALES GROUP BY BOSTON, DALLAS ; A single sort on (BOSTON, DALLAS) groups the query results. Because each (BOSTON, DALLAS) value is unique after grouping, each (BOSTON, DALLAS, BOSTON-DALLAS, COUNT(*)) value is also unique.
Loading Data Sorting From NonStop SQL/MP joining column, FastSort also sorts the outer table data and stores it in a second temporary entry-sequenced table. The two temporary tables are then merged to form the sort merge join result. By default, FastSort creates these temporary tables on the default swap volume. To avoid disk space contention, move the swap file to a volume other than the default.
Configuring a CREATE INDEX Statement Sorting From NonStop SQL/MP Figure 8-1. Parallel Loading Data Into a Partitioned Index Table Base Table Partitions Disk Process Block Mode Interface RECGEN RECGEN RECGEN SORTPROG SORTPROG SORTPROG SQL Catalog Manager SQLLOAD Routines Index Table Partitions VST801.vsd Because parallel processing uses more concurrent CPU cycles and disk processes than serial processing, loading data in parallel could temporarily monopolize system resources.
Configuring a CREATE INDEX Statement Sorting From NonStop SQL/MP When you create a partitioned index on a base table and specify PARALLEL EXECUTION ON, you can use either the default configuration or a custom configuration file. A configuration file defines attributes of record generator and sort processes. You can specify the name of a configuration file in the PARALLEL EXECUTION clause of a CREATE INDEX statement. If you specify no configuration file, FastSort uses the default configuration.
Configuring a CREATE INDEX Statement Sorting From NonStop SQL/MP same volume, processor and disk space contention problems can result. Use a configuration file to specify a unique swap volume for each remote partition. Using a Custom Configuration File When you create an index and specify PARALLEL EXECUTION ON, you can use the CONFIG option to specify a custom configuration file. The configuration file must be an EDIT file.
Configuring a CREATE INDEX Statement Sorting From NonStop SQL/MP The CREATE INDEX statement precedes all configuration information for the parallel load operation. SQL/MP reads only CREATE INDEX statements in the configuration file. The syntax for CREATE INDEX is as follows: {LOCALONLY} CREATEINDEX{BASETABLE}{DEFAULT [node-name] default-attr {INDEX }{partition attr [,attr ]... } } where default-attr is: [ [ [ [ [ [ [ [ CPU ( num [, num ] ... ) NOSCRATCHON (scratchvol [, scratchvol ]...
Configuring a CREATE INDEX Statement Sorting From NonStop SQL/MP DEFAULT [node-name] default-attr specifies an attribute-value pair for partitions on a node for which no value has been explicitly specified. If you omit node-name, SQL/MP applies the DEFAULT statement to the node where the parallel index load is initiated. For more information on how to use this option, see Assigning Default and Explicit Values on page 8-14.
Sorting From NonStop SQL/MP Configuring a CREATE INDEX Statement NUMRECS ( number ) is valid only if you specify INDEX. NUMRECS specifies the approximate number of records to load into the index partition. Use NUMRECS if the index is unevenly partitioned across volumes. FastSort uses this number to calculate initial scratch file size. PRI ( priority ) is valid only if you specify INDEX or BASETABLE. PRI specifies the priority at which to run the record generator or sort process.
Configuring a CREATE INDEX Statement Sorting From NonStop SQL/MP When you specify overflow scratch volumes in a configuration file, the values you specify override any values in a =_SORT_DEFAULTS DEFINE. You cannot specify both NOSCRATCHON and SCRATCHON. If you do not specify either SCRATCHON or NOSCRATCHON, FastSort considers using any volume, except $SYSTEM and TMF audit trail volumes, for overflow scratch files.
Configuring a LOAD Statement Sorting From NonStop SQL/MP == == == == $DATA4.SALES.AGEINDEX $DATA5.SALES.AGEINDEX \NEWYORK.$DATA2.SALES.AGEINDEX \NEWYORK.$DATA3.SALES.AGEINDEX Set up a default priority for the RECGEN processes: CREATEINDEX BASETABLE DEFAULT PRI ( 140 ) CREATEINDEX BASETABLE DEFAULT \NEWYORK PRI ( 140 ) == == Set up a default pool of scratch files for the sort processes.
Configuring a LOAD Statement Sorting From NonStop SQL/MP When you execute a LOAD statement from SQLCI, you invoke FastSort if data is unsorted and the target table is key-sequenced, or if PARALLEL EXECUTION is set to ON. This subsection discusses only the LOAD options that affect sort operations. For a full description of LOAD statement syntax, see the SQL/MP Reference Manual.
Configuring a LOAD Statement Sorting From NonStop SQL/MP DEFINE. For more information about partitioned scratch files, see Using a Partitioned Scratch File on page 9-8. If you specify the SORTED option, you can omit the SCRATCH option. Loading Large Tables Use the following sort workspace guidelines to load data into a large table. Setting MAX Number of Records LOAD uses the MAX parameter to estimate file and extent size for an initial scratch file.
Configuring a LOAD Statement Sorting From NonStop SQL/MP FastSort Manual—429834-003 8-18
9 Optimizing Sort Performance Factors that affect FastSort performance include environmental options, sort workspace, and system resources. The total elapsed time for a sort operation also depends on whether you automate routine tasks, such as setting up DEFINEs. This section helps you understand FastSort software behavior and requirements. It contains a discussion of scratch and swap files, VLM, and other factors that affect sort performance.
Using Scratch Files Optimizing Sort Performance Manually Creating a Scratch File You can use the FUP CREATE command to manually create an unstructured scratch file. You can also programmatically create a scratch file with the CREATE system procedure.
Using Scratch Files Optimizing Sort Performance Specify a scratch file that does not exist. SORTPROG creates an initial scratch file on a volume selected by DEFINEs or volume characteristics. Specify only a volume name. SORTPROG creates an initial scratch file on the specified volume. Even when SORTPROG creates a scratch file, the file is sometimes too small to hold all of the records.
Using Scratch Files Optimizing Sort Performance DEFINE, FastSort automatically selects a scratch volume based on volume characteristics. For more information about selection criteria, see How Volume Characteristics Affect Selection on page 9-5.
Using Scratch Files Optimizing Sort Performance scratch volumes. FastSort supports up to 32 total scratch volumes: one initial volume in the SCRATCH attribute and up to 31 SCRATCHON overflow volumes. Note. FastSort uses up to 100 percent of the disk space on volumes you specify in the SCRATCH and SCRATCHON attributes. Therefore, if you explicitly specify scratch volumes, ensure that other processes do not currently require disk space on those volumes.
Using Scratch Files Optimizing Sort Performance Table 9-1. How FastSort Chooses Scratch Volumes What You Specify: How FastSort Responds: Nothing: no scratch file, no DEFINEs with scratch attributes, no scratch volumes, no restrictions on scratch volumes Uses volume characteristics to select a scratch volume. FastSort creates scratch files on this volume until it is 80 percent full, selects another scratch volume if necessary, and so on. A scratch file Uses the file until it becomes full.
Optimizing Sort Performance Selecting a Scratch Volume for Parallel Sorts Each scratch file extent can be up to 2048 pages, or 4 KB. For scratch files that FastSort creates, the default extent size is 4 KB and MAXEXTENTS is 978 extents. Depending on extent sizes, a nonpartitioned scratch file can be up to 1 TB in size. Selecting a Scratch Volume for Parallel Sorts For parallel sorts, each subsort process uses its own initial and overflow scratch files.
Using a Partitioned Scratch File Optimizing Sort Performance Using a Partitioned Scratch File A partitioned scratch file is a single scratch file partitioned across multiple disk volumes. The multiple volumes can exist on separate nodes. A partitioned scratch file functions in essentially the same manner during a sort operation as multiple scratch files.
Using a Partitioned Scratch File Optimizing Sort Performance The following syntax creates the file SCRATCH with two secondary partitions: FUP CREATE SCRATCH, PART (1, $data3, 64, 8), PART (2, $data4, 64, 8) In this example, a primary file partition, SCRATCH, is created on the current node, volume, and subvolume. Two secondary partitions, also named SCRATCH, are created on $data3. and $data4. on the current node.
Using Swap Files Optimizing Sort Performance Using Swap Files A swap file is the disk file used for data swapping during a sort or merge run. Data swapping is the process of copying data between physical memory and storage. Swapping, or paging, occurs when the extended memory segment is larger than the available physical memory. Swapping also occurs when processes contend for available memory.
Using VLM Optimizing Sort Performance extended memory, if available. FastSort uses the additional extended memory either to complete the sort in a single pass or to store partial information until the sort is complete. Without VLM, the maximum number of records that FastSort can sort in memory is 32,767. This limit applies regardless of the amount of memory available. With VLM, available memory and extended segment size determine the number of records that can be sorted in memory.
Calculating Data Stack Space Optimizing Sort Performance VLM can also increase performance for sorts that do require scratch files. For sorts that require an intermediate merge pass, FastSort uses the additional memory to store partial information. The additional storage space reduces reads and writes to scratch files. How VLM Affects Statistics When VLM is on, the FastSort statistics format changes slightly. For interactive FastSort, BUFFER PAGES changes from an INT to an INT(32) value when VLM is on.
Managing Sort Failures Optimizing Sort Performance For information about TAL compiler directives, see the TAL Reference Manual. For information about the Binder SET command, see the Binder Manual. Other Data Stack Space Considerations In addition to the requirements listed above, if you specify either the SCRATCHON or NOSCRATCHON attributes in a SORT DEFINE, FastSort requires up to 138 additional words (276 bytes) of stack space.
Managing Sort Failures Optimizing Sort Performance $SYSTEM.SYS01.SORTPROG Binder timestamp: Version procedure: Target CPU: 10NOV94 10:12:17 T9620D30^31OCT94^AAU^31OCT94 TNS, TNS/R Compare the information VPROC returns with version information in the softdocs shipped with your NonStop software. These documents contain product and RVU numbers for the specific product they describe. Softdocs also contain software compatibility information.
Automating FastSort Tasks Optimizing Sort Performance first turn on SAVEABEND and then duplicate the failure in order to save information in a save file. Note. Turning SAVEABEND ON also sets the Binder INSPECT option to ON. For more information about the Binder CHANGE command, see Binder Manual.
Automating DEFINEs Optimizing Sort Performance Automating DEFINEs The following examples show how you can use command files to set up FastSort DEFINEs. Using a Command File to Set DEFINEs from TACL The following is an example of a TACL command file that sets SORT DEFINEs for an interactive sort operation: DELETE DEFINE =SORT_ONE SET DEFMODE ON SET DEFINE CLASS SORT SET DEFINE SCRATCH $data.fastsort.scratch SET DEFINE BLOCK 28762 SET DEFINE SCRATCHON ($data??) SET DEFINE SEGMENT 256 SET DEFINE SWAP $data.
Automating DEFINEs Optimizing Sort Performance configures the =_SORT_DEFAULTS DEFINE for a parallel sort operation. SCRATCH1 specifies only volume names for scratch and swap files, as follows: SET DEFINE SCRATCH $DATA2 SET DEFINE SWAP $SPARE2 You could use the SCRATCH1 configuration if the SQL optimizer chooses a parallel plan for a query that invokes FastSort. A second command file, SCRATCH2, configures scratch and swap space for loading data from a large SQL/MP table.
Automating DEFINEs Optimizing Sort Performance FastSort Manual—429834-003 9-18
A FastSort Syntax Summary This appendix contains a syntax summary of the FastSort interactive commands and system procedures. Interactive Commands The FastSort interactive commands are: ASC[ENDING] field [ type ] [ , field [ type ] ]... CLEAR { ALL } { ASC[ENDING] } { COLLATE } { CPUS } { DESC[ENDING] } { FROM [ filename ] } { KEYS } { NOTCPUS } { SUBSORT } { TO } COLLATE filename COLLATEOUT filename CPUS [ ALL ] [ cpu-list ] DESC[ENDING] field [ type ] [ , field [ type ] ]...
Interactive Commands FastSort Syntax Summary NOTCPUS cpu-list RUN [scratch-file |scratch-vol] [ , AUTOMATIC ] [ , BLOCK size ] [ , CPU processor ] [ , MEM memory ] [ , MINSPACE ] [ , MINTIME ] [ , PRI priority ] [ , { REMOVEDUPS | REMD } ] [ , DEFINE define-name ] [ , SEGMENT size ] [ , PROGRAM file ] [ , SWAP file ] [ , NOSCRATCHON(scratch-vol,scratch- vol,...)] [ , SCRATCHON(scratch-vol [,scratchvol]...
FastSort Procedures FastSort Syntax Summary FastSort Procedures The FastSort system procedures are: { status := } SORTBUILDPARM ( ctlblock { CALL } ,[cpu-mask] ,[not-cpu-mask] ,[buffer] ,[buffer2] ,[buffer-length] ,[build-flags] ,[define-name] !reserved1! !reserved2! ,[scratchvols ] ) ! ! ! ! ! ! ! ! { length := } SORTERROR { CALL } ! i ! o ( ctlblock ,buffer ) i i i i i i i i ! i { status := } SORTERRORDETAIL( ctlblock ) { CALL } ! i { length := } SORTERRORSUM { CALL } ! ! ! ! ! ! ( ctlblock ,
FastSort Procedures FastSort Syntax Summary { status := } SORTMERGESTART ( ctlblock { CALL } , key-block ,[ num-merge-files ] ,[ num-sort-files ] ,[ in-file-name ] ,[ in-file-exclusion-mode ] ,[ in-file-count ] ,[ in-file-record-length ] ,[ format ] ,[ out-file-name ] ,[ out-file-exclusion-mode ] ,[ out-file-type ] ,[ flags ] ,[ errnum ] ,[ errproc ] ,[ scratch-file-name ] ,[ scratch-block ] ,[ process-start ] ,[ max-record-length ] ,[ collate-sequence-table ] ,[ dslack ] ,[ islack ] ,[ flags2 ] ,[ subsor
B FastSort Error Messages This appendix lists the FastSort error messages in three lists: An alphabetic list of programmatic messages starting on B-1 A numeric list of programmatic messages starting on B-6 An alphabetic list of interactive messages (interactive messages are not numbered) starting on B-35 The numeric list of programmatic error messages and the alphabetic list of interactive error messages include the text for the error code, a possible cause, and recovery strategies.
Alphabetic List of Programmatic Messages FastSort Error Messages Error Code Message Text (page 2 of 6) 4 AN ERROR HAS PREVENTED CREATION OF THE SORT PROCESS. 39 AN INPUT RECORD EXCEEDED THE RECORD SIZE. 59 AN INPUT RECORD IS TOO SMALL. 81 BLOCKED INTERFACE NOT ALLOWED WITH MERGE. 133 CANNOT INCREASE THE SCRATCH FILE SIZE. 57 COLLATING SEQUENCE TABLE MUST BE PRESENT. 20 COMMUNICATIONS WITH SORTPROG HAVE BROKEN DOWN. 21 COMMUNICATIONS WITH SORTPROG WERE GARBLED.
Alphabetic List of Programmatic Messages FastSort Error Messages Error Code Message Text (page 3 of 6) 66 INVALID DATA SLACK SPECIFIED. 49 INVALID EXCLUSION MODE SPECIFIED. 72 INVALID EXTENDED SEGMENT SIZE. 51 INVALID FILE TYPE SPECIFIED FOR TO FILE. 12 INVALID FLAG OR COMBINATION OF FLAGS. 73 INVALID FORMAT OF THE PROCESS STRUCTURE. 89 INVALID FROM FILE RECORD SIZE. 139 INVALID FROM-FILE SPECIFIED TO RECGEN. 67 INVALID INDEX SLACK SPECIFIED. 65 INVALID KEY FOR KEY-SEQUENCED FILE.
Alphabetic List of Programmatic Messages FastSort Error Messages Error Code Message Text (page 4 of 6) 117 NULL KEY SPECIFIED FOR NON-SQL FILE. 118 NULLVAR KEY SPECIFIED FOR NON-SQL FILE. 125 NUMBER OF SORTPROG OPENERS EXCEEDED SPECIFIED LIMIT. 25 ONE OF THE KEY FIELDS IS OF AN UNDEFINED TYPE. 52 ONLY ONE FILE MAY BE SORTED VIA SORTMERGESEND. 116 ONLY ONE FILE CAN BE SORTED BY SORTMERGESQL. 79 PARAMETERS ARE MUTUALLY EXCLUSIVE. 126 PROCESS ALREADY OPEN AND SORTPROC_OPEN_ CALLED.
Alphabetic List of Programmatic Messages FastSort Error Messages Error Code Message Text (page 5 of 6) 132 SORTMERGESUPREC CALLED UNEXPECTEDLY. 10 SORTMERGEFINISH HAS BEEN CALLED UNEXPECTEDLY. 8 SORTMERGERECEIVE HAS BEEN CALLED UNEXPECTEDLY. 7 SORTMERGESEND HAS BEEN CALLED UNEXPECTEDLY. 111 SORTMERGESQL CALLED UNEXPECTEDLY. 58 SORTMERGESTART CALLED UNEXPECTEDLY. 47 SORTMERGESTATISTICS HAS BEEN CALLED UNEXPECTEDLY. 128 SORTPROC_SEND_ CALLED UNEXPECTEDLY.
Numeric List of Programmatic Messages FastSort Error Messages Error Code Message Text (page 6 of 6) 93 UNEXPECTED RETURN FROM LOADALTFILE PROCEDURE. 92 UNEXPECTED RETURN FROM LOADCLOSE PROCEDURE. 90 UNEXPECTED RETURN FROM LOADOPEN PROCEDURE. 91 UNEXPECTED RETURN FROM LOADWRITE PROCEDURE. 153 UPS NOT SUPPORTED IN THIS ENVIRONMENT. 154 UPS WORKSPACE BAD. 103 USER-SPECIFIED DEFINE IS NOT OF CLASS SORT.
Numeric List of Programmatic Messages FastSort Error Messages If you are using the programmatic interface, call the SORTERRORDETAILDETAIL or SORTERRORSUM procedure to display the error in a specialized 32-bit format. Read the format as follows: Parameter Word Bits 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 High-Order File System Error or PROCESS_CREATE_ Error Subcode Low-Order FastSort Input File Index FastSort Error Code Cause. The calling process and SORTPROG could not exchange messages.
Numeric List of Programmatic Messages FastSort Error Messages Recovery. Change the length parameter in the call to SORTMERGESEND. For more information on the description of length, see SORTMERGESEND Procedure on page 5-15. 10 SORTMERGEFINISH HAS BEEN CALLED UNEXPECTEDLY. Cause. The calling process called SORTMERGEFINISH at the wrong time. Recovery. Correct your program logic.
Numeric List of Programmatic Messages FastSort Error Messages Recovery. Follow recovery recommendations in the Guardian Procedure Errors and Messages Manual for the operating system error code returned with this FastSort error code. 21 COMMUNICATIONS WITH SORTPROG WERE GARBLED. Cause. Some condition in the system interfered with communications. Recovery.
Numeric List of Programmatic Messages FastSort Error Messages 26 A KEY FIELD LOCATION EXCEEDS THE RECORD SIZE. Cause. A key field does not lie entirely within the record. Recovery. Correct either the key-field offset or the record length. 27 THE TO FILE ALREADY EXISTS AND CANNOT BE PURGED. Cause. The specified output file exists but is too small or has a wrong type. SORTPROG cannot purge the file and create a new one because of the file's security, current usage, or some other condition in the system.
Numeric List of Programmatic Messages FastSort Error Messages Also check the recovery recommendations in the Guardian Procedure Errors and Messages Manual for the file-system error number returned with this FastSort error code. 31 A READ HAS FAILED FROM THE FROM FILE. Cause. SORTPROG could not read an input file. Recovery. Follow recovery recommendations in the Guardian Procedure Errors and Messages Manual for the operating system error code returned with this FastSort error code.
Numeric List of Programmatic Messages FastSort Error Messages recommendations in the Guardian Procedure Errors and Messages Manual for the operating system error code returned with this FastSort error code. 37 CREATION OF THE TO FILE HAS FAILED. Cause. FastSort could not create the output file. Recovery. Follow recovery recommendations in the Guardian Procedure Errors and Messages Manual for the operating system error code returned with this FastSort error code. 38 THE TO FILE COULD NOT BE OPENED.
Numeric List of Programmatic Messages FastSort Error Messages 44 INVALID CONTROL BLOCK, PROCEDURE CALL REJECTED. Cause. The calling process corrupted the FastSort control block. Recovery. Correct your program so that it does not overwrite the control block. 45 INVALID SCRATCH FILE BLOCK SIZE. Cause. The scratch file block size was incorrect in the call to SORTMERGESTART. Recovery. Correct the value in the scratch-block parameter.
Numeric List of Programmatic Messages FastSort Error Messages Recovery. Change the format of the output file or specify another file that is not an EDIT file. 51 INVALID FILE TYPE SPECIFIED FOR TO FILE. Cause. The output file type specified in the call to SORTMERGESTART was invalid. Recovery. Change the value of the out-file-type parameter. For the description of out-file-type, see SORTMERGESTART Procedure on page 5-19. 52 ONLY ONE FILE MAY BE SORTED VIA SORTMERGESEND. Cause.
Numeric List of Programmatic Messages FastSort Error Messages Recovery. Set flags.<10:10> to 0 or provide an alternate collating sequence table. For the description of the flags parameter, see SORTMERGESTART Procedure on page 5-19. 58 SORTMERGESTART CALLED UNEXPECTEDLY. Cause. The calling process called SORTMERGESTART at the wrong time. Recovery. Correct your program logic.
Numeric List of Programmatic Messages FastSort Error Messages Recovery. Specify a positive length for the key field in the key-block parameter. For the description of key-block, see SORTMERGESTART Procedure on page 5-19. 63 RESERVED FLAGS MAY NOT BE SET. Cause. The call to SORTMERGESTART or SORTBUILDPARM specified a flag value that you cannot set. Recovery. Set all unused flag bits to 0. For SORTMERGESTART, Table 5-6 shows flags bits you can use. The only flags2 bits you can use are <.4> and <.15>.
Numeric List of Programmatic Messages FastSort Error Messages Cause. The flags2 value in the call to SORTMERGESTART was not valid. Recovery. Correct the value of the flags2 parameter. For the description of flags2, seeunder the SORTMERGESTART Procedure on page 5-19. 69 INVALID NUMBER OF SUBSORT PROCESSES. Cause. The number of subsort processes specified in the call to SORTMERGESTART was outside the limits. Recovery.
Numeric List of Programmatic Messages FastSort Error Messages information about buffers and nowait I/O, see the description of the SORTBUILDPARM Procedure on page 5-2. 75 INVALID BLOCK LENGTH SPECIFIED. Cause. The block buffer length specified in the call to SORTBUILDPARM was outside the limits. Recovery. Correct the value of the buffer-length parameter. For the description of buffer-length, see SORTBUILDPARM Procedure on page 5-2. 76 START OF SUBSORT PROCESS HAS FAILED. Cause.
Numeric List of Programmatic Messages FastSort Error Messages Cause. The license PROMS were not purchased for your system. Recovery. Because of changes in the way unlicensed software is detected in C00 and later RVUs, you should not see this message. Contact your service provider. 81 BLOCKED INTERFACE NOT ALLOWED WITH MERGE. Cause. The call to SORTBUILDPARM specified a buffer for record blocking and the call to SORTMERGESTART specified input streams for merging through SORTMERGESEND. Recovery.
Numeric List of Programmatic Messages FastSort Error Messages Cause. A FROM file is an SQL object. Recovery. The FROM file cannot be an SQL object. Specify a file other then an SQL object for the FROM file. 88 INVALID OBJECT SPECIFIED AS SWAP FILE. Cause. A swap file is an SQL object. Recovery. Specify an Enscribe file as the swap file or use the default. 89 INVALID FROM FILE RECORD SIZE. Cause. The record size specified for an input file is greater than 4080. Recovery.
Numeric List of Programmatic Messages FastSort Error Messages Recovery. Report the internal error number returned with this FastSort error code to your service provider. The high-order word of the error-code parameter returned by SORTERRORSUM and SORTERRORDETAIL contains the internal error number. 99 DEFAULT DEFINE IS NOT OF CLASS SORT. Cause. A DEFINE with the reserved name “_SORT_DEFAULTS” was created, but is not of class SORT. Recovery. Delete the DEFINE and optionally recreate it as a SORT DEFINE.
Numeric List of Programmatic Messages FastSort Error Messages Recovery. Set DEFMODE to ON or determine why DEFMODE is not ON. 106 SUBSORT DEFINE IS NOT OF CLASS SUBSORT. Cause. The DEFINE class must be SUBSORT. Recovery. Specify CLASS SUBSORT in your SUBSORT DEFINEs. 107 ERROR OCCURRED WHILE ACCESSING A SUBSORT DEFINE. Cause. Unacceptable DEFINE name encountered, attribute missing, or error from procedure call. Recovery. Check that a valid DEFINE name was specified.
Numeric List of Programmatic Messages FastSort Error Messages Recovery. Generate and save a copy of your SAVEABEND file. Then contact your service provider. 112 ERROR FROM SQL FILESYSTEM VALIDATION ROUTINES. Cause. A programming error occurred on a call to an internal procedure. Effect. The SQL DDL or DML operation in progress terminates abnormally. Recovery. Generate and save a copy of your SAVEABEND file. Then contact your service provider. 113 INPUT FILE FOR SORTMERGESQL NOT TYPE SQL. Cause.
Numeric List of Programmatic Messages FastSort Error Messages Effect. The SQL DDL or DML operation in progress terminates abnormally. Recovery. Generate and save a copy of your SAVEABEND file. Then contact your service provider. 118 NULLVAR KEY SPECIFIED FOR NON-SQL FILE. Cause. A programming error occurred on a call to an internal procedure. Effect. The SQL DDL or DML operation in progress terminates abnormally. Recovery. Generate and save a copy of your SAVEABEND file.
Numeric List of Programmatic Messages FastSort Error Messages recommendations in the Guardian Procedure Errors and Messages Manual for the filesystem error code returned with this FastSort error code. If this strategy fails to resolve the problem, generate and save a copy of your SAVEABEND file. Then contact your service provider. 123 DATETIME CONVERSION FIELD NOT FOUND. Cause. A date-time field was specified as needing date-time conversion, but no datetime field was found. Effect.
Numeric List of Programmatic Messages FastSort Error Messages Recovery. Generate and save a copy of your SAVEABEND file. Then contact your service provider. 128 SORTPROC_SEND_ CALLED UNEXPECTEDLY. Cause. A programming error occurred on a call to an internal procedure. Effect. The SQL DDL or DML operation terminates abnormally. Recovery. Generate and save a copy of your SAVEABEND file. Then contact your service provider. 129 SORTPROC_CLOSE CALLED UNEXPECTEDLY. Cause. Stated in the error message.
Numeric List of Programmatic Messages FastSort Error Messages An increase of the maximum number of extents for the scratch file would cause the file to exceed its maximum limit. FastSort also returns file system error 21 (ILLEGAL count SPECIFIED). A file-system error (other than number 21) occurred when SORTPROG tried to increase the number of extents for the scratch file. Recovery. For the first error, create a partitioned scratch file large enough to hold all of the records.
Numeric List of Programmatic Messages FastSort Error Messages Recovery. Generate and save a copy of your SAVEABEND file. Then contact your service provider. 138 INVALID TO-FILE SPECIFIED TO RECGEN. Cause. Stated in the error message. Effect. The SQL DDL or DML operation in progress terminates abnormally. Recovery. Generate and save a copy of your SAVEABEND file. Then contact your service provider. 139 INVALID FROM-FILE SPECIFIED TO RECGEN. Cause. Stated in the error message. Effect.
Numeric List of Programmatic Messages FastSort Error Messages the problem, generate and save a copy of your SAVEABEND file. Then contact your service provider. 143 RECGEN ERROR READING BASE TABLE. Cause. The disk process or file system found an error in a base table record. Effect. The SQL DDL or DML operation in progress terminates abnormally. Recovery. Attempt to correct the condition described by the file-system error number.
Numeric List of Programmatic Messages FastSort Error Messages Recovery. Generate and save a copy of your SAVEABEND file. Then contact your service provider. 148 RECGEN SORTPROC_CLOSE ERROR. Cause. Internal error Effect. The SQL DDL or DML operation in progress terminates abnormally. Recovery. Generate and save a copy of your SAVEABEND file. Then contact your service provider. 149 RECGEN CALCULATES A BAD MULTIPLE MESSAGE ADDRESS. Cause. Stated in the error message. Effect.
Numeric List of Programmatic Messages FastSort Error Messages Effect. The SQL DDL or DML operation in progress terminates abnormally. Recovery. Check to see if any of the above conditions is true. If so, correct the condition. 154 UPS WORKSPACE BAD. Cause. The eye-catcher field in the UPS workspace is corrupted. Effect. The SQL DDL or DML operation in progress terminates abnormally. Recovery. Check that the UPS workspace is valid and not affected by the user program. 156 INVALID COLLATION ARRAY LENGTH.
FastSort Error Messages Numeric List of Programmatic Messages interface, the error parameter in the accompanying message identifies the error code. Follow the recovery recommendations in the Guardian Procedure Errors and Messages Manual for the error code. 163 PROCESS CREATE EXTENDED SWAP FILE ERROR. Cause. A PROCESS_CREATE_ error occurred on the extended swap file parameter. Recovery. For the programmatic interface, call the SORTERRORDETAIL or SORTERRORSUM procedure to determine the error code.
FastSort Error Messages Numeric List of Programmatic Messages interface, the error parameter in the accompanying message identifies the error code. Follow the recovery recommendations in the Guardian Procedure Errors and Messages Manual for the error code. 167 PROCESS CREATE LIBRARY FILE FORMAT ERROR. Cause. A PROCESS_CREATE_ error occurred because the program file has an invalid format. Recovery.
Numeric List of Programmatic Messages FastSort Error Messages Effect. None; the problem occurs at process termination time. Recovery. No recovery is necessary. However, you should report this error to your service provider. 172 SORTPROG VERSION TOO OLD; CANNOT SUPPORT REQUIRED NEW FEATURE. Cause. Your system’s version of SORTPROG is older than the FastSort system library procedures. Effect. The SQL DDL or DML operation in progress terminates abnormally. Recovery.
Alphabetic List of Interactive Messages FastSort Error Messages Recovery. Contact your system manager or service provider to have the correct version of FastSort or the operating system installed on your system. Alphabetic List of Interactive Messages This subsection describes the interactive FastSort messages in alphabetic order. This description includes the error message text, the probable cause of the error, and the recommended recovery.
Alphabetic List of Interactive Messages FastSort Error Messages This sort operation requires more than 32 scratch files, and an increase of the maximum number of extents for the last scratch file would cause the file to exceed 2 GB or 978 extents. FastSort also returns file system error 21 (ILLEGAL count SPECIFIED). A file-system error other than 21 occurred when SORTPROG tried to increase the number of extents for a scratch file. Recovery.
Alphabetic List of Interactive Messages FastSort Error Messages Recovery. Check that a valid DEFINE name was specified. FEATURE NOT SUPPORTED YET text. Cause. Information only. FILE NAME NOT SPECIFIED. Cause. You omitted the file name. Recovery. Specify the file name. FROM filename Cause. Information only. IGNORING UNUSABLE STRING OF LETTERS text. Cause. You mistyped the command or included text in the command that is not needed. Recovery.
Alphabetic List of Interactive Messages FastSort Error Messages Recovery. Check the file name to make sure it was correctly typed. See the Guardian Programmer's Guide for rules about specifying files. INVALID OR NON-EXISTENT USER-SPECIFIED DEFINE NAME. Cause. The DEFINE name you specified was not valid. Recovery. Specify a valid DEFINE name. INVALID SYNTAX text. Cause. You used incorrect syntax. Recovery. Check the syntax and correct the error. MORE THAN ONE SUBSORT SHOULD BE SPECIFIED. Cause.
Alphabetic List of Interactive Messages FastSort Error Messages Cause. Information only. NO TO STATEMENT HAS BEEN ISSUED. Cause. Information only. NOTCPUS cpu-list. Cause. Information only. ONLY ONE FROM STATEMENT MAY SPECIFY INPUT FROM THE COMMAND FILE. Cause. You issued too many FROM commands. Recovery. Issue the command FROM in-file command only once.
Alphabetic List of Interactive Messages FastSort Error Messages Cause. Information only. SUBSORT , PROGRAM program name. Cause. Information only. SUBSORT , SEGMENT segment. Cause. Information only. SUBSORT , SWAP swap-file-name. Cause. Information only. SUBSORT scratch-file-name, Cause. Information only. THE ALTERNATE COLLATING SEQUENCE SPECIFICATION IS INCORRECT. Cause. You specified the alternate collating sequence incorrectly. Recovery.
Alphabetic List of Interactive Messages FastSort Error Messages Recovery. Correct the command so that the number is less than 16. THE MEMORY SIZE IS NOT IN RANGE 1-64. Cause. You specified memory size not in the range from 1 to 64. Recovery. Omit the MEM parameter of the RUN command. THE PRIORITY IS NOT IN RANGE 1-199 Cause. You specified a priority not in the valid range. Recovery. Correct the PRI parameter of the RUN command so that the priority is within the range from 1 to 199.
Alphabetic List of Interactive Messages FastSort Error Messages Recovery. Clear the existing TO command or do not specify another one. THIS PROGRAM CAN ONLY HANDLE 63 KEYS. Cause. You specified too many keys in an ASCENDING or DESCENDING command. Recovery. Retype the command and specify 63 or fewer keys. TRUNCATING OUTPUT RECORD LENGTH TO DEVICE WIDTH. Cause. The output file record size is too small to hold an output record. Recovery.
C Using Supported File Types For input files, FastSort accepts records from unstructured, relative, entry-sequenced, key-sequenced, EDIT, and partitioned files. FastSort does not accept input records from the following: Blocked tape files Key-sequenced files with increased limits You might use buffered interface to send records from key-sequenced files with increased limits to FastSort. You do not specify the type for input files.
Unstructured Files Using Supported File Types Table C-1. Summary of Output File Types Output File Type TO Command TYPE Parameter SORTMERGESTART Procedure output-file Parameter Key-sequenced K 3 EDIT Use EDIT to create the file and copy data from another file type. – Tape Use FUP to load the file. – For information about the different types of files, see the Guardian Programmer’s Guide and the Enscribe Programmer’s Guide.
Relative Files Using Supported File Types Relative Files You can use relative files as input or output files. For a relative output file, specify either of the following: R in the TYPE parameter of the TO command 1 in the out-file-type parameter of the SORTMERGESTART procedure If the type of your first input file is relative, the default output file type is relative. Entry-Sequenced Files You can use entry-sequenced files as input and output files.
EDIT Files Using Supported File Types The DSLACK and ISLACK parameters of the RUN command, if you want the data blocks to have a different percentage of slack space than the index blocks The dslack and islack parameters of the SORTMERGESTART procedure The default for SLACK, dslack, and islack is 0 percent. The default for DSLACK and ISLACK is the value of SLACK. FastSort currently does not load alternate-key files directly. You can use FUP to load alternate-key files.
Tape Files Using Supported File Types Tape Files If you want to use input records from a blocked tape file, use FUP to deblock the records by loading them into a disk file. Then specify the disk file as an input file for the sort or merge run. If you want to store output records in a blocked tape file, use FUP to block the records by loading them to the tape file from a disk file. Then specify the disk file as the output file for the sort or merge run.
Partitioned Output Files Using Supported File Types FastSort Manual—429834-003 C-6
D ASCII Character Set Table D-1.
ASCII Character Set Table D-1.
ASCII Character Set Table D-1.
ASCII Character Set Table D-1.
E FastSort Limits This appendix summarizes parameter values that FastSort accepts in commands and procedure calls. Table E-1.
FastSort Limits FastSort Manual—429834-003 E-2
Glossary alternate collating sequence. An EDIT file instructs FastSort to collate sort results by specific alphanumeric key fields or string type data. The default FastSort collating sequence is the ASCII character set. application . One or more processes that achieve a specific objective. Processes in an application often communicate with each other using the message system and file system. See also program and process. command file. An EDIT file containing a sequence of commands to execute.
program Glossary program. A static set of instruction codes and initialized data, such as compiler output or the Binder program, that is not currently executing. A program usually resides in a program file on disk. scratch file. A temporary work file for FastSort. When a sort-merge operation cannot be performed in memory, SORTPROG temporarily stores partial information in one or more scratch files. serial sort-merge operation. A FastSort operation that uses one SORTPROG process to sort or merge records.
Index A Abbreviated and equivalent character assignments 3-7 ALSO keyword, alternate collating sequence 3-7 ALTER DEFINE TACL command 7-9 Alternate collating sequence assigning characters to 3-7 COLLATE command 3-6 COLLATEOUT command 3-9 defining in SORTMERGESTART 5-28 equating characters 3-7 examples 3-8 invalid files 3-8 reading from EDIT file 3-6 reading from unstructured file 3-9 SORTMERGESTART 3-10 storing in table 3-9 Application process communicating with SORTPROG process 4-1 FastSort procedures 4-1
C Index Characters, alternate collating sequence 3-7 CLEAR command examples 3-6 syntax 3-5 COBOL85 parallel sort example 6-20 SORT or MERGE statement 5-19 sorting records 4-13 COLLATE command assigning characters 3-7 examples 3-7 syntax 3-6 COLLATEOUT command 3-10 Collating sequence, alternate See Alternate collating sequence Command file comments 2-2 DEFINEs 9-16 interactive sort 2-1 interactive sort example 2-3 nested levels in SQLCI 9-16 specifying SORT DEFINE 7-10 Command parameters deleting 3-5 displ
D Index CREATE command (FUP) 1-9 CREATE INDEX statement, SQL/MP and FastSort 1-10 CONFIG option 8-15 configuration file 8-10 configuring 8-8 default configuration 8-9/8-10 description 1-10 loading multiple indexes in parallel 8-15 CROSSREF program, and FastSort 1-10 Ctrl-Y keys 2-1 D Data blocks 3-30 Data slack, SORTMERGESTART 5-28 Data types ASCENDING command 3-2 DESCENDING command 3-11 DATAPAGES compiler directive 5-2 Deblocking records to reduce interprocess messages 5-15 Default values for SORT DEFIN
F Index Error codes and messages displaying B-1 format 2-14 interactive B-35 parallel sorting 6-14 programmatic, alphabetic list B-1 programmatic, numeric list B-6 SORTERROR 4-3, 5-9 SORTERRORDETAIL 4-3, 5-9 SORTERRORSUM 5-10 Error procedure, use with SORTMERGESTART 5-36 EXCLUSION parameter, TO command 3-28 Execution priority distributor-collector process 6-13 subsort processes 6-10 Existing output files 3-31 EXIT command 3-13 Exiting an interactive process 2-1 Extended memory segment buffer addresses 5-8
G Index G GROUP BY clause, SQL/MP 8-4 H HELP command 3-18 I Identifying errors in parallel sorting 6-14 Index blocks, specifying for key-sequenced output files 3-30 Index slack, SORTMERGESTART 5-28 Index, SQL/MP 1-10, 8-8 INFO DEFINE TACL command 7-8 Initial scratch file See Scratch file, initial Input files EDIT C-4 entry-sequenced C-2 example 2-4, 2-6 FastSort features 1-3 file access 5-21 file types 1-8 FROM command 3-14 key-sequenced C-3 number allowed 1-8 number of records allowed 3-15 partitioned
L Index Key fields (continued) specifying in ASCENDING command 3-2, 3-4 Key sort 2-9 KEYS parameter, TO command 3-29 Key-block, key-field definitions 5-29 Key-sequenced files data slack C-3 description of C-3 index slack C-3 specifying output file slack space 3-30 L Limits, sorting E-1 List file specifying in TACL RUN command 2-1 SPOOL DEFINE 1-9 LOAD command, SQL/MP configuring 8-15 description 8-7 loading large tables 8-17 MAX option 8-16 protecting table auditing 8-15 SCRATCH option 8-16 SORTED option
O Index O Online help, HELP command 3-18 ORDER BY clause, SQL/MP 1-10, 8-4 OSIMAGE file 1-4 Output files EDIT C-4 entry-sequenced C-2 estimating size 4-8 EXCLUSION mode, TO command 3-28 existing 3-31 file types, SORTMERGESTART 1-9, 5-24 key-sequenced 3-30, C-3 options summary 1-3 partitioned C-5 relative C-2 requirements for existing 3-31 slack space for key-sequenced 3-30 specifying file access with SORTMERGESTART 5-23 summary of types C-1 tape C-4 TO command 3-28, 3-29 unstructured C-2 using FUP to load
Q Index PERMUTATION parameter, TO command 3-29 Permutation sort, example 2-10 PRI attribute SORT DEFINE 7-3 SUBSORT DEFINE 7-6 PRI parameter RUN command 3-21 SUBSORT command 3-27 Priority RUN command 3-21 SUBSORT command 3-27 SUBSORT DEFINE 7-6 Procedures description 5-1 parallel sorting 6-3 SORTBUILDPARM 5-2 SORTERROR 5-9 SORTERRORSUM 5-10 SORTMERGEFINISH 5-12 SORTMERGERECEIVE 5-13 SORTMERGESEND 5-15 SORTMERGESTART 5-19 SORTMERGESTATISTICS 5-37 syntax summary A-1 user-written error procedure 5-36 Procedu
S Index REMOVEDUPS parameter, RUN command 3-21 Removing duplicate records RUN command 3-21 SORTMERGESTART 5-31 RESET DEFINE TACL command 7-10 Restart option SORTBUILDPARM 4-3 SORTMERGESTART 5-4, 5-24, 5-33, 5-34 Returning errors SORTERRORDETAIL 5-9 SORTERRORSUM 5-10 RUN command AUTOMATIC parameter 3-20 BLOCK parameter 3-20, 6-11 CPU parameter 3-20 DEFINE parameter 3-22 description of 3-19 examples 3-23 improving performance in parallel sorting 6-11 limiting elapsed time 3-20 MEM parameter 3-21 MINSPACE pa
S Index SEGMENT attribute SORT DEFINE 7-4 SUBSORT DEFINE 7-6 SEGMENT parameter RUN command 3-22 SUBSORT command 3-27 Segment, extended memory See Extended Memory Segment SELECT statement, SQL/MP 1-10 Selecting processors 6-8 Serial sort operation COBOL85 example 4-14 TAL example 4-15 SET DEFINE TACL command 7-8 SET DEFMODE TACL command 7-7 SET EXTENDSTACK, BINDER command 5-2 Setting up subsort processes 3-26 SHOW command examples 3-26 syntax 3-25 SLACK parameter, TO command 3-30 SORT DEFINE MODE attribute
S Index SORTMERGESTART procedure automatic configuration 6-5 configuring subsort processes 6-6 data slack 5-28 defining key fields 5-20 example 5-35 flags parameter 4-9, 5-31 index slack 5-28 initial scratch file 5-25 input file access 5-21 output file access 5-23 output file type 5-24 output record format 5-22 parallel sorting 6-3, 6-11 priority word in process-start parameter 6-10, 6-13 process-start parameter 6-9 returning records to an application process 4-6 scratch-block parameter 6-11 segment word
T Index Statistics displaying 2-13 parallel sort 2-14, 6-14 procedures to obtain 4-3 SORTMERGESTATISTICS procedure 5-37 SORTMERGESTATISTICS statistics structure 5-38 VLM 9-12 Stopping a sort or merge 4-2 SUBSORT attribute, SORT DEFINE 7-5 SUBSORT command BLOCK parameter 3-26 CPU parameter 3-27 initial scratch file 3-26 MEM parameter 3-27 parallel sorting 6-5 PRI parameter 3-27, 6-10 PROGRAM parameter 3-27, 6-10 SEGMENT parameter 3-27, 6-9, 6-12 SWAP parameter 3-27, 6-9 syntax, examples 3-26, 3-27 SUBSORT
U Index TACL DEFINEs See DEFINEs, TACL TAL parallel sort example 6-23 serial sort example 4-15 Tape files C-4 THRU keyword, alternate collating sequence 3-7 To 6-5 TO command examples 3-32 EXCLUSION parameter 3-28 KEYS parameter 3-29 NOPURGE parameter 3-30 PERMUTATION parameter 3-29 SLACK parameter 3-30 syntax 3-28 TYPE parameter 3-29 U Unblocked records, sending and receiving 5-7 UNION clause, SQL/MP 8-5 Unmirrored disks, in parallel sorting 6-13 Unstructured file description C-2 storing alternate colla
Special Characters Index FastSort Manual—429834-003 Index-14
