HP Code Profiling Utilities Manual Abstract This manual describes the Code Profiling Utilities for HP Integrity NonStop NS-Series servers.
Document History Part Number Product Version Published 542684-001 Code Coverage Tool H06 August 2006 542684-002 Code Profiling Utilities H06 February 2007 542684-003 Code Profiling Utilities H06 August 2010
HP Code Profiling Utilities Manual Glossary Index Figures What’s New in This Manual v Manual Information v New and Changed Information About This Manual vii Notation Conventions Tables v vii 1. Introduction to the Code Profiling Utilities Features of the Utilities 1-1 Required Hardware and Software 1-2 Usage Overview 1-2 Profile-Guided Optimization 1-2 Code Coverage 1-5 Usage Scenarios 1-7 Topics This Manual Covers 1-7 2.
3. Building the Application (continued) Contents 3. Building the Application (continued) Example 3-6 4. Running the Application Task Overview 4-1 Prepare the Application for Testing 4-1 Preparing Test Cases 4-1 Insulating the Application 4-2 Run the Application 4-2 Verify Output 4-2 Example 4-3 5.
8. Interpreting the Code Coverage Report Contents Prepare to Run codecov (continued) Cleaning Up from Previous Runs 7-4 Run codecov 7-4 Verify Output 7-6 The Code Coverage Report 7-6 codecov Use of Standard Error and Output Files Example 7-7 7-7 8.
Figures Contents Figures Figure 1-1. Figure 1-2. Figure 3-1. Figure 4-1. Figure 5-1. Figure 7-1. Figure 8-1. Figure 8-2. Figure 8-3. Figure 9-1. Figure 9-2. Figure 9-3.
What’s New in This Manual Manual Information HP Code Profiling Utilities Manual Abstract This manual describes the Code Profiling Utilities for HP Integrity NonStop NS-Series servers.
New in the February 2007 revision: What’s New in This Manual New in the February 2007 revision: • • • Added information and modifications throughout the manual to describe the new profile-guided optimization capability of the Code Profiling Utilities, including a new section, Section 6, Optimizing a Program Using Profile-Guided Optimization. Retitled the manual to a more inclusive title to reflect the utilities’ new capabilities.
About This Manual Notation Conventions Hypertext Links Blue underline is used to indicate a hypertext link within text. By clicking a passage of text with a blue underline, you are taken to the location described. For example: This requirement is described under Backup DAM Volumes and Physical Disk Drives on page 3-2. General Syntax Notation This list summarizes the notation conventions for syntax presentation in this manual. UPPERCASE LETTERS.
General Syntax Notation About This Manual each side of the list, or horizontally, enclosed in a pair of brackets and separated by vertical lines. For example: FC [ num ] [ -num ] [ text ] K [ X | D ] address { } Braces. A group of items enclosed in braces is a list from which you are required to choose one item. The items in the list can be arranged either vertically, with aligned braces on each side of the list, or horizontally, enclosed in a pair of braces and separated by vertical lines.
Notation for Messages About This Manual Line Spacing. If the syntax of a command is too long to fit on a single line, each continuation line is indented three spaces and is separated from the preceding line by a blank line. This spacing distinguishes items in a continuation line from items in a vertical list of selections. For example: ALTER [ / OUT file-spec / ] LINE [ , attribute-spec ]… !i and !o.
Change Bar Notation About This Manual lowercase italic letters. Lowercase italic letters indicate variable items whose values are displayed or returned. For example: p-register process-name [ ] Brackets. Brackets enclose items that are sometimes, but not always, displayed. For example: Event number = number [ Subject = first-subject-value ] A group of items enclosed in brackets is a list of all possible items that can be displayed, of which one or none might actually be displayed.
HP Encourages Your Comments About This Manual The message types specified in the REPORT clause are different in the COBOL85 environment and the Common Run-Time Environment (CRE). The CRE has many new message types and some new message type codes for old message types. In the CRE, the message type SYSTEM includes all messages except LOGICAL-CLOSE and LOGICAL-OPEN. HP Encourages Your Comments HP encourages your comments concerning this document.
About This Manual HP Encourages Your Comments HP Code Profiling Utilities Manual—542684-003 xii
1 Introduction to the Code Profiling Utilities This manual describes the Code Profiling Utilities. It addresses application developers who create application components for NonStop servers, and who want to do one or both of the following: • • Optimize their applications using a technique called profile-guided optimization. Profile-guided optimization is supported for C/C++ and pTAL applications.
Introduction to the Code Profiling Utilities Required Hardware and Software Code Cover Utility (codecov) is used to generate the code coverage report; it runs only on the Windows platform.
Introduction to the Code Profiling Utilities Profile-Guided Optimization 2. Run the instrumented application on the NonStop server. When executed, the instrumented application creates a raw data file that contains information about the execution path of the application. You can run the instrumented application multiple times, typically using different input data, representing different workloads, for each run.
Profile-Guided Optimization Introduction to the Code Profiling Utilities Figure 1-1. Profile-Guided Optimization Task Overview Source Files 1. Compile the application. Compiler (pTAL, C/C++) Instrumented Object Files 2. Run the application. Guardian, OSS Raw Data Files 3. Merge raw data files. profmrg Dynamic Profiling Information (DPI) File 4. Recompile the application. Compiler (pTAL, C/C++) Optimized Object File vst030.
Introduction to the Code Profiling Utilities Code Coverage Code Coverage To measure code coverage for an application: 1. Compile and link application components on the NonStop server or a workstation. For parts of the application that you wish to instrument for code coverage, use compiler options described in Section 3, Building the Application. The compiler generates an instrumented binary file and a static profiling information (SPI) file. 2. Run the instrumented application on the NonStop server.
Introduction to the Code Profiling Utilities Code Coverage Figure 1-2. Code Coverage Task Overview Source Files 1. Compile the application. Compiler (pTAL, COBOL, C/C++) Static Profiling Information (SPI) Files Instrumented Object Files 2. Run the application. 4a. Concatenate SPI files. Guardian, OSS Platform Text Editor Raw Data Files 3. Merge raw data. profmrg Dynamic Profiling Information (DPI) File 4. Generate code coverage report.
Introduction to the Code Profiling Utilities Usage Scenarios Usage Scenarios Code coverage analysis and profile-guided optimization have different usage scenarios. With code coverage, you are usually trying to determine what portions of your code are being executed during testing; the goal is to execute as much of the application as possible.
Introduction to the Code Profiling Utilities HP Code Profiling Utilities Manual—542684-003 1 -8 Topics This Manual Covers
2 Installing the Code Profiling Utilities The Code Profiling Utilities consist of two products: • • T0746: Code Cover Utility (codecov) T0747: Profile Merge Utility (profmrg) Support for code profiling instrumentation is implemented in the following compiler products: • • • • T0356: COBOL compiler (ECOBOL) (code coverage only; the COBOL compiler does not support profile-guided optimization) T0549: TNS/E C/C++ compiler - for Guardian, Windows, and OSS (CCOMP, CPPCOMP) T8164: Standard C/C++ compiler util
Installing the Code Profiling Utilities Installing codecov Installing codecov 1. After installing the H06.07 or later RVU, locate the file $SYSTEM.ZCODECOV.T0746SET. 2. Copy T0746SET to your workstation. 3. Change the name of T0746SET to setup.exe. 4. Run setup.exe. A file named Hewlett-Packard\CodeCoverage\codecov.exe is created within the program files folder. codecov.exe is the executable codecov program. Note. You might wish to put the location of codecov.exe in your PATH variable.
3 Building the Application Task Overview The first step in performing profile-guided optimization or measuring code coverage is to generate instrumented object files for all parts of the application for which you want code profiling information. To do this, you must compile and link the source files, specifying compiler and possibly linker options to support code profiling.
Building the Application Understanding Code Coverage Concepts Understanding Code Coverage Concepts Code coverage reports are based on a model in which the application consists of source files, which contain functions, which contain basic blocks. A basic block is a sequence of code that is entered only at the beginning and exited at the end. Once code enters a basic block, it executes the entire block unless an exception is raised within the block.
Building the Application Build the Application To create a new SPI file instead of adding to an existing one, you can: • • Move, rename, or delete the old file. Compile the application in a different directory or subvolume.
Building the Application Compiling the Source Files If you do not specify the basename option, then the source files used with profuse compilations must be in the same locations (that is, they must have the same pathnames) as when the same files were compiled with the profgen option. For more information about using the basename option, see Section 6, Optimizing a Program Using Profile-Guided Optimization.
Building the Application Verify the Output When code that was generated from separate compilations will later be included in a single application, you should specify the same raw data file location for all the compilations. If you are concurrently compiling multiple applications, you can use the profdir option to cause the raw data files for each application to be written to a separate directory or subvolume.
Building the Application Linking the Object Files Linking the Object Files To link instrumented object files, you must specify the option -l pgo if invoking the linker directly. If the linker is invoked through the C/C++ or COBOL compiler and you have specified one of the options to create an instrumented object file, the compiler automatically inserts the -l pgo option.
4 Running the Application Task Overview In this step, you run the application with test data or a representative workload. A program or DLL containing instrumented code generates a distinct raw data file each time it completes execution. This raw data file is the record of what functions and basic blocks were executed during the test run. If an application is built by linking many source files and DLLs, the run generates one raw data file that contains code coverage data from all instrumented components.
Running the Application Insulating the Application Insulating the Application By default, the raw data file is placed in the default subvolume (or current working directory) when the program or DLL is initiated. To keep the raw data files for different runs and different applications separate: • • • • Use the profdir option when compiling your source files to specify different subvolumes or directories for the raw data files.
Running the Application Example Example See Run the Application on page 9-10.
Running the Application Example HP Code Profiling Utilities Manual—542684-003 4 -4
5 Converting Raw Data Files to DPI Files Task Overview In this step, you combine the raw data files from application runs to produce a dynamic profiling information (DPI) file for input to a compiler or the codecov utility. If you have previously generated DPI files for the same application, you can combine the DPI files from the previous runs with raw data files from the latest run to produce a new DPI file.
Converting Raw Data Files to DPI Files Prepare profmrg Input Files Prepare profmrg Input Files The profmrg utility requires the following inputs: • • Zero or more raw data files, generated by runs of instrumented code Zero or more DPI files, created by previous runs of profmrg The output of profmrg is a DPI file that combines the information from all the input files.
Converting Raw Data Files to DPI Files Saving DPI Files from Previous Runs must first use FTP or some other mechanism to copy the files from the NonStop server to the workstation where profmrg will run. Saving DPI Files from Previous Runs If you want to save the file generated by a previous version of profmrg, to prevent the tool from replacing the existing file, you can: • • Move the existing file to a different location, or rename it. (The default name for the DPI file is pgopti.
Converting Raw Data Files to DPI Files Run profmrg To exclude a file from processing, rename it or remove it from the directory or subvolume profmrg will use. Run profmrg To run profmrg, use a command line of the form: profmrg options where options can be any set of options from the following list. Options must be in lowercase. Note. A filename specified in a profmrg command option must not begin with a slash(/). -a dpi_file_list specifies the DPI files to use as input.
Converting Raw Data Files to DPI Files Verify Output -prof_dir directory-or-subvolume specifies the name of a directory or subvolume relative to which profmrg looks for its input files and creates its output file. By default, profmrg uses the current working directory (Windows, OSS) or the default subvolume (Guardian). Filenames specified in the -prof_dpi and -a options are concatenated with the directory or subvolume name specified in this option.
Converting Raw Data Files to DPI Files Example is the last token on the command line, profmrg displays an error message and a brief syntax message. Following any such messages is a banner consisting of one line that displays the name and VPROC of the tool, and a second line stating the copyright. For example: profmrg - T0747 Copyright 2006 Hewlett-Packard Company.
6 Optimizing a Program Using ProfileGuided Optimization To optimize your application using profile-guided optimization, you simply recompile the application components, specifying, as input to the compiler, the DPI file described in Section 5, Converting Raw Data Files to DPI Files, and the source programs you want to optimize. The compiler interprets information in the DPI file to create an optimized object file. Note. Profile-guided optimization is supported for C/C++ and pTAL programs only.
Optimizing a Program Using Profile-Guided Optimization Location of Source Files Location of Source Files The DPI files input to the compilers contain the names of the source files that were compiled with profgen. By default, these are fully qualified source file names, following naming conventions for the respective host environments (Guardian, OSS, or Windows).
Optimizing a Program Using Profile-Guided Optimization Additional Usage Considerations Then the raw data file generated by subsequent execution of myprog.o will contain only the name myprog.c. If -Wbasename were not specified, the full path name /homedir/subdir/myprog.c would be included in the raw data file. Note. If you move a source file between an OSS directory and a Windows directory or Guardian subvolume, then all characters in the OSS file name must be uppercase.
Optimizing a Program Using Profile-Guided Optimization Additional Usage Considerations HP Code Profiling Utilities Manual—542684-003 6 -4
7 Running the Code Cover Utility Task Overview In this step, you run the Code Cover Utility (codecov), making available as input: • • • The source files you compiled, as described in Section 3, Building the Application The SPI file you created, as described in Section 3, Building the Application The DPI file you created, as described in Section 5, Converting Raw Data Files to DPI Files The Code Cover Utility generates the code coverage report in a set of HTML files you can view with any standard browse
Running the Code Cover Utility Prepare to Run codecov Prepare to Run codecov The meaningfulness of the code coverage report depends on the consistency of the SPI file, the DPI file, and the source files provided as input. Specifically, for the results of a study to be meaningful, all inputs should reflect the same versions of the source files. The codecov utility performs no validation in this regard.
Running the Code Cover Utility Preparing the SPI File Preparing the SPI File The codecov utility requires a single SPI file. The default name for the SPI file depends on where the application was compiled: • • • • pgopti.spi if the program was compiled on Windows pgopti.
Running the Code Cover Utility Cleaning Up from Previous Runs DPI file but not in the SPI file, codecov ignores the profile information about that file, because the report includes information only for source files listed in the SPI file. These assumptions make it necessary for corresponding source file names in the SPI files and DPI files to match.
Running the Code Cover Utility Run codecov -fcolor color specifies the name or hexadecimal code of the HTML color used in reports to show functions that were uncovered (never called). The default is #ffcccc, which is pink. -h or -help causes codecov to stop processing the command line, print out a syntax description of all options that it supports, and terminate. -host string provides a host address to use for access to a NonStop server.
Running the Code Cover Utility Verify Output contains ten or more functions, codecov prints the percentage each time it completes analysis of one-tenth of the functions, so the progress meter is updated at most ten times. Progress messages appear on the same line unless interrupted by other messages, such as warning messages about source files that codecov cannot find. -passwd string specifies the password for the user name given in the -login option.
Running the Code Cover Utility codecov Use of Standard Error and Output Files also have uppercase names with the extension .HTML. Any existing files of the same names are overwritten. Among the HTML files that codecov creates are some that correspond to individual source files. WARNING. The relation between source file names and HTML file names is not guaranteed to be the same in future versions of this product. Hyperlinks among the files within the CodeCoverage folder use simple pathnames.
Running the Code Cover Utility Example HP Code Profiling Utilities Manual—542684-003 7 -8
8 Interpreting the Code Coverage Report Opening the Code Coverage Report To examine the code coverage report, use a browser to open the file named CODE_COVERAGE.HTML. It displays a top-level summary of the code coverage for all relevant source files, as in the following example: Figure 8-1. Coverage Summary The coverage summary indicates how many files, functions, and basic blocks were covered or uncovered. The number of files is the number that the report covers.
Interpreting the Code Coverage Report Execution Counts in the Source Display Figure 8-2. Coverage Display for a Source File The left half lists the functions that were completely uncovered and the functions that were at least partially covered. For each function that was at least partially covered, the display tells how many basic blocks were or were not covered. In these lists, the names of C++ functions are mangled.
Interpreting the Code Coverage Report Representation of #include Files appears under the code directly beneath the source position where the block begins. The count is immediately preceded by a circumflex (^). If more than one basic block is generated for the code at a source position, the number of generated basic blocks and the number of executed basic blocks follow the execution count.
Interpreting the Code Coverage Report Understanding Color Coding in the Code Coverage Report Understanding Color Coding in the Code Coverage Report Table 8-1 summarizes the colors used in the code coverage report. Table 8-1. Color Coding in the Code Coverage Report Default Color Meaning yellow Indicates uncovered basic blocks in a function for which some basic blocks were covered and others were not. white (no color) Indicates basic blocks that were covered.
Interpreting the Code Coverage Report Understanding Color Coding in the Code Coverage Report As an example of how color coding can be misleading, consider the following code sequence. The example shows the color coding that would appear in the report. void a() { return; c(); } In this example, procedure a is called and has an unconditional return in the second line. Thus, the two lines following the return are uncovered, and you would expect them to be colored yellow.
Interpreting the Code Coverage Report Understanding Color Coding in the Code Coverage Report The count, specifically, the caret (^), appears directly under the point where codecov “thinks” the block begins. For example: int gethostsex (void) { ^ 2 This procedure was executed, so it should have been uncolored. However, the int at the beginning of the line is shown as partially covered (light brown).
Interpreting the Code Coverage Report Understanding Color Coding in the Code Coverage Report For another example, consider the following partial codecov output: if (n==1)||(n++2) ^10(1/2) In this example, the notation (1/2) indicates that two blocks were generated for the if statement, and only one of them was covered; the covered block was executed 10 times, and the other block was not executed.
Interpreting the Code Coverage Report Understanding Color Coding in the Code Coverage Report HP Code Profiling Utilities Manual—542684-003 8 -8
9 Code Profiling Usage Scenarios This section provides two examples of code profiling: one for optimizing an application using profile-guided optimization and one for the creation and analysis of a code coverage report. These examples include all the steps discussed in the preceding sections. Profile-Guided Optimization Example Profile-guided optimization provides a way of improving the performance of an application.
Code Profiling Usage Scenarios Profile-Guided Optimization Example Figure 9-1. Traditional Method of Optimization Write program Compile at optimization level 0 and test Does program contain errors? Debug program Yes No Recompile at optimization level 1 and retest Does program contain errors? Debug program Yes No Recompile at optimization level 2 and retest Measure the program's performance No Does program contain errors? Debug program Yes vst032.
Code Profiling Usage Scenarios Profile-Guided Optimization Example compiler to generate more efficient code (for example, by using the NO_EXCEPTIONS compiler option). Profile-guided optimization is typically used as an additional step in the process shown in the preceding figure. Once the program has compiled and run at optimization level 2, follow the steps shown in Figure 9-2 to use profile-guided optimization to produce even more efficient object code. Figure 9-2.
Code Profiling Usage Scenarios Run the Application to Obtain a Baseline (Unoptimized) Measurement These steps are summarized here: Note. As a preliminary step, you might want to run your application and measure its performance to provide a baseline for comparison with its performance after profile-guided optimization. 1. After compiling and running successfully at optimization level 2, recompile the program using the profgen option to create an instrumented object file. 2.
Code Profiling Usage Scenarios Build the Instrumented Application Build the Instrumented Application Following is the output of a files command, showing the contents of a subvolume that so far contains only the source code (which have names ending in C) and required header files (which have names ending in H) for the elfdump application. $DATA05.TEST 2> files $DATA05.
Code Profiling Usage Scenarios Generate the DPI File In this step, the elfdump executable that has just been built is run exactly as before (two times, each with a different input). The following display shows the subvolume after the two runs (the output files OUTPUT1 and OUTPUT2 have been purged). $DATA05.TEST 6> files $DATA05.
Code Profiling Usage Scenarios Run the Optimized Application In this build step, note that the profuse compiler option is specified and the -l pgo linker option is not specified. Since the profuse option does not specify a file path, the compiler looks for the DPI file in the current subvolume. The contents of the subvolume are the same as before, except that the output files from the compile and link steps have been replaced by newer versions.
Code Profiling Usage Scenarios Build the Application Figure 9-3. Strategic Use of Code Coverage Technology Run Test Cases Evaluate Code Coverage Run Customer Use Cases Compare Code Coverage Enhance Test Cases vst011.vsd Build the Application The application in this example is called elfdump. It is a program that dumps information about object files on a NonStop server. 1. The following display shows the build directory for the elfdump application.
Code Profiling Usage Scenarios Build the Application header files (which have names ending in H). The application will be built and executed in the Guardian environment. $DATA05.TEST 2> files $DATA05.TEST ELFCOMH ELFDUMPC NSKELFH PLATDEPC PLATDEPH YOSELFC YOSELFH 2. The following command lines invoke the compiler and linker with options related to code coverage. The codecov option causes the compiler to create an instrumented object file. The generated SPI file will have the name PGOSPI.
Code Profiling Usage Scenarios Run the Application Run the Application 1. The following command lines run the application twice, each time with different input. (In each case, $system.system.eld is the input file, but the -h and -p options cause the runs to vary.) elfdump /out output1/ -h $system.system.eld elfdump /out output2/ -p $system.system.eld 2. The following display shows the working directory, with the new raw data files (which have names starting ZZPF).
Code Profiling Usage Scenarios Measure Code Coverage Measure Code Coverage 1. The following display shows the directory containing the input files for codecov. $ ls pgopti.dpi pgopti.spi zzpfo9uj zzpfu7 2. This command runs the codecov tool, specifying the host address and login information required to retrieve the elfdump source files from the Guardian file system. The prompt for a password would occur in practice but is omitted from this example.
Code Profiling Usage Scenarios Evaluate the Code Coverage Report 2.
Code Profiling Usage Scenarios Evaluate the Code Coverage Report 3. Scrolling down the right side of the detailed display, you see the source code, with color coding to show what was covered and what was not. The following example shows a portion of the detailed display: Note: • • • • Most of the lines are uncolored, meaning that they were executed. The “2” below line 111 indicates that the procedure was called twice (once for each execution of elfdump).
Code Profiling Usage Scenarios Evaluate the Code Coverage Report HP Code Profiling Utilities Manual—542684-003 9- 14
10 Usage Considerations This section summarizes usage considerations for the Code Profiling Utilities. Most of these issues are discussed in other sections of this manual but are repeated here for your convenience. Compilation Issues To support profile-guided optimization or code coverage analysis, you must recompile existing program files or DLLs with the appropriate compiler and linker options, described in Section 3, Building the Application.
Usage Considerations Application Performance Application Performance Code profiling is intended for data generation and collection in a test environment only. The use of instrumented code is not recommended for production environments. Applications compiled with the code profiling instrumentation will run much more slowly than noninstrumented code.
Glossary basic block. A sequence of code that is entered only at the beginning and exited at the end. Once code enters a block, it executes the entire block unless an exception is raised within the block. Although the terms “basic block” and “block” are often used interchangeably, “block” is more generic and usually refers to any block (or sequence) of instructions. code coverage. Information about which parts of the source code of a program file were executed during runs of the program file.
Glossary uncovered. uncovered. Never executed during a test run.
Index B basename compiler option 3-3, 6-2 Block defined Glossary-1 introduced 3-2 C Code Cover Utility installing 2-1 running 7-1 see also codecov Code coverage concepts 3-2 defined Glossary-1 report 1-1 display for individual source file 8-1 execution counts in 8-2 filenames 7-6 interpreting 8-1 title displayed on 7-6 Code coverage overview 1-5 Code Profiling Utilities hardware and software requirements 1-2 installing 2-1 overview 1-1 codecov command syntax 7-4 codecov compiler option 3-4 codecov options
Index E DLL dynamic unloading not supported 10-1 DPI file created by profmrg 5-5 generating 5-1 introduced 1-3, 1-5 list as input to profmrg 5-4 specifying name to codecov 7-4 specifying name to compiler 6-1 Dynamic profiling information (DPI) file see DPI file HTML files constituting code coverage report 7-6 E linker options -l pgo 3-6 Linking the object files 3-6 Login name for codecov access to NonStop server 7-5 Embedded SQL, instrumentation supported for 1-2 ETK see HP Enterprise Toolkit Examples
Index R Performance of instrumented code 4-1, 10-2 pgopti.
Index U on code coverage report 7-6 U Uncovered block color code for 7-4, 7-5 defined Glossary-2 Usage considerations 10-1 scenarios 9-1 Special Characters #include files 8-3 -l pgo linker option 3-6 HP Code Profiling Utilities Manual—542684-003 Index -4
