Debugging threads with HP Wilde Beest Debugger HP Part Number: 5992-4663 Published: September 2008 Edition: 1.
© Copyright 2008 Hewlett-Packard Development Company, L.P Legal Notices The information in this document is subject to change without notice. Hewlett-Packard makes no warranty of any kind with regard to this manual, including, but not limited to, the implied warranties of merchantability and fitness for a particular purpose.
Table of Contents About This Document.....................................................................................................................7 Intended Audience...............................................................................................................7 Typographic Conventions....................................................................................................7 Related Information.................................................................................
Thread-debugging in Batch mode.................................................................................37 Pre-requisites for Batch Mode of Thread Debugging..............................................38 Steps to debug threads in Batch Mode....................................................................38 Limitations in Batch Mode of Thread Debugging...................................................40 Known issues with thread debugging for interactive and batch mode..................
List of Tables 1 2 Documentation for HP WDB........................................................................................8 Thread Debugging Commands...................................................................................
About This Document This whitepaper describes the various commands and options available in HP Wilde-Beest Debugger (WDB) to debug threads in programs. In addition, this paper addresses the most common thread programming issues along with the various thread-debugging commands in HP WDB. Intended Audience This document targets the developers who want to use HP WDB to debug their threaded applications developed in C and C++.
{} ... | WARNING CAUTION IMPORTANT NOTE TIP The contents are required in syntax. If the contents are a list separated by |, you must choose one of the items. The preceding element can be repeated an arbitrary number of times. Indicates the continuation of a code example. Separates items in a list of choices. A warning calls attention to important information that if not understood or followed will result in personal injury or nonrecoverable system problems.
http://www.hp.
What are threads? Threads are subsets of ‘Process’, which aid in accelerating the execution of any task. The usage of threads increases efficiency that results from the intended concurrency in the threaded-programming practice. Threads share the same resources as Process and hence do not have resource overheads. This property of threads is important when processing speed becomes a criterion to measure efficiency and if programmers use more than one thread to complete a single process.
Common conditions or events in thread programming The following are the most common conditions or events in thread programming that could lead to errors: 1. 2. 3. 4. 5. 6. 7. 8. 9. The thread attempts to acquire a non-recursive mutex that it currently holds. The thread attempts to unlock a mutex or a read-write lock that it has not acquired. The thread waits on a mutex or a read-write lock that is held by a thread with a different scheduling policy.
Thread Debugging Support in HP WDB HP WDB provides thread-debugging support for kernel, user, and MxN threads. You can exclusively disable or enable specific thread execution. Advanced thread debugging support in HP WDB enables you to view information on pthread primitives and detect certain thread-related conditions. NOTE: WDB supports pthread Parallelism, but it does not support compiler-generated parallelism like parallelism with Directives.
The backtrace_other_thread command prints the backtrace of all stack frames for a thread with stack pointer SP, program counter PCand address of gr32 in the backing storeBSP. This command enables you to view the stack trace when the stack is corrupted. When using this command, you must ensure that the SP,PC , and BSP values are valid.
For debugging purposes, GDB associates its own thread number - always a single integer - with each thread in your program. Following commands are used to debug multi-threaded programs: • • • thread threadno, a command to switch among threads info threads, a command to inquire about existing threads thread apply [threadno] [all] args, a command to apply a command to a list of threads info threads Display a summary of all threads currently in your program. GDB displays for each thread (in this order) : 1.
thread apply [threadno] [all] args The thread apply command allows you to apply a command to one or more threads. Specify the numbers of the threads that you want affected with the command argument threadno. The threadno is the internal GDB thread number, as shown in the first field of the info threads display. To apply a command to all threads, use thread apply all args.
You can use the thread qualifier on conditional breakpoints as well; in this case, place thread threadno before the breakpoint condition, like this: ((gdb)) break frik.c:13 thread 28 if bartab > lim Whenever your program stops under GDB for any reason, all threads of execution stop, not just the current thread. This allows you to examine the overall state of the program, including switching between threads. Conversely, whenever you restart the program, all threads start executing.
• • • • Advanced thread-debugging requires the pthread tracing library. The pthread tracer library is available by default in systems running on HP-UX 11i v2 or later. HP WDB 5.5 and later versions support enhanced thread debugging. The installation scripts for HP WDB 5.5 and later versions of the debugger automatically add links at /opt/langtools/lib/ to replace the standard libpthread library with libpthread tracer library at run time. HP WDB uses librtc.sl to enable thread debugging support.
4. Invoke the executable that you want to debug: (gdb) file 5. Enable thread check along with the specific option as required. (gdb) set thread-check [option][on|off] 6. Execute the file with the following command: (gdb) run The thread-check command The advanced thread debugging features can be enabled only if the set thread-check[on] command is enabled.
/* Print error information, exit with -1 status.
/* Rest of application code here */ /* * Create a thread */ ret_val = pthread_create(&tid1, (pthread_attr_t *)NULL, (void *(*)())start_routine, (void *)1); check_error(ret_val, "pthread_create 1 failed"); /* * Wait for the threads to finish */ ret_val = pthread_join(tid1, (void **)NULL); check_error(ret_val, "pthread_join: tid1"); } void start_routine(int thread_num) { int ret_val; sched_yield(); /* Lock the recursive lock recursively.
ret_val = pthread_mutex_unlock(&n_mtx); check_error(ret_val, "mutex_unlock n_mtx"); printf("Thread %d - released n_mtx\n", thread_num); } At run-time, the debugger keeps track of each mutex in the application and the thread that currently holds each mutex. When a thread attempts to acquire a lock on a non-recursive mutex, the debugger checks if the thread currently holds the lock object for the mutex.
#define check_error(return_val, msg) { if (return_val != 0) fatal_error(return_val, msg); } \ \ \ main() { pthread_t extern void int tid1; start_routine(int num); ret_val; /* * Create a thread */ ret_val = pthread_create(&tid1, (pthread_attr_t *)NULL, (void *(*)())start_routine, (void *)1); check_error(ret_val, "pthread_create 1 failed"); /* * Wait for the threads to finish */ ret_val = pthread_join(tid1, (void **)NULL); check_error(ret_val, "pthread_join: tid1"); } void start_routine(int thread_num) {
NOTE: In some rare predictable situations the thread might attempt to unlock an object that it has no control over. For example, an application which instructs the thread to unlock a mutex when it encounters a C++ destructor, irrespective of the history of the processing of the C++ constructor. Problem: The Thread waits on a mutex or a read-write lock that is held by a thread with a different scheduling policy Consider the following scenario: Thread 1 is scheduled using Policy1, SP1.
return((void *)NULL); } void * thread2_func() { int ret_val; ret_val = pthread_mutex_lock(&mtx); check_error(ret_val, "mutex_lock mtx"); printf("In thread2_func()\n"); sleep(5); ret_val = pthread_mutex_unlock(&mtx); check_error(ret_val, "mutex_unlock mtx"); return((void *)NULL); } main() { pthread_t pth_id[2]; int ret_val, scope; int old_policy; pthread_attr_t attr; struct sched_param param, old_param; /* Initialize the threads attributes object */ ret_val = pthread_attr_init(&attr); check_error(ret_val,
ret_val = pthread_attr_setschedparam(&attr, ¶m); check_error(ret_val, "attr_setschedparam() 2"); ret_val = pthread_create(&pth_id[1], &attr, thread2_func, NULL); check_error(ret_val, "pthread_create() 2"); /* Destroy the thread attributes object */ ret_val = pthread_attr_destroy(&attr); check_error(ret_val, "attr_destroy()"); /* wait for the threads to finish */ ret_val = pthread_join(pth_id[0], (void **)NULL); check_error(ret_val, "pthread_join() 1"); ret_val = pthread_join(pth_id[1], (void **)NULL); c
producer_thread(pthread_mutex_t* job_lock) { int ret_val; /* Acquire the associated mutex lock */ if ((ret_val = pthread_mutex_lock(job_lock)) != 0) fatal_error(ret_val, "p mtx_lock failed"); /* Signal the condvar to wakeup one thread */ if ((ret_val = pthread_cond_signal(&job_cv)) != 0) fatal_error(ret_val, "cond_signal failed"); /* Release the associated mutex */ if ((ret_val = pthread_mutex_unlock(job_lock)) != 0) fatal_error(ret_val, "mtx_unlock failed"); } void consumer_thread(pthread_mutex_t* job_loc
l1 = &job_lock1 l2 = &job_lock2 } else { l1 = l2 = &job_lock1 } alarm(20); /* Create two threads to do the work */ ret_val = pthread_create(&tid1, (pthread_attr_t *)NULL, (void *(*)())consumer_thread, (void *) l1); check_error(ret_val, "pthread_create 1 failed"); ret_val = pthread_create(&tid2, (pthread_attr_t *)NULL, (void *(*)())producer_thread, (void *) l1); check_error(ret_val, "pthread_create 2 failed"); if (l1 != l2) { ret_val = pthread_create(&tid3, (pthread_attr_t *)NULL, (void *(*)())consumer_threa
} The following command enables you to check this condition in a threaded application. (gdb) set thread-check cv-multiple-mxs[on|off] The debugger transfers the execution control to the user and prints a warning message when this condition is detected. The following is a segment of the HP WDB output: Starting program: /home/gdb/enh_thr_cv_multiple_mxs bad [Switching to thread 4 (system thread 39531)] warning: Attempt to associate condition variable 0 with mutexes 1 and 2.
if (return_val != 0) fatal_error(return_val, msg); \ \ } main() { pthread_t int tid; ret_val; /* Create two threads to do the work */ ret_val = pthread_create(&tid, (pthread_attr_t *)NULL, (void *(*)())producer_thread, (void *) &job_lock1); check_error(ret_val, "pthread_create 2 failed"); /* Wait for the threads to finish */ ret_val = pthread_join(tid, (void **)NULL); check_error(ret_val, "pthread_join: tid"); exit(0); } void fatal_error(int err_num, char *function) { char *err_string; err_string = stre
TIP: • If the remaining segments of the application require access to the locked mutex, modify the code segment of the terminating thread to unlock the mutex before it terminates. • If the termination is the result of an exception, then consider using a condition handler (in C++) or POSIX Threads library TRY/FINALLY blocks. Problem: The thread waits on a condition variable for which the associated mutex is not locked.
#define check_error(return_val, msg) { if (return_val != 0) fatal_error(return_val, msg); } \ \ \ main() { pthread_t tid1, tid2; int ret_val; alarm (20); /* Create two threads to do the work */ ret_val = pthread_create(&tid1, (pthread_attr_t *)NULL, (void *(*)())consumer_thread, (void *) &job_lock1); check_error(ret_val, "pthread_create 1 failed"); ret_val = pthread_create(&tid2, (pthread_attr_t *)NULL, (void *(*)())producer_thread, (void *) &job_lock1); check_error(ret_val, "pthread_create 2 failed"); /*
Starting program: /home/gdb/enh_thr_cv_wait_no_mx [Switching to thread 2 (system thread 39559)] warning: Attempt by thread 2 to wait on condition variable 0 without locking the associated mutex 1. 0x800003ffeffcc608 in __rtc_pthread_dummy+0 () from ../librtc64.sl NOTE: This is an additional check that HP WDB provides and is not a POSIX.1 standard requirement for the pthread_cond_wait() routine.
fatal_error(return_val, msg); \ } main() { pthread_t int tid1, tid2, tid3; ret_val; /* Create two threads to do the work */ ret_val = pthread_create(&tid1, (pthread_attr_t *)NULL, (void *(*)())my_thread, (void *)1); check_error(ret_val, "pthread_create 1 failed"); ret_val = pthread_create(&tid2, (pthread_attr_t *)NULL, (void *(*)())my_thread, (void *)2); check_error(ret_val, "pthread_create 2 failed"); ret_val = pthread_create(&tid3, (pthread_attr_t *)NULL, (void *(*)())my_thread, (void *)3); check_erro
Starting program: /home/gdb/enh_thr_exit_no_join_detach In thread 1 In thread 2 In thread 3 warning: Attempt to exit thread 4 which has neither been joined nor detached. 0x800003ffeffcc608 in __rtc_pthread_dummy+0 () from ../librtc64.sl NOTE: A violation of this condition implies outstanding resources that are not released. If the number of violations is small, or if they occur on an error path that causes abrupt termination of the application, you can disable this check on threads.
if ((ret_val = pthread_mutex_lock(&job_lock)) != 0) fatal_error(ret_val, "p mtx_lock failed"); for (int i = 0; i < 100; i++) more_stack[i] = i; for (int i = 0; i < 1000; i++); /* Release the associated mutex */ if ((ret_val = pthread_mutex_unlock(&job_lock)) != 0) fatal_error(ret_val, "mtx_unlock failed"); my_thread(); } #define check_error(return_val, msg) { if (return_val != 0) fatal_error(return_val, msg); } \ \ \ main() { pthread_t int tid; ret_val; /* Create two threads to do the work */ ret_val =
The following is a segment of the HP WDB output: (gdb) set thread-check stack-util 101 Invalid value: stack utilization must be between 0 and 100. (gdb) set thread-check stack-util 80 (gdb) run Starting program: /home/gdb/enh_thr_stack_util [Switching to thread 2 (system thread 39877)] warning: Thread 2 exceeded stack utilization threshold of 80%. 0x800003ffeffcc608 in __rtc_pthread_dummy+0 () from ../librtc64.sl This warning indicates that the thread attempts to exceed its stack utilization limit.
In batch mode, the debugger detects the all the thread-conditions that are detected during an interactive debugging session. The debugger reports extended information such as variable address, name, id and other specifications related to the involved pthread objects. In addition, it displays the stack trace of the executing thread at the point of error.
NOTE: The configuration file contains lines of the following form: set thread-check [on|off] | [option] [on|off] | [option] [num] And/Or set frame-count [num] And/Or files = 4. 5. Set the environment variable BATCH_RTC to on as export set BATCH_RTC=on Complete one of the following steps to preload the librtc runtime library: • Set the target application to preload librtc by using the +rtc option for the chatr command.
To explicitly preload the librtc runtime library and start the target application, enter one of the following commands: — For 32 bit IPF applications, LD_PRELOAD=/opt/langtools/lib/hpux32/librtc.so — For 64 bit IPF applications, LD_PRELOAD=/opt/langtools/lib/hpux64/librtc.so — For 64-bit PA applications, LD_PRELOAD=/opt/langtools/lib/pa20_64/librtc.
This error message appears because the thread-error information for the forked process is not available. However, if the forked process exec()s another binary, the thread-error information is available for the exec -ed binary. Issue 2: In both interactive and batch modes, if the applications exceed their thread stack utilization, the following error message appears: Error accessing memory address This occurs when GDB attempts a command line call on an already overflowing thread stack.
NOTE: The chatr +rtc option preloads the librtc runtime library from the following default paths: — For 32-bit IPF applications, /opt/langtools/lib/hpux32/librtc.so — For 64-bit IPF applications, /opt/langtools/lib/hpux64/librtc.so — For 32-bit PA applications, opt/langtools/lib/librtc.sl — For 64-bit PA applications, /opt/langtools/lib/pa20_64/librtc.sl To preload the librtc runtime library from a path that is different from the default paths, you must use the LD_PRELOAD environment variable.
If LD_PRELOAD and chatr +rtc are used to preload the librtc runtime library, the librtc runtime library is loaded from the path specified by LD_PRELOAD. 3. Complete one of the following steps: • Attach the debugger to the required process and enable thread debugging, as follows: gdb -thread -p or gdb -thread • Alternately, you can attach the process to the debugger and consequently invoke thread debugging, as follows: $ gdb ...
where, is the process identifier. Miscellaneous notes on Advanced thread debugging feature in HP WDB The following commands enable you to view extended information on threads, mutexes, read-write locks and conditional variables in multi-threaded applications: info thread [thread-id] The info thread [thread-id] command displays a list of known threads. If you provide a thread-id, the command displays extended information on the specified thread.
• • • Consider writing a recursive function in an iterative form, as sometimes an iterative function demonstrates greater resource efficiency than a recursive function. Minimize the size and number of the stack local variables to reduce the stack usage of a thread. Allocate large data items dynamically in a heap than in arrays to ensure that stack utilization does not exceed the allocated or default percentage. Appendix Pertinent terms used throughout the white paper.
Stack trace Stack trace is a summary of the stack frames. These stack frames contain the function calls which the program initiates during its execution. Thread Debugging commands at a glance Table 2 Thread Debugging Commands Command Description set thread-check on/off Enables or disables Advanced thread debugging feature. The default setting is off. info-thread[thread-id] Displays a list of known threads. info mutex[mutex-id] Displays a list of known mutexes.
