Q: How do I debug a VR Juggler application crash?
When I run a VR Juggler application, it crashes with an error message. How do I debug this and get assistance on the mailing list?
A: Use a debugger to get information about the state of the application when it crashed.
There are several things that cause natively compiled software to crash when executed. Errors most commonly seen are the following:
- Segmentation fault: Bad memory access
- Assert failed: Failed run-time correctness test
- Abort: Uncaught C++ exception
- Bus error
On UNIX-based systems (IRIX, Linux, Solaris, FreeBSD, etc.), the first three of the above usually leaves a "core" file in the directory where the application was run. This file is the application memory image dumped to disk, and it can be loaded into a debugger. On Windows, memory images of applications can be dumped to disk, but it is usually easier to load the crashed application directly into the Visual Studio debugger. The MSVC run time library will pop up a dialog box giving you the option to do this.
Debuggers can show information including, but not limited to, the following:
- The exact line in the code where the crash occurred.
- The list of all function calls leading up to the crash (also known as the call stack).
- The state of all application threads when the crash occurred.
- The values of variables on the heap and on the stack.
Giving the Debugger the Information it Needs
Debuggers require special information from the application binary and associated shared libraries in order to their jobs effectively. First, you must make sure that you have compiled debugging symbols into the code you want to debug. This means passing the right options to the compiler when making the .o or .obj files. These options vary from compiler to compiler, but most UNIX C++ compilers accept the -g flag. For Visual C++, you can set an option in the application project file using the GUI to add debugging symbols.
Next, the debugger you use can handle the binary code you have. This is especially important on operating systems with multiple compilers and/or debuggers installed. For example, the code generated by the SGI MIPSpro Compilers cannot be loaded by GDB, the GNU Debugger. Similarly, code compiled by the GCC C++ compiler cannot be loaded by DBX or CVD.
Finally, if the crash is occuring in the VR Juggler code, make sure that you run the application with the debugging versions of the Juggler libraries. If you link against the shared libraries (those with one of the extensions .so, .sl, .dylib, or .dll), change your shared library lookup path to point at the debug directory. For example:
% setenv LD_LIBRARY_PATH $VJ_BASE_DIR/lib/debug
When you run your application the next time, it will use the debug-enabled versions of the libraries, thus allowing you to get debugging information about Juggler code in the core file.
Using Debuggers
There are different types of debuggers for C++ code, and these days, there are graphical frontends for all the common debuggers. The following lists the debuggers we have used:
- DDD: A graphical frontend to GDB, the GNU Project Debugger, and debuggers for other languages.
- CVD: A graphical debugger available on SGI IRIX systems.
- Visual Studio debugger
If you are planning to submit a crash report to one of the VR Juggler mailing lists, the first thing that you will be asked to provide is the list of function calls leading up to the crash. This is also referred to as a "backtrace," a "stack trace," or the "call stack." The method for getting this information varies from one debugger to the next. The following shows how to get it using GDB (the call stack is shown in bold red text, and the commands you type are shown in plain bold text):
% gdb appname core
GNU gdb Red Hat Linux (5.2.1-4)
Copyright 2002 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB. Type "show warranty" for details.
This GDB was configured as "i386-redhat-linux"...
Core was generated by `./cubes standalone.jconf'.
Program terminated with signal 11, Segmentation fault.
Reading symbols from /.../lib/i686/libvrj.so...done.
Loaded symbols for /.../lib/i686/libvrj.so
Reading symbols from /.../lib/i686/libvrj_ogl.so...done.
Loaded symbols for /.../lib/i686/libvrj_ogl.so
Reading symbols from /.../lib/i686/libsonix.so...done.
Loaded symbols for /.../lib/i686/libsonix.so
Reading symbols from /.../lib/i686/libgadget.so...done.
Loaded symbols for /.../lib/i686/libgadget.so
Reading symbols from /.../lib/i686/libjccl.so...done.
Loaded symbols for /.../lib/i686/libjccl.so
Reading symbols from /.../lib/i686/libvpr.so...done.
Loaded symbols for /.../lib/i686/libvpr.so
Reading symbols from /usr/X11R6/lib/libX11.so.6...done.
Loaded symbols for /usr/X11R6/lib/libX11.so.6
Reading symbols from /lib/libuuid.so.1...done.
Loaded symbols for /lib/libuuid.so.1
Reading symbols from /lib/libdl.so.2...done.
Loaded symbols for /lib/libdl.so.2
Reading symbols from /usr/X11R6/lib/libGLU.so.1...done.
Loaded symbols for /usr/X11R6/lib/libGLU.so.1
Reading symbols from /usr/lib/libGL.so.1...done.
Loaded symbols for /usr/lib/libGL.so.1
Reading symbols from /usr/lib/libstdc++.so.5...done.
Loaded symbols for /usr/lib/libstdc++.so.5
Reading symbols from /lib/i686/libm.so.6...done.
Loaded symbols for /lib/i686/libm.so.6
Reading symbols from /lib/libgcc_s.so.1...done.
Loaded symbols for /lib/libgcc_s.so.1
Reading symbols from /lib/i686/libpthread.so.0...done.
Loaded symbols for /lib/i686/libpthread.so.0
Reading symbols from /lib/i686/libc.so.6...done.
Loaded symbols for /lib/i686/libc.so.6
Reading symbols from /lib/ld-linux.so.2...done.
Loaded symbols for /lib/ld-linux.so.2
Reading symbols from /usr/lib/libGLcore.so.1...done.
Loaded symbols for /usr/lib/libGLcore.so.1
Reading symbols from /usr/X11R6/lib/libXext.so.6...done.
Loaded symbols for /usr/X11R6/lib/libXext.so.6
Reading symbols from /.../lib/jccl/plugins/corba_rtrc_d.so...done.
Loaded symbols for /.../lib/jccl/plugins/corba_rtrc_d.so
Reading symbols from /.../lib/i686/libtweek.so...done.
Loaded symbols for /.../lib/i686/libtweek.so
Reading symbols from /usr/lib/libomniORB4.so.0...done.
Loaded symbols for /usr/lib/libomniORB4.so.0
Reading symbols from /usr/lib/libomnithread.so.3...done.
Loaded symbols for /usr/lib/libomnithread.so.3
Reading symbols from /usr/lib/libomniDynamic4.so.0...done.
Loaded symbols for /usr/lib/libomniDynamic4.so.0
Reading symbols from /lib/libnss_files.so.2...done.
Loaded symbols for /lib/libnss_files.so.2
#0 0x4207da1c in memcpy () from /lib/i686/libc.so.6
(gdb) backtrace
#0 0x4207da1c in memcpy () from /lib/i686/libc.so.6
#1 0x0805c160 in cubesApp::myDraw(vrj::User*) (this=0x808f1b0, user=0x813e458)
at /home/users/patrick/src/Juggler/main/juggler/modules/vrjuggler/test/Draw/OGL/cubes/cubesApp.cpp:279
#2 0x08064438 in cubesApp::draw() (this=0x808f1b0)
at /home/users/patrick/src/Juggler/main/juggler/modules/vrjuggler/test/Draw/OGL/cubes/cubesApp.h:248
#3 0x400baf55 in vrj::GlPipe::renderWindow(vrj::GlWindow*) (this=0x813f800,
win=0x813ffa0)
at /home/users/patrick/src/Juggler/main/juggler/modules/vrjuggler/vrj/Draw/OGL/GlPipe.cpp:414
#4 0x400b9d69 in vrj::GlPipe::controlLoop(void*) (this=0x813f800,
nullParam=0x0)
at /home/users/patrick/src/Juggler/main/juggler/modules/vrjuggler/vrj/Draw/OGL/GlPipe.cpp:189
#5 0x400bd59f in vpr::ThreadMemberFunctor::operator()() (
this=0x813f8e0)
at /mnt/rigby/home13/users/patrick/src/Juggler/main/juggler/build.linux-rh80.posix.gcc32/instlinks/include/vpr/Thread/ThreadFunctor.h:128
#6 0x403c584d in vpr::ThreadPosix::startThread(void*) (this=0x813f8f8,
nullParam=0x0)
at /home/users/patrick/src/Juggler/main/juggler/modules/vapor/vpr/md/POSIX/Thread/ThreadPosix.cpp:275
#7 0x403c63f9 in vpr::ThreadMemberFunctor::operator()() (
this=0x813f980)
at /home/users/patrick/src/Juggler/main/juggler/modules/vapor/vpr/Thread/ThreadFunctor.h:128
#8 0x403c09b1 in vprThreadFunctorFunction (args=0x813f980)
at /home/users/patrick/src/Juggler/main/juggler/modules/vapor/vpr/Thread/ThreadFunctor.cpp:74
#9 0x406d4881 in pthread_detach () from /lib/i686/libpthread.so.0
There is a lot of extra information that is normally not needed when responding to a request for a stack trace. Only the text shown in bold red text is needed initially.
There are tutorials for using other debuggers available online. Use Google or one of the links below to get more information about the debugger you are using.
Tips
GDB
- The GCC documentation has some [[][great information]] on using the libstdc++ debugging features.
- GDB can be configured to make C++ debugging easier. Adding the following options to
~/.gdbinit is recommended:
set print pretty on
set print object on
set print static-members on
set print vtbl on
set print demangle on
set demangle-style gnu-v3
Visual Studio Debugger
- To display the call stack, start the Visual Studio Debugger. After doing this, open the "Debug" menu and then the "Windows" sub-menu. When the debugger is running, there will be options to enable and disable various views, one of which is the call stack. This menu option is only available when the debugger is running.
Common Problems
Below are some common issues encountered when trying to debug a crashed application. Note that any commands you should type are printed in bold text.
When I run the application, it crashes but there is no core file in the directory afterwards.
Your shell is preventing core files from being stored. Run the limit command to get something similar to the following:
% limit
cputime unlimited
filesize unlimited
datasize unlimited
stacksize 8MB
coredumpsize 0kB
memoryuse unlimited
maxproc 4092
descriptors 1024
memorylocked unlimited
addressspace unlimited
maxfilelocks unlimited
Note that coredumpsize is set to 0 kilobytes. To change this, run the following:
% unlimit coredumpsize
Back to: LiveFaq
|
