may day: debugging

If your program is causing strange problems, it might be a good opportunity to lean back and think for a while. 

• What were the last changes that you made? (A source code revision system e.g. SVN, CVS or RCS might help.) 

• Reduce the optimization level of your compilation. 

• Chose a smaller data set. Try to build a specific test case for your problem. 

• Look for compiler messages and warnings. Use tools for a static program analysis. 

• Try a dynamic analysis with appropriate compiler options. 

• Reduce the number of CPUs in a parallel program, try a serial program run if possible. 

• Use a debugger like TotalView. Use the smallest case which shows the error. 

• In case of an OpenMP program, use a threadchecking tool like the Sun Thread Analyzer or the Intel ThreadChecker. 

• If it is an OpenMP program, try to compile without optimization, e.g. with -g -O0 -xopenmp=noopt for the Sun compilers. 

• In case of an MPI program, use a MPI tool or a parallel debugger like TotalView. Try another MPI implementation version and/or release. 

• Try an other compiler. Maybe you run into a compiler bug?