Not to Able analayze the Core dump issue for Multithreaded application.........(Help Required)

Question

I am working on multhithreading application when ever the process dump it always generates core as shown below i am not able to understand where it is actually dumping.

GNU gdb Red Hat Linux (6.5-25.el5rh)
Copyright (C) 2006 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB.  Type "show warranty" for details.
This GDB was configured as "x86_64-redhat-linux-gnu"...(no debugging symbols found)
Using host libthread_db library "/lib64/libthread_db.so.1".


warning: exec file is newer than core file.
Core was generated by `multithreadprocess                                                                       '.
Program terminated with signal 11, Segmentation fault.
#0  0x0000000000448f7a in std::ostream::operator<< ()
(gdb) where
0x000000000044bd32 in std::ostream::operator<< ()
#1  0x0000000000450b21 in std::ostream::operator<< ()
#2  0x000000000042eda9 in std::string::operator= ()
#3  0x00000030582062e7 in start_thread () from /lib64/libpthread.so.0
#4  0x00000030576ce3bd in clone () from /lib64/libc.so.6

(gdb)thread apply all bt 
Thread 6 (process 11674):
#0  0x000000305820a687 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
#1  0x0000000000431140 in std::string::operator= ()
#2  0x00000030582062e7 in start_thread () from /lib64/libpthread.so.0
#3  0x00000030576ce3bd in clone () from /lib64/libc.so.6

Thread 5 (process 11683):
#0  0x000000305820cbfb in write () from /lib64/libpthread.so.0
#1  0x0000000000449151 in std::ostream::operator<< ()
#2  0x000000000043b74a in std::string::operator= ()
#3  0x000000000046c3f4 in std::string::substr ()
#4  0x000000000046e3c1 in std::string::substr ()
#5  0x00000000004305a4 in std::string::operator= ()
#6  0x00000030582062e7 in start_thread () from /lib64/libpthread.so.0
#7  0x00000030576ce3bd in clone () from /lib64/libc.so.6

Thread 4 (process 11744):
#0  0x00000030576c5896 in poll () from /lib64/libc.so.6
#1  0x0000000000474f1c in std::string::substr ()
#2  0x000000000043b889 in std::string::operator= ()
#3  0x0000000000474dbc in std::string::substr ()
#4  0x00000000004306a5 in std::string::operator= ()
#5  0x00000030582062e7 in start_thread () from /lib64/libpthread.so.0
#6  0x00000030576ce3bd in clone () from /lib64/libc.so.6

Thread 3 (process 11864):
#0  0x000000305820a687 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
#1  0x0000000000431140 in std::string::operator= ()
#2  0x00000030582062e7 in start_thread () from /lib64/libpthread.so.0
#3  0x00000030576ce3bd in clone () from /lib64/libc.so.6

Thread 2 (process 11866):
#0  0x000000305820a687 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
#1  0x0000000000431140 in std::string::operator= ()
#2  0x00000030582062e7 in start_thread () from /lib64/libpthread.so.0
#3  0x00000030576ce3bd in clone () from /lib64/libc.so.6

Thread 1 (process 11865):
#0  0x000000000044bd32 in std::ostream::operator<< ()
#1  0x0000000000450b21 in std::ostream::operator<< ()
#2  0x000000000042eda9 in std::string::operator= ()
#3  0x00000030582062e7 in start_thread () from /lib64/libpthread.so.0
#4  0x00000030576ce3bd in clone () from /lib64/libc.so.6

If i give bt full it is showing like this

(gdb) bt full
#0  0x000000000044bd32 in std::ostream::operator<< ()
No symbol table info available.
#1  0x0000000000450b21 in std::ostream::operator<< ()
No symbol table info available.
#2  0x000000000042eda9 in std::string::operator= ()
No symbol table info available.
#3  0x00000030582062e7 in start_thread () from /lib64/libpthread.so.0
No symbol table info available.
#4  0x00000030576ce3bd in clone () from /lib64/libc.so.6
No symbol table info available.

Answer 1

Looks to me like you're using iostream inside a multithreaded application without the appropriate flags. See this. In particular, note that it says

When you build an application that uses the iostream classes of the libC library to run in a multithreaded environment, compile and link the source code of the application using the -mt option. This option passes -D_REENTRANT to the preprocessor and -lthread to the linker.

This is for a particular platform; your requirements may vary.

Answer 2

GDB 6.5 is quite old. You will likely get significantly better stack traces from (current) GDB 7.0.1.

You also appear to be trying to debug optimized code, built without -g flag, and you may not be debugging the right executable (GDB warns that your executable is newer than your core).

Make sure that your executable and all the libraries listed in info shared GDB output exactly match between the system where your core was produced and the system on which you are analyzing the core (if they are not the same) -- this is paramount -- if there is a mismatch, you'll likely get bogus stack traces (and the stack traces you've posted do look completely bogus to me).