3

I'm using gdb-7.11.1 and I get this message on my embedded powerpc system. Some more background, the libpthread I use has been stripped off all the non-dynamic symbols, including nptl_version, which libthread_db uses to make sure it is compatible with libpthread.

Coming to my problem, gdb says it won't be able to debug threads, but it seemingly can as evidenced below. Am I simply misunderstanding what 'thread debugging' means? (The ?? you see are naturally due to the missing symbol table in libpthread)

(gdb) break fn2
Breakpoint 1 at 0x1000052c: file test.c, line 7.
(gdb) run
Starting program: /tmp/test
warning: Unable to find libthread_db matching inferior's thread library, thread debugging will not be available.
[New LWP 21312]
[New LWP 21313]
[New LWP 21314]
[New LWP 21315]
[New LWP 21316]
[New LWP 21317]
[Switching to LWP 21315]

Thread 5 hit Breakpoint 1, fn2 () at test.c:7
7   test.c: No such file or directory.
(gdb) thread apply all bt

Thread 7 (LWP 21317):
#0  0x0fdcf030 in ?? () from /lib/libpthread.so.0
#1  0x0fdc892c in pthread_mutex_lock () from /lib/libpthread.so.0
#2  0x00000000 in ?? ()

Thread 6 (LWP 21316):
#0  0x0fdcf030 in ?? () from /lib/libpthread.so.0
#1  0x0fdc892c in pthread_mutex_lock () from /lib/libpthread.so.0
#2  0x00000000 in ?? ()

Thread 5 (LWP 21315):
#0  fn2 () at test.c:7
#1  0x0fdc6d8c in ?? () from /lib/libpthread.so.0
#2  0x0fd26074 in clone () from /lib/libc.so.6

Thread 4 (LWP 21314):
#0  0x0fdcf030 in ?? () from /lib/libpthread.so.0
#1  0x0fdc892c in pthread_mutex_lock () from /lib/libpthread.so.0
#2  0x00000000 in ?? ()

Thread 3 (LWP 21313):
#0  0x0fdcf030 in ?? () from /lib/libpthread.so.0
#1  0x0fdc892c in pthread_mutex_lock () from /lib/libpthread.so.0
#2  0x00000000 in ?? ()

Thread 2 (LWP 21312):
#0  0x0fdcefdc in ?? () from /lib/libpthread.so.0
#1  0x0fdc892c in pthread_mutex_lock () from /lib/libpthread.so.0
#2  0x00000000 in ?? ()

Thread 1 (LWP 21309):
#0  0x0fd26038 in clone () from /lib/libc.so.6
#1  0x0fdc5f2c in ?? () from /lib/libpthread.so.0
#2  0x0fde6150 in ?? () from /lib/libpthread.so.0
#3  0x0fdc6424 in pthread_create () from /lib/libpthread.so.0
#4  0x100006a4 in main () at test.c:23
(gdb)
Improve this question
2
  • Are you cross-debugging, by any chance? Most probably you will need to set solib-search-path or set libthread-db-search-path to have GDB find the proper shared libs. Commented Sep 15, 2016 at 7:30
  • @tofro No I'm actually running gdb on the board itself. I can see that gdb finds both libpthread and libthread_db in /lib/, and those are the only copies in the entire fs. Commented Sep 15, 2016 at 7:56

2 Answers 2

Reset to default
1

On Linux (at least, and others), an important part of the threading library is implemented in the kernel: that the "kernel-thread", called LWPs (for light-weight process).

GDB doesn't need libthread_db help to track them, as the OS itself can give the information the key information about them: their CPU registers (mainly IP, SP, FP).

I'm not sure what libthread_db provides in that context. The only thing I can think of is the LWP <-> Thread id mapping:

* 3    Thread 0x7ffff6d19700 (LWP 21571) "erato" primes_computer_runner2 (param=0x7fffffffca50) at erato.c:46
  1    Thread 0x7ffff7fad700 (LWP 21565) "erato" 0x00007ffff7bc568d in pthread_join () from /usr/lib/libpthread.so.0

(gdb) print/x thread_handle
$1 = 0x7ffff6d19700

See, Thread 0x7ffff7fad700 maps to LWP 21565.

In comparison, without libthread_db it just gives the LWP id (in another run):

* 3    LWP 22060 "erato" primes_computer_runner2 (param=0x7fffffffca50) at erato.c:46
  1    LWP 22058 "erato" 0x00007ffff76037b1 in clone () from /usr/lib/libc.so.6

If you want further details about pthread_db purpose, and why it's mandatory (or something equivalent) for user and hybrid threading libraries, you can take a look at this article I wrote several years ago:

Improve this answer
Sign up to request clarification or add additional context in comments.

2 Comments

"what libthread_db provides in that context" -- it provides notification of new thread creation (among other things), so GDB can attach new threads. Without it, newly created threads may run unattached (at least until next breakpoint), and if the new thread crashes, you may lose the entire inferior with it.
I see ... but couldn't it use the same technique as what is used for fork and/or exec? (I mean, catch fork and catch exec.) I don't see any fundamental difference between them and thread creation
1

The common cause for this error message:

Unable to find libthread_db matching inferior's thread library, ...

is having libpthread.so.0 that is fully stripped. Don't do that.

In particular, libthread_db.so needs nptl_version (local) symbol. You can verify whether your libpthread.so.0 has it with:

nm /path/to/libpthread.so.0 | grep version

which should produce something like:

0000000000012cc6 r nptl_version
Improve this answer

8 Comments

Is there any way to bypass the nptl_version check? I have a hung process which uses stripped version of libpthread.so. After attaching to it with gdb I cannot debug threads due to aforementioned error. Is there any way to force gdb to skip the version check? I have an unstripped version of libpthread.so, if that matters.
@AlexChe Indeed this is possible (two different ways). Care to ask a separate question.
Just chiming in that I can use 'nm' and 'file' to confirm my libpthread is not stripped, yet I'm seeing the error.
@JoeManiaci Do you have libthread_db.so.1 at all? Did it come from the same package as libpthread.so.0 ?
No, I'm working out of an embedded ARM environment if that makes a difference. I am also getting this warning while attempting to analyze a core dump as well.
|

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.