3

I want to debug a C program running multiple TCP server threads. I cannot set up a sophisticated Debugging tool as I have to debug in an embedded linux(busy box). I hoped it would natively support gdb.

SO I started with gdb. Once I type run, the server seems to run in the background but gdb returns the prompt and "Program received signal SIG64 (Real-time event 64)" message(related to pthread I guess). I know it has to do be something with the main getting forked into several threads. But I have no clue how to debug this. Any starting points would be so helpful.

Also, is there someother "trace" like debuggers, small footprint that I can use?

Please help

Improve this question

5 Answers 5

Reset to default
9

Most often, debugging a multi-threaded application is difficult with a debugger. The best way is to either try and isolate the bug to a single-threaded case, or use debug prints in suspect locations until the bug is discovered.

It's no help with your specific issue, but it's the best advice I've learned while working with multi-threaded applications, especially embedded.

Improve this answer
Sign up to request clarification or add additional context in comments.

7 Comments

Given the multithreaded nature of the program, it makes sense to add that, while using debug prints, do not rely on the order of the prints accross different threads.
@als : it is a good idea in this case to have a "logger" thread, where you send the message you want to print (via msgqueue for example). The logger thread timestamps message and print them in the order in receive them from the msg queue.
Nice suggestion Xavier. Perhaps I have to try that out :)
if the problem is just coming due to just keeping multithread then its useless to isolate the bug to a single-threaded case & study
@Xavier: It's a good suggestion indeed, It won't help much with Timing sensitive issues though, but worth a try for sure.
|
3

See in such case I usually do this:

  1. Make a thread wise log file and have all stdout and stderr output redirect in that log file ... maybe this will help you for that: In multi thread application how can i redirect stderr & stdout in separate file as per thread?

  2. Keep track of global variables between all threads. Improper use of global variables tends to cause problems.

  3. If you are using a mutex then check that it will not create deadlocks. In conditional & semaphore design always try to track of all those threads on paper.

Improve this answer

Comments

1

I recommend you to use memory access check program like valgrind. In my case, many of bugs are caused by illegal memory handling. It's hard to find bug on multi-threaded program so using memory leak checking program is a better way to figure out bugs causes.

Improve this answer

3 Comments

Valrgrind probably won't be able to run on an embedded Linux with BusyBox.
@Eli lser I have tried to use valgrind on s3c6410 embedded board. compiling is success but actually i've failed to load valgrind because some of shared object version is mismatched with valgrind. if busybox he has contains proper so files it would work.
I compiled my code with -lmcheck and resolved several "leaky" pointers :)
0

some tricky idea ..

  1. add sleep(some times) into the thread to debug as starting point
  2. after program is running, check pid of thread (using ps with -L option)
  3. run gdb program {pid} or call attach {pid} in gdb prompt
  4. after sleep, u can trace next step for that thread

Don't forget attaching must be done before sleep time is gone.

As remarked above, testing on single thread or using text logging facility is good choice.

Improve this answer

Comments

0

GDB has a few functions for debugging with multiple threads. For example, you can run the command i threads to view the currently existing threads. Here is a link that may be helpful: https://ftp.gnu.org/old-gnu/Manuals/gdb/html_node/gdb_24.html.

Improve this answer

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.