Troubleshooting Tips: Failed Debugging with GDB

Three years ago I published “Debugging Failure: Check List and Hints” and unfortunately this article is one of the most popular ones: obviously debugging problems are very common. Debugging with GDB works usually fine, but if things are failing, then it can be hard to find the cause for it. Recently I have been asked to check some failures, so here are two more hints about what could go wrong…

Error while launching command: arm-none-eabi-gdb --version

Error while launching command: arm-none-eabi-gdb –version

I’m assuming a cross-debugging environment with Eclipse, the Eclipse GDB client and an external GDB server (see “GDB Client and Server: Unlocking GDB“) with the GNU MCU Eclipse plugins (although behaviour can be very similar for other combinations).

GDB Debugger Version

Debugger Console

Debugger Console

As a diagnostic hint: check the version of the debugger/gdb in the Debugger console view. Verify that this version matches what you *think* you are using.

One common problem is that cygwin or other tool chains are installed and have changed the global system path, and the wrong toolchain and GDB gets used. Mixing different GNU versions (compiler/debugger) is always a bad thing.

💡 see the next sections about which GNU tools are used.

Error while launching command: arm-none-eabi-gdb –version

Error while launching command: arm-none-eabi-gdb --version

Error while launching command: arm-none-eabi-gdb –version

If you get that error, this basically means that the gdb has not been found and cannot be launched.

Check your global toolchain folder if the gdb executable really exists in that path. Below is the setting for the machine:

Global Toolchain Folder

Global Toolchain Folder

There is the possibility to overwrite the global setting with a setting for the workspace. I keep it usually empty (so it uses the global setting), but check that path as well:

Workspace Toolchain Path

Workspace Toolchain Path

Lastly, there is setting for each project. If it is empty, then it uses the workspace setting. So check that project setting too:

Project Toolchain Path

Project Toolchain Path

Check as well in the debugger launch configuration if the settings are correct:

Debugger Launch Configuration

Debugger Launch Configuration

For the GDB Client executable, you might consider directly to point to the gdb executable to avoid any path search issues.

Global System Path used in Eclipse

The other thing to check is the global system path used by Eclipse when it launches an executable. You can find this in the screenshot below in the workspace settings:

System Path in Eclipse Workspace Settings

System Path in Eclipse Workspace Settings

With this, I hope you can find the problem. Otherwise have a read in the articles listed in the Links section below.

Happy Debugging 🙂

Links

19 thoughts on “Troubleshooting Tips: Failed Debugging with GDB

  1. hi Erich,
    I have a very weird problem. first of all I used kds with the sdk 1.3, the mcu is k64. I have written a program which works fine. however, if i make a copy of it and try to run that, it doesn’t work like to original one. or if i add just a few more codes, like do simple arithmetic with values obtained from the adc, the adc all of the sudden produces very different values. or when switch between run mode and very low power stop mode periodically, the debug console either stop working or if it works then only 3, 4 times before it stops completely.
    do you have any ideas why?
    thanks

    Like

  2. Hi Erich,

    I have Eclipse Oxygen set up per this guide:

    Breathing with Oxygen: DIY ARM Cortex-M C/C++ IDE and Toolchain with Eclipse Oxygen

    I also have Processor Expert installed along with the components that you’ve created.

    With those tools I am able to create a project for a K61 with FreeRTOS. I create 4 tasks – each task simply blinks an LED.

    This gets loaded into our board with a P&E Multilink Universal – and it runs 🙂

    But as soon as I put in a breakpoint the debug session terminates.

    Although I set this up yesterday, and everything should be at the latest version, it looks similar to this issue:
    https://community.nxp.com/message/883677

    What can I check next?

    Regards,
    Allen

    ps Thanks for all of the great articles!

    GNU gdb (GNU Tools for Arm Embedded Processors 7-2017-q4-major) 8.0.50.20171128-git
    Copyright (C) 2017 Free Software Foundation, Inc.
    License GPLv3+: GNU GPL version 3 or later
    This is free software: you are free to change and redistribute it.
    There is NO WARRANTY, to the extent permitted by law. Type “show copying”
    and “show warranty” for details.
    This GDB was configured as “–host=i686-w64-mingw32 –target=arm-none-eabi”.
    Type “show configuration” for configuration details.
    For bug reporting instructions, please see:
    .
    Find the GDB manual and other documentation resources online at:
    .
    For help, type “help”.
    Type “apropos word” to search for commands related to “word”.

    monitor selectcore 0
    continue
    Continuing.
    Note: automatically using hardware breakpoints for read-only addresses.

    Temporary breakpoint 1, main () at ../Sources/main.c:65
    65 PE_low_level_init();

    Program received signal SIGINT, Interrupt.
    prvIdleTask (pvParameters=) at ../Generated_Code/tasks.c:3357
    3357 if( listCURRENT_LIST_LENGTH( &( pxReadyTasksLists[ tskIDLE_PRIORITY ] ) ) > ( UBaseType_t ) 1 )
    /tmp/jenkins/jenkins-GCC-7-build_toolchain_docker-633_20171130_1512067137/src/gdb/gdb/thread.c:93: internal-error: thread_info* inferior_thread(): Assertion `tp’ failed.
    A problem internal to GDB has been detected,
    further debugging may prove unreliable.
    Quit this debugging session? (y or n) [answered Y; input not from terminal]

    This is a bug, please report it. For instructions, see:
    .

    /tmp/jenkins/jenkins-GCC-7-build_toolchain_docker-633_20171130_1512067137/src/gdb/gdb/thread.c:93: internal-error: thread_info* inferior_thread(): Assertion `tp’ failed.
    A problem internal to GDB has been detected,
    further debugging may prove unreliable.
    Create a core file of GDB? (y or n) [answered Y; input not from terminal]

    This application has requested the Runtime to terminate it in an unusual way.
    Please contact the application’s support team for more information.

    Like

    • Just a bit of extra information…

      The above was done on our custom hardware using a K61. It also occurs on a TWR-K60F120M with a project set up the same way.

      It works properly (can set a breakpoint and single-step through the code) on a TWR-K22F120M with a project set up the same way.

      Regards,
      Allen

      Like

  3. Hello, Erich.

    I’m using the MCUXpresso IDE and trying to debug in a remote server with J-Link GDB Server V6.30j up, i’m running into a lot of different errors, sometimes it connects and disconnects right away, sometimes it says “cant get jtag device”, sometimes it shows on the console that it’s still trying to connect localy.

    If possible, could you guide me through the debug set up process?

    Regards,
    Arthur Passos.

    Like

    • Hi Arthur,
      You are already on such a guide for that process (this article). What comes to my mind with what you describe might be a hardware/noise issue. Does it happen with your own custom hardware or with an evaluation board? Other than that, check that the target is powered properly too.

      Like

      • Hi, Erich, thanks for your answer.

        I believe it’s not a hardware issue due to the fact that i can successfully debug with the KDS, my problem is with MCUXpresso.

        For this test, i’m using a MKL16Z128xxx4 processor, and as stated before, it works on KDS but not in the MCUXpresso.

        The remote debug settings in KDS are IP of server and 2331 as port number.

        MCUXpresso doesn’t seem to contain a remote debug settings section, but it does contain a field for an IP, so that’s where i’ve set the server IP with the port appended (2331) to the end

        On the J-Link GDB server it connects but disconnects right away (it’s printed on the console connected and then disconnected), the error i’m getting on MCUXpresso is the following:

        Error in final launch sequence
        Failed to execute MI command:
        -target-select remote localhost:2331

        Error message from debugger back end:
        localhost:2331: No connection could be made because the target machine actively refused it.
        localhost:2331: No connection could be made because the target machine actively refused it.

        The rest of the debugging configurations are set to default (i generated a local one with the quick start menu on the left and made a copy with the IP option selected).

        Regards,
        Arthur Passos.

        Like

        • Hi Arthur,
          ok, good news, your hardware and connection is fine :-).
          I’m wondering why you are modifying/changing the launch configuration for MCUXpresso IDE. The good thing with that IDE is that it creates a working configuration. I suggest you remove/delete your custom launch configuration. And instead during launch of the debug let the IDE detect your attached probe and create the configuration for it. You still can touch it afterwards, but I believe you have some settings wrong, and the IDE created one would be a better base to start with?

          Like

      • Hi, Erich.

        The thing is that even if i’m connected to the remote GDB Server through the GDB commander, MCUXpresso doesn’t find it as an available probe, it only finds my local Jlink GDB server.

        That’s why i felt the need to change the debug configuration, just like i did in the KDS.

        I still can’t put remote debugging to work, sometimes i get an error message coming from the Jlink gdb “unknown emu command #28… “. (the jlink software versions are the same).

        Any ideas?

        Regards,
        Arthur Passos.

        Like

        • This is on Windows, right? I ask because on Mac Segger has changed the name of the GDB server executable. Keep in mind that on Windows KDS is using a local version of the J-Link software inside the IDE folder, while MCUXpresso used the one installed in the ‘program files’ folder. Are you sure you are using the same version?
          >>it only finds my local Jlink GDB server.
          Maybe I’m not getting it, but this sounds like a good thing?

          Like

      • Yes, it’s on Windows.

        The way i’m doing it (at least trying to) right now is as follows:

        1 – I open the JLink remote server (i tried lan and tunnel) in a machine next to me.
        2 – I open Jlink (which is a CLI) and type ip “ip number of the machine next to me”, then hit enter, it prints the following:

        Connecting to 192.168.1.19
        Disconnecting from J-Link…O.K.
        Connecting to J-Link via IP…O.K.
        Firmware: J-Link V9 compiled Mar 29 2018 17:46:13
        Hardware version: V9.20
        S/N: -83602661
        License(s): RDI, GDB, FlashDL, FlashBP, JFlash
        VTref = 3.369V

        3 – Then i go to the MCUXpresso and hit search available probes, it finds only the segger connected to my machine, the one i’m connected through tcp/ ip (at least i think i am, given the above output) is not showing.

        NOTE: All of these JLink softwares were started from the program files/segger… folder.

        ————-

        I’ve also tried to do step 1 and in step 2, instead of connecting using the terminal, i’ve opened JLink GDB Server in my client, checked tcp/ ip and informed the server (machine next to me) ip address.

        When i try this, it prompts me that error message i told you about “unknown #28…” on the server side.

        Like

        • The MCUXpresso IDE will only find locally (USB) attached Segger probes, not networked ones. I suggest that you a) attach a Segger probe with the USB cable b) let it detect by MCUXpresso c) check that it works that way and then d) change it in the created launch configuration to use the TCP/IP version
          I hope that works.

          Like

      • I have tried this already (switch the created configuration from USB to IP). As mentioned before, in the server side it prints the “unknown… #28…” and in the client side it gives me a lot of errors..

        Any ideas?

        Like

  4. Pingback: Adding a Rocktech Capacitive Touch LCD to the NXP i.MX RT1052 EVK | MCU on Eclipse

  5. (Recovered comment, because it was marked by WordPress as spam 😦 )
    Hi Erich,

    maybe you are all old hands at this. Then you are at least going to have a good laugh at what I just found out. But here goes:

    Just one more tip for your readers in regards to the dreaded “CAN’T FIND A SOURCE FILE AT” error when debugging with GDB. I know you already posted a couple of blog entries about this topic, all of which I went through with great interest. But here is a fresh wrinkle. This was at least new to me, and I had searched long and hard for a solution to my problem.

    Unfortunately I seemed to all of a sudden have run into an especially nasty case of the “CAN’T FIND A SOURCE FILE AT” issue. Even though the files were all there, perfectly lined up in plain sight. And yes, the usual places to define the paths to my source files and the whole assortment of project odds and ends were all set up correctly — I triple checked.

    So after hours of experimenting, here is a word of caution, especially when fiddling with the #LINE pre-processor directive.

    But let me recap in chronological order:

    I had just implemented a new error handler for my latest embedded system. It is intended to spit out helpful records like these:

    ::::

    Since the available RAM is an extremely rare commodity in this particular case, I tried to store the records in a compressed format, pushed onto a small stack. A low priority task then pops these off, expanding them into a more user friendly, human readable form before finally firing each one out (via UART or other channels) for further storage/analysis.

    My goal was to try and reduce the amount of memory required to store the various bits of information. The file name plus path was one of them. I happened upon the fact that the C compiler’s pre-processor #LINE directive can be used for exactly that purpose.

    Not only does it accept any line number you wish to assign but it also lets you specify a shorter form of the source file name. You can then access that shorter version via the usual, predefined __FILE__ macro.

    There are numerous places around the web explaining this… even suggesting that it is a great way of controlling what line number and file name will be shown should you ever need ‘more descriptive’ error messages. At least when compared to what the compiler usually does on its own, I guess. For reference, just have a quick look here:

    https://msdn.microsoft.com/en-us/library/b5w2czay.aspx

    Little did I know that this would invite a whole host of problems before I even got to debug my code.

    One of my source files started with the innocuous looking: #LINE 2 “SMB.c” directive. The idea was to dispense with all the path/subdirectory & long form file name, condensing it down to a still descriptive but shorter version. Using the #LINE directive looked far more inviting to me than having to extract just the file name portion myself from the long form path+filename (which is what you get when you use __FILE__ in it’s pristine form).

    Fast forward a few hours now. I will not bore you with the trial and tribulations I went through, looking in all the wrong places why GDB did not seem to find my source files no more.

    Suffice to say that I finally discovered that the filename you specify using the #LINE directive determines the source file name that GDB searches for(!) when trying to load the sources belonging to your project!

    One more thing… at least on Windows I also had to include a ‘\\’ in front of the file name for it to work properly. Otherwise MCUXPRESSO/GDB would mix things up like this:

    “…

    Can’t find a source file at “:\\/”

    …”

    Note the funny looking, lonely forward slash in there! Adding the escaped backslash in front of the name I provided with the #LINE directive solved that last part of the mystery.

    That’s it. Hope this helps. Let me know if MCUXPRESSO behaves the same way on other systems. I ran into this under Win10 with the latest MCUXPRESSO version currently available.

    Cheers,

    Ralf.

    PS: Now if you could just tell me that there is a way to stop & kill a previously started GDB session when you hit the DEBUG icon (which I recall was configurable years ago… in one of the prior incarnations of the IDE) instead of being greeted with the silly: “can not debug since there is already a debug session active”… forcing the user to always remember to manually stop & remove the old session first before hitting the DEBUG button again, then I would be perfectly happy.

    Like

    • Hi Ralf,
      the #LINE directive should be rarely used by custom code: it is useful for code generation tools like assembler or others when the generate code for something else. So no suprise to me that you can confuse gdb with this. the thing with addding ‘\\’ in front of it is new to me, so thanks for that hint.
      And what you see is not specific to MCUXpresso IDE at all: this is specific to gdb (the GNU Debugger), so applies to every debug solution using gdb.

      Like

    • About the ‘stop & kill’ thing: In my view this is really not simple to get it implemented. In Eclipse the IDE and the debugger are really only loosely coupled. So if you start the debugger and there is already a session running: how can you know which one to terminate? Many times I do have multiple debug sessions running, with multiple gdb servers: if it would automatically kill them, that would not be a good thing to me?

      Like

What do you think?

This site uses Akismet to reduce spam. Learn how your comment data is processed.