Solving the 8192 Character Command Line Limit on Windows

There is a really annoying issue with using command line tools on Windows: the maximum length of the command line passed to cmd.exe is 8192 characters (see http://blogs.msdn.com/b/oldnewthing/archive/2003/12/10/56028.aspx). So you think this is not a problem for you, as you would not pass such a long command line to cmd.exe (the DOS shell under Windows)? Well, if you are using Eclipse (as I do) which generates make files (which is the normal way), then the cmd.exe very likely is involved to call the compiler/linker/etc, indirectly with the usage of make.exe. Compiling files is usually not a problem as it does not hit that 8192 limit. However, it is likely that link phase will end up with an error:

Error in the Problems View

Error in the Problems View

If you have such a problem, there is a solution ….

Failure to Link?

Looking at the Console output, it is clearly the link phase which failed:

Linker failed because of 8192 character limit

Linker failed because of 8192 character limit

So what happened? I’m using source files with a rather long directory path, and many of them (e.g. building the CMSIS library, see “Tutorial: Using the ARM CMSIS Library“). Because that library/project has many, many files which need to be linked, it will fail at the link phase. In my above test example I have a command line length passed to the linker which is 12316 characters long!

The error might be a strange one, maybe saying something like it cannot find or execute an object file. Because the command line gets cut somewhere, the error message reported usually does not give a clear clue.

Command Line Length Limit

The problem is not the problem of Eclipse: it is rather the problem of make how much it accepts on the command line and long the command line to the gcc compiler and linker can be. The problem can be easly reproduce with my DIY toolchain and is present in other vendor toolchains like the Freescale Kinetis Design Studio (tried with v2.0.0) on Windows.

💡 CodeWarrior for MCU solves that problem with a custom build tools integration: instead of passing the full set of arguments on the command line, it uses files with the options/commands in it. So it passes the options to the called process with pointing to a file. Unfortunately this is not present in the widely used GNU ARM Eclipse plugins (see support request #25 and support request #32).

Oh yeah, I hear now already the voices singing: “Dude, it’s your fault, why are you using Windows? Use Linux!” because this problem does not exist on Linux. Oh well, do not get me there 😉

The obvious solution is to keep the command line as short as possible, especially for the linker:

  1. Use short directory names
  2. Use short file names
  3. Reduce the command line length with using relative paths instead of absolute file paths
  4. If this does not help: build part of the application as a library and then link the library with the final application

Needless to say that this is all very time consuming and just a workaround for the problem. The solution is to use the GNU ARM Eclipse Build Tools package instead.

GNU ARM Eclipse Build Tools

Luckily, there is now a solution available for Windows, thanks to the Liviu and the GNU ARM Eclipse work :-). The solution is available from the version v2.6.1 or later of the GNU ARM Eclipse Plugins build tools, and the release notes on http://gnuarmeclipse.github.io/blog/2015/02/28/plugins-v2.6.1-201502281154-released/ reads:

“The new Build Tools v2.3 include a better shell version, instead of the Windows cmd.exe, so the annoying 8K limit on command line lengths was removed.”

Oh, wow! So this is the solution is to use these build tools, and then that 8192 character command line limit should be gone? Yes, it is :-)!

Installing the Solution

The latest build (v2.4 at the time of this writing) is available from https://sourceforge.net/projects/gnuarmeclipse/files/Build%20Tools/.

GNU ARM Eclipse Build Tools

GNU ARM Eclipse Build Tools

Download the setup executable and run it.

GNU ARM Eclipse Build Tools Installer

GNU ARM Eclipse Build Tools Installer

On my machine, the build tools get installed into

C:\Program Files (x86)\GNU ARM Eclipse\Build Tools\2.4-201503242026\bin
GNU ARM Eclipse Build Tools Installed

GNU ARM Eclipse Build Tools Installed

Now make sure that these tools are used instead of the ones with the 8192 command line character limit. For Kinetis Design Studio v2.0.0, the build tools are located in these sub folders:

  1. make.exe is in <kds_2.0.0>\toolchain\bin
  2. echo.exe and rm.exe are in <kds_2.0.0>\bin

Rename the existing make.exe, rm.exe and echo.exe so you have a backup. Then copy the files from the GNU ARM Eclipse build tools into the KDS bin folder:

GNU ARM Eclipse Build Tools in KDS

GNU ARM Eclipse Build Tools in KDS

Because I have renamed the existing make.exe, rm.exe and echo.exe, they are not found any more and the new versions are used instead with the sh.exe which fixes that annoying Windows problem. And if I do a build now with my long command line, it does works :-):

Success building with GNU and very long command line

Success building with GNU and very long command line

Summary

Windows (better, cmd.exe) has a command line length limit of 8192 characters. The GNU ARM Eclipse build tools provided by Liviu (see his site on http://gnuarmeclipse.livius.net/) have fixed this with a new sh.exe. Using the GNU ARM Eclipse build tools allowed me overcome that Windows limit. The fix is easy to apply: replace the limited cmd.exe based tools with teh GNU ARM Eclipse ones which use the sh.exe.

Happy Shelling 🙂

PS: Liviu, you probably read this, and I know you are using Mac OS, and you are not a fan of Windows environment for good reasons. On behalf of the Windows user community: THANK YOU, THANK YOU, THANK YOU!

Advertisements

65 thoughts on “Solving the 8192 Character Command Line Limit on Windows

    • Hi Liviu

      I did changed the exe’s at path
      C:\Freescale\KDS_3.0.0\bin

      but I still get the same problem, “no such file or directory”
      if I move that file to different directory where the directory path character reduces, it works fine.

      am i missing anything here.

      Like

      • I’m not sure I can follow you. what path did you change?

        I don’t know in KDS, but in the original GNU ARM Eclipse plug-ins, you need to install the latest Windows Build Tools and then tell the plug-ins where these tools are. once you do this, the Windows cmd.exe 8192 chars limitation should no longer be a problem, since my package includes not only make.exe, but also sh.exe, so no need for the limited cmd.exe.

        Like

      • I downloaded “gnuarmeclipse-build-tools-win32-2.6-201507152002-setup.exe” and installed it in windows.

        now I have gnu arm at below path
        C:\Program Files (x86)\GNU ARM Eclipse

        I copied
        make.exe, rm.exe, busybox.exe, echo.exe, sh.exe, intl.dll
        from below path
        C:\Program Files (x86)\GNU ARM Eclipse\Build Tools\2.6-201507152002\bin

        to KDS directory

        C:\Freescale\KDS_3.0.0\bin

        Like

      • I don’t know how KDS manages the PATH, apparently it should work, but it looks like we are missing something. 😦

        Like

  1. One small comment: the latest version of the managed build plug-in also adds a mechanism to manage the path of the build tools (see Preferences -> C/C++ -> Build -> Global Tools Paths and Workspace Tools Paths). If you run the build tools setup, regardless where you decide to install the package, there are some keys written to the Windows Registry, and the plug-in will automatically add the correct path to the build path.

    All recent GNU ARM Eclipse tools packages (Build Tools, OpenOCD, and the under development QEMU) allow to install multiple versions of the same package, and the Windows Registry keys (or some folder search on other platforms) are able to easily identify the tools location, and finally adjust the plug-in configuration, either automatically or with minimal effort.

    I don’t know how this fits into the KDS current structure, but for plain GNU ARM Eclipse installations, I highly recommend to use this mechanism.

    Like

    • Hi Liviu,
      at least KDS v2.0.0 adds the toolchain and bin path in front of the PATH variable, so is not as flexible as your generic approach. I agree that modification of the PATH is not needed and not something which should be done, and this should be removed.

      Like

    • The GNU ARM Eclipse managed build plug-in uses the code from CDT to generate the make files, and this specific CDT code is not particularly friendly with plug-ins that want to extend it. In practical terms, extending is possible only by completely rewriting the classes, which is not practical, since keeping in sync with future CDT versions becomes too manually intensive.

      So… ‘no can do’… 😦

      Like

  2. Hey guys, looks like “clearn project” (rm) is still crapping on when there are a lot of .o files.

    rm -rf ./Source/Generated_Code/CD.o ./Source/Generated_Code/CS.o ./Source/Generated_Code/Events.o ./Source/Generated_Code/LCDPort.o
    ****** CLIPPED TONS OF OUTPUT HERE *****
    ./Source/Config/LCD_X_8080_8.d libSegger-GUI.a, …) failed.
    make (e=87): The parameter is incorrect.

    Like

  3. Make isn’t working for me either:

    arm-none-eabi-ar -r “libemWin_V528b.a” ./Sources/Generated_Code/CD.o ./Sources/Generated_Code/CS.o ./Sources/Generated_Code/Events.o ./Sources/Generated_Code/LCDPort.o ./Sources/Generated_Code/POR_DIS.o ./Sources/Generated_Code/RD.o ./Sources/Generated_Code/RST.o
    ..
    ..
    ..
    ./Sources/GUI/ConvertMono/GUICC_8.o ./Sources/Config/GUIConf.o ./Sources/Config/GUI_X.o ./Sources/Config/LCDConf.o ./Sources/Config/LCD_X_8080_8.o, …) failed.
    make (e=87): The parameter is incorrect.

    make: *** [libemWin_V528b.a] Error 87

    Like

    • Hi Mike,
      are you sure that the new make is used? Have you removed/renamed the one in the toolchain/bin? I think that on your machine still the ‘old’ tools are called.

      Like

      • Hi guys!

        With your solution Eric, now i am able to build my project, but the clean function does not work. I did everything exactly as you described, and i used the latest GNU ARM Eclipse Build Tools.

        Maybe do you have any other idea, what can be the problem?

        Thanks for any advice in advance.

        Bye,
        Steven

        Like

      • I’m also running into a problem with clean (others have mentioned it above as well). If I manually delete the debug folder, I can do a build from scratch no problem, but clean fails. I’ve tried copying all of the build tools into the proper folder, and I’ve also tried pointing to them in an external location using the “Tools Path” feature under project properties. No matter what I do I get the following error:

        make (e=87): The parameter is incorrect.

        make: [clean] Error 87 (ignored)
        makefile:197: recipe for target ‘clean’ failed

        But, if I manually edit the makefile and remove some of the .mk includes, clean works again.

        Any ideas?

        Like

      • Hi Andrew,
        I had that problem only once or twice, and it disappeared (not sure why).
        Could it be that you have another rm present in your PATH settings (on system level)?

        Erich

        Like

      • My Windows system path has only “C:\Windows\system32” in it (no rm in there). In my project settings PATH variable, the path to the GNU ARM Eclipse build tools folder is the first thing listed.

        It looks like the right RM is being called:

        process_begin: CreateProcess(C:\Freescale\2.6-201507152002\bin\rm.exe, rm -rf V2_MQX_PEx.siz

        But before it even gets to that, I get the error:

        makefile:197: recipe for target ‘clean’ failed

        Apparently this is the line where make fails:

        -$(RM) $(SECONDARY_SIZE)$(C++_DEPS)$(OBJS)$(C_DEPS)$(ASM_DEPS)$(CC_DEPS)$(CPP_DEPS)$(CXX_DEPS)$(C_UPPER_DEPS)$(S_UPPER_DEPS) V2_MQX_PEx.elf

        Like

      • One more thing,

        I tried changing the RM line in the makefile and split it up like this:

        clean:
        -$(RM) $(SECONDARY_SIZE)
        -$(RM) $(C++_DEPS)$(OBJS)
        -$(RM) $(C_DEPS)
        -$(RM) $(ASM_DEPS)
        -$(RM) $(CC_DEPS)
        -$(RM) $(CPP_DEPS)
        -$(RM) $(CXX_DEPS)
        -$(RM) $(C_UPPER_DEPS)
        -$(RM) $(S_UPPER_DEPS)
        -$(RM) V2_MQX_PEx.elf

        It works fine now. This is not a great solution going forward, but it does hint at a lingering character limit issue.

        Like

      • Hi Andrew,
        I come to the conclusion that the make (http://gnuarmeclipse.github.io/windows-build-tools/releases/) is able to deal with longer arguments (>8192), but the rm command still fails with that long command line. So the easy workaround is to delete the output folder (e.g. ‘debug’) instead let rm remove all object files. I have filed a bug report (https://github.com/gnuarmeclipse/windows-build-tools/issues/3), be free to add any more details.
        thanks!

        Like

      • Eric,

        Thanks for your help with this, I will followup with the bug report if I have anything more to add.

        Thanks again!

        Like

      • Hi Liviu,

        Yes, I installed the build tools using setup, renamed the old ones, and copied the new ones over exactly as described in the instructions.

        Yes, make.exe and sh.exe are in C:/Freescale/KDS_2.0.0/bin

        I think I may know the problem: I am building a static library, not an executable, so I’m calling the archiver not linker.

        Have you tried your new build tools for static libraries?

        Like

      • Also this message is displayed over and over:

        c:/freescale/kds_2.0.0/eclipse/../bin/sh: echo.: not found

        My global toolchain path is set to:

        ${eclipse_home}/../toolchain/bin

        Is that correct?

        Like

      • if the new echo.exe is present in the bin folder, than you have a path problem. you should use all new executables.

        personally I would recommend to leave them where the setup installed them, remove them from any other location, and configure the global build tools path there. (the preferences windows to configure the build tools path is available only in the latest plug-in version, you might need to update).

        Like

      • Hi Liviu,

        I have installed the latest updates for KDS, is that what you’re talking about?

        How do I change the build tools path?

        Thanks for your help

        Like

  4. Pingback: Comparing CodeWarrior with Kinetis Design Studio | MCU on Eclipse

  5. This is super helpful, thank you for sharing. It worked great with KDS 2.0, and I am currently trying it out with KDS 3.0. Seems to be fine there, too!

    Like

      • Yes, this should work for any Eclipse build plug-ins, not only the GNU ARM Eclipse managed builds, as long as the **sh.exe** executable is available in the same folder as **make.exe** and the folder is named **bin**.

        Like

  6. I have the same problem with these tools (I’ve installed gnuarmeclipse-build-tools-win32-2.4-201503242026-setup.exe):

    ~/alexandruc_win_project $ echo $SHELL
    /bin/sh
    ~/alexandruc_win_project $ which make
    C:/Program Files (x86)/GNU ARM Eclipse/Build Tools/2.4-201503242026/bin/make.exe
    ~/alexandruc_win_project $ make
    Building file: ../Inventory/Inventory_CeraOS.cpp
    Invoking: GCC C++ Compiler
    ….
    The command line is too long.

    Am I doing something wrong ?

    Like

  7. I ran into one issue after installing the new build tools – the new make only accepts forward slashes (/), not back slashes (\). If you have build variables defined in your KDS project you will have to edit them and change “\” to “/”.
    Installation fixed the linker issue but would not build a clean project.

    Like

    • Good point! I have not been running into this as for a long time I started to use forward slashes in my project settings whenever possible to keep things in sync with Linux. And the wizard in Eclipse seems to use forward slashes whenever possible too.

      Like

  8. Hi, I am running KDS3.0 on win8. I ran into this building error and downloaded 64 bits version.
    It always crushed when I try to build project. Any solution for that??

    Like

  9. Followed the instructions to the letter. Now it looks like anything I try to do with make – either clean or build, simply doesn’t do anything. Not sure what is missing here but this is not working out as a solution. I am on KDS 2.0.0 and downloaded GNU ARM Eclipse Build Tools 2.6-201507152002 😦

    Like

    • I don’t know why it should not work with KDS v2.0.0. But v2.0.0 is very old. Is there a reason why not using v3.0.0? V3.0.0 comes with a much better GNU tool. I recommend that you install 3.0.0 (you can keep 2.0.0 installed, just in case).

      Like

      • Very simple. We are nearing production and it is late in the game to make a tool change unless absolutely necessary. it would mandate a lot of regression testing to confirm the new tools have not introduced new problems. (Tool updates are often a trade off between different bugs, as opposed to the elimination of all bugs, but your mileage may vary.)

        Like

      • Generally each GNU ARM Eclipse binary packages installs in a separate folder, exactly to allow strict environments to return to a desired version in case new ones break any compatibility.

        Like

  10. Hi I’ve just tried with KDS3.0 & SDK 1.3.0: it works, but you have to copy ALL files into “C:\Freescale\KDS_3.0.0\bin” (and not only echo.exe, rm.exe and make.exe), othervise it doesn’t compile/link.
    However great job,
    thanks so much

    Like

  11. Pingback: NXP Kinetis Design Studio v3.2.0 | MCU on Eclipse

  12. After making the changes and not using Command Line, now I have another issue with createProcessFunction() in Windows which has limitation of 32K characters.

    I change the builder to external builder and selected the installed tools path in Builder Path Configuration.

    Am I doing something wrong?

    If no, is there a patch around it.

    Like

    • I’m using External Builder setting too, and it works fine. Are you using Kinetis Design Studio V3.2.0? If not, you can give that one a try as it comes with the binaries included and works fine for me. Except that the ‘clean’ operation does not work with >8192 characters.

      Like

  13. Pingback: Tips for Making Copy of Eclipse CDT Projects Easier | MCU on Eclipse

What do you think?

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s