With my DIY tool chain (see “Constructing a Classroom IDE with Eclipse for ARM“) I get a complete tool chain. I do not need to build that tool chain from the sources for Windows, as all the binaries are nicely pre-compiled and made available. But there is one issue I face from time to time: as the libraries provided by ARM do not come with sources and debug information enabled, I end up with that “No source available for …” message in the debugger:
The solution is to grab the C/C++ library sources from the ARM launchpad site and get it built locally the way I need it.
Building the Libraries on Windows
I’m rebuilding the libraries on my Windows 7 64bit machine. Unfortunately, I have not found any good steps or tutorials how to do this anywhere (that’s why I’m documenting my findings here, otherwise I simply could point to an existing tutorial).While it would (or should?) be possible to re-build all the binaries ARM provided, I’m only interested in building the newlib and newlib-nano libraries, so I can have debug and source information enabled.
Actually it seems that doing things like this on a Windows machine is even more challenging that on Linux: the make files and scripts are all tuned for a Linux system, so to have it built on my machine, some extra steps and tools are needed. Oh well….
Setup of Tools and Environments
The first step is to setup the environment and to download and install the tools and files needed.
Setup of arm-none-eabi Tools
Because I want to build the libraries with the provided arm-none-eabi tools (compiler, linker, etc), I need to make sure they are found by the build process.
π‘ Usually I do *not* want the GNU binaries in my global system path. Setting global environment variables is a road to disaster in my view. With my Eclipse tool environment, I reference the tool chain locally from my IDE/tools folder. Oh well, it looks like at least for building the GNU libraries I have to break with my principles yet again π¦
MinGW and MSYS
The build process is assuming a Linux environment and utilities. To run it on a Windows host, the corresponding environment needs to be installed. This is MinGW (Minimalist GNU for Windows) and MSYS (a set of shell utilities). The build process needs to run from the MinGW shell, and cannot be run from a normal Windows DOS prompt.
Go to http://www.mingw.org/ and download the installer (links on the left hand side):
In the MinGW Installation Manager, make sure these tools are installed (marked with green box):
We need to use the MinGW/MSys shell later on, and it needs to be run from a shortcut. So from the msys.bat, create a short cut on the desktop or in the start menu:
Later I will run the shell from that shortcut:
The documentation about MSYS can be found at http://www.mingw.org/wiki/MSYS
System PATH
I have the path where I have the compiler (arm-none-eabi-gcc.exe) added to the end of my PATH system variable. For my DIY IDE this is
c:\tools\IDE\gcc\bin
arm-none-eabi-cc
The next thing is a weird thing: somehow the library build process is looking for a arm-none-eabi-cc.exe. If not found, it will report an error later in the build process that arm-none-eabi.cc.exe has not been found.
Others have reported this problem too (see http://stackoverflow.com/questions/3660672/getting-error-on-compiling-newlib). To solve that problem, I have copied arm-none-eabi-gcc.exe and named it arm-none-eabi-cc.exe.
Installing the Library Sources
From https://launchpad.net/gcc-arm-embedded, download the source package (at this time it is gcc-arm-none-eabi-4_8-2014q2-20140609-src.tar.bz2), then unpack it.
In this tutorial, I have unpacked the files into a folder named ‘lib_build’ inside my tool chain:
C:\Tools\IDE\gcc\lib_build\gcc-arm-none-eabi-4_8-2014q2-20140609
There are multiple zip files in it, but I only need the newlib-nano-2.1.tar.bz2 and the newlib.tar.bz2 unpacked. I have them unpacked into the ‘src’ folder (see above screenshot).
Patching the Library Build
Unfortunately, the build does not work out of the box (at least under Windows). I have found two issues which need to be fixed to run the build successfully:
- Need to change line endings for MakeInfo in a .texi file
- Fix the make file rules in Makefile.in file
Fix for MakeInfo
On Windows 64bit system there seems to be a long-standing issue that MakeInfo fails with:
Unknown index 'fn' and/or 'cp' in ...
MakeInfo creates documentation, and seem failing to read library info (while it works on Linux). See https://sourceware.org/bugzilla/show_bug.cgi?id=14678 for details on this bug. The problem is very likely because makeinfo has a problem with Windows line endings. Open the file
src\newlib-nano-2.1\etc\standards.texi
in Eclipse and convert the file line delimiters to Unix style, then save the file:
Do the same thing for the newlib file
src\newlib\etc\standards.texi
Patching the Library Makefile.in File
There is a problem with the make file rules in the libgloss part:
*** No rule to make target '../../../../../newlib-nano-2.1/libgloss/arm/../config/default.hm', needed by 'Makefile'. Stop.
The workaround is to patch the ‘Makefile.in’ file:
src\newlib-nano-2.1\libgloss\arm\cpu-init\Makefile.in
The patch details:
From: Agustin Henze <tin@debian.org> Date: Fri, 3 Jan 2014 11:29:55 -0300 Subject: [PATCH] Fix wrong path to libgloss/config/default.mh --- Β libgloss/arm/cpu-init/Makefile.in | 3 ++- Β 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/libgloss/arm/cpu-init/Makefile.in b/libgloss/arm/cpu-init/Makefile.in index 547c58d..d63149f 100644 --- a/libgloss/arm/cpu-init/Makefile.in +++ b/libgloss/arm/cpu-init/Makefile.in @@ -18,6 +18,7 @@ libdir = @libdir@ Β tooldir = $(exec_prefix)/$(target_alias) Β Β objtype = @objtype@ +host_makefile_frag = $(srcdir)/../../config/default.mh Β Β INSTALL = @INSTALL@ Β INSTALL_PROGRAM = @INSTALL_PROGRAM@ @@ -80,7 +81,7 @@ info: Β install-info: Β clean-info: Β -Makefile: Makefile.in ../config.status @host_makefile_frag_path@ +Makefile: Makefile.in ../config.status ${host_makefile_frag_path} Β Β Β Β $(SHELL) ../config.status --file cpu-init/Makefile Β Β ../config.status: ../configure -- 1.8.5.2
Either apply the patch file, or open the Makefile.in in an editor and add:
host_makefile_frag = $(srcdir)/../../config/default.mh
Down further the file, change the Makefile line to:
Makefile: Makefile.in ../config.status ${host_makefile_frag_path}
Makefile.in Patch Part 2
Do the same for this make file in the newlib library sources:
src\newlib\libgloss\arm\cpu-init\Makefile.in
With the two patches applied to both the newlib-nano and newlib, I’m able to build the libraries in the next step.
Building the Libraries
Building each of the libraries (newlib and newlib-nano) is a multi-step process:
- Configure the build
- Run the make file to build
- Run the make file to install/copy the files
The question is: what are the correct parameters to configure the build?
The answer is: have a look at the build-toolchain.sh provided in the source package, and locate the parameters for the newlib-nano and newlib packages:
I have created two shell scripts which will do the necessary calls for me, but you can execute the commands one-by-one on the shell too:
This is the build script for the newlib-nano:
# Script to build newlib and newlib-nano # IMPORTANT: # - make sure the cross-tool chain is present in the PATH # - untar newlib-nano-2.1.tar.tar.bz2 # - untar newlib.tar.bz2 TARGET=arm-none-eabi ROOT=/C/tools/IDE/gcc/lib_build/ BUILDDIR_NATIVE=$ROOT/install-native.newLibnano echo Creating build directory... mkdir newlib-nano.build echo Entering build directory... cd newlib-nano.build # run configure program to configure the build echo Configuring newlib-nano build... ../gcc-arm-none-eabi-4_8-2014q2-20140609/src/newlib-nano-2.1/configure \ --target=$TARGET \ --prefix=$BUILDDIR_NATIVE/target-libs \ --disable-newlib-supplied-syscalls \ --enable-newlib-reent-small \ --disable-newlib-fvwrite-in-streamio \ --disable-newlib-fseek-optimization \ --disable-newlib-wide-orient \ --enable-newlib-nano-malloc \ --disable-newlib-unbuf-stream-opt \ --enable-lite-exit \ --enable-newlib-global-atexit \ --disable-nls echo Building newlib-nano... make -j echo Installing... make installAnd the same for the newlib:
# Script to build newlib # IMPORTANT: # - make sure the cross-tool chain is present in the PATH # - untar newlib.tar.bz2 TARGET=arm-none-eabi ROOT=/C/tools INSTALLDIR_NATIVE=$ROOT/install-native.newLib INSTALLDIR_NATIVE_DOC=$INSTALLDIR_NATIVE/share/doc/gcc-arm-none-eabi echo Creating build directory... mkdir newlib.build echo Entering build directory... cd newlib.build # run configure program to configure the build echo Configuring newlib build... ../gcc-arm-none-eabi-4_8-2014q2-20140609/src/newlib/configure \ --target=$TARGET \ --prefix=$INSTALLDIR_NATIVE \ --infodir=$INSTALLDIR_NATIVE_DOC/info \ --mandir=$INSTALLDIR_NATIVE_DOC/man \ --htmldir=$INSTALLDIR_NATIVE_DOC/html \ --pdfdir=$INSTALLDIR_NATIVE_DOC/pdf \ --enable-newlib-io-long-long \ --enable-newlib-register-fini \ --disable-newlib-supplied-syscalls \ --disable-nls #echo Building newlib-nano... make -j #echo Installing... make installTo build the libraries, launch the msys.bat with the shortcut previously created, and cd (change directory) into the folder we have unpacked the source package and run the above shell scripts, or execute the commands on the command line.
If doing it directly on the command line, I show it here for the newlib-nano.
Create a folder ‘newlib-nano.build’
mkdir newlib-nano.buildCd (change directory) into that folder.
cd newlib-nano.buildRun the configure command:
../newlib-nano-2.1/configure --target=arm-none-eabi --prefix=/C/tools/newlib-nano-arm-none-eabi --disable-newlib-supplied-syscalls --enable-newlib-reent-small --disable-newlib-fvwrite-in-streamio --disable-newlib-fseek-optimization --disable-newlib-wide-orient --enable-newlib-nano-malloc --disable-newlib-unbuf-stream-opt --enable-lite-exit --enable-newlib-global-atexit --disable-nlsThen do a make witih
make -jand finally install it with
make installThe needed configuration options can be taken from the ‘build-toolchain.sh’ present in the root of the source zip file.
The build is then started with
make -jThe build process takes about 30 minutes.
If finished successfully, use
make installTo ‘install’ the libraries into the folder specifed with –prefix in the configuration settings.
Assembling the Libraries
Now I have the two libraries built:
- newlib-nano in lib_build\install-native.newLibnano\target-libs\arm-none-eabi\lib
- newlib in lib_build\install-native.newLib\arm-none-eabi\lib
The newlib-nano libs have a *_s.a extension, so I rename them with a shell script:
NEWLIB_NANO_LIB=./install-native.newLibnano/target-libs/arm-none-eabi/lib INST_LIB=./build.install/arm-none-eabi/lib echo Creating directories mkdir ./build.install cd ./build.install mkdir ./arm-none-eabi cd ./arm-none-eabi mkdir ./lib cd ./lib mkdir ./armv6-m mkdir ./armv7-ar mkdir ./armv7e-m mkdir ./armv7-m mkdir ./cpu-init mkdir ./fpu mkdir ./thumb cd ../.. echo Rename newlib-nano files mv $NEWLIB_NANO_LIB/libc.a $(NEWLIB_NANO_LIB)/libc_s.a mv $NEWLIB_NANO_LIB/libg.a $NEWLIB_NANO_LIB/libg_s.a mv $NEWLIB_NANO_LIB/armv6-m/libc.a $(NEWLIB_NANO_LIB)/armv6-m/libc_s.a mv $NEWLIB_NANO_LIB/armv6-m/libg.a $NEWLIB_NANO_LIB/armv6-m/libg_s.a mv $NEWLIB_NANO_LIB/armv7-ar/libc.a $NEWLIB_NANO_LIB/armv7-ar/libc_s.a mv $NEWLIB_NANO_LIB/armv7-ar/libg.a $NEWLIB_NANO_LIB/armv7-ar/libg_s.a mv $NEWLIB_NANO_LIB/armv7e-m/libc.a $NEWLIB_NANO_LIB/armv7e-m/libc_s.a mv $NEWLIB_NANO_LIB/armv7e-m/libg.a $NEWLIB_NANO_LIB/armv7e-m/libg_s.a mv $NEWLIB_NANO_LIB/armv7-m/libc.a $NEWLIB_NANO_LIB/armv7-m/libc_s.a mv $NEWLIB_NANO_LIB/armv7-m/libg.a $NEWLIB_NANO_LIB/armv7-m/libg_s.a mv $NEWLIB_NANO_LIB/fpu/libc.a $NEWLIB_NANO_LIB/fpu/libc_s.a mv $NEWLIB_NANO_LIB/fpu/libg.a $NEWLIB_NANO_LIB/fpu/libg_s.a mv $NEWLIB_NANO_LIB/thumb/libc.a $NEWLIB_NANO_LIB/thumb/libc_s.a mv $NEWLIB_NANO_LIB/thumb/libg.a $NEWLIB_NANO_LIB/thumb/libg_s.a
And then move the newlib libraries over the newlib-nano libraries (merge):
mv -r $NEW_LIB/* $NEWLIB_NANO_LIB/*
With this, I have a new ‘lib’ folder I can use. I rename my original gcc/arm-none-eabi/lib one to lib.orig and copy my new library folder:
With this, I have my new library in place, and can go back to the original state if necessary.
Debugging the Library with Source Code
The the benifit of all the work: I can now debug and step through the library code with source files displayed:
π π π
Summary
It takes some time and knowledge to build the newlib and newlib-nano libraries from the sources. But doing so, I can build the libraries with debug information. What is missing now is a good way to provide my custom optimization options for the library build, and an automated way/script to build my set of library files.
Having the ability to re-build the library sources the way I want is a huge advantage of using open source libraries and tools. Yet another reason the swap out proprietary libraries as shown in this post :-).
Happy Libraring π
http://sourceforge.net/projects/bleeding-edge/files/140708/
There you can find compiled toolchain for arm with no striped library with latest packages.
Then You only need show tell eclipse where the source code is;).
LikeLike
Hi Radek,
I knew Freddie Chopin, but I missed the fact that he published that bleeding edge toolchain and libraries. Thanks for that link!
Erich
LikeLike
Hi,
You don’t handle the include files, and the one delivered with KDS are different from the one which is downloaded in this paper. One more thing is that newlib.h differs in newlib and newlib-nano directories….
LikeLike
Yes, the KDS tools and libraries are different from the GNU ARM Embedded ones, so for KDS I have swapped it out: https://mcuoneclipse.com/2014/07/11/switching-arm-gnu-tool-chain-and-libraries-in-kinetis-design-studio/
LikeLike
I’m trying to do this same thing with the latest distribution (4_9-2015q2-20150609) and I’m having a few issues. First of all, there does not appear to be separate zip files for newlib and newlib-nano, but the release notes seem to indicate that both are included. Do I just unzip the newlib.tar.bz and make a copy of it for newlib-nano? Also when making the libraries I’m getting errors (see below). Any advice?
../../../../../../gcc-arm-none-eabi-4_9-2015q2-20150609/src/newlib/newlib/libc/s
earch/hash.c:277:1: internal compiler error: Segmentation fault
}
^
Please submit a full bug report,
with preprocessed source if appropriate.
See for instructions.
make[8]: *** [lib_a-hash.o] Error 1
make[8]: Leaving directory `/c/tools/IDE/gcc/lib_build/newlib.build/arm-none-eab
i/thumb/newlib/libc/search’
make[7]: *** [install-recursive] Error 1
make[7]: Leaving directory `/c/tools/IDE/gcc/lib_build/newlib.build/arm-none-eab
i/thumb/newlib/libc’
make[6]: *** [install-recursive] Error 1
make[6]: Leaving directory `/c/tools/IDE/gcc/lib_build/newlib.build/arm-none-eab
i/thumb/newlib’
make[5]: *** [multi-do] Error 1
make[5]: Leaving directory `/c/tools/IDE/gcc/lib_build/newlib.build/arm-none-eab
i/newlib’
make[4]: *** [install-data-local] Error 2
make[4]: Leaving directory `/c/tools/IDE/gcc/lib_build/newlib.build/arm-none-eab
i/newlib’
make[3]: *** [install-am] Error 2
make[3]: Leaving directory `/c/tools/IDE/gcc/lib_build/newlib.build/arm-none-eab
i/newlib’
make[2]: *** [install-recursive] Error 1
make[2]: Leaving directory `/c/tools/IDE/gcc/lib_build/newlib.build/arm-none-eab
i/newlib’
make[1]: *** [install-target-newlib] Error 2
make[1]: Leaving directory `/c/tools/IDE/gcc/lib_build/newlib.build’
make: *** [install] Error 2
LikeLike
Hi Doug,
I have not done it with the 4.9q2-2015 distribution yet, but your findings are not encouraging π¦
>Do I just unzip the newlib.tar.bz and make a copy of it for newlib-nano?
Hmm, no, that won’t work, as newlib does not include the newlib-nano implemenation.
>>../../../../../../gcc-arm-none-eabi-4_9-2015q2-20150609/src/newlib/newlib/libc/search/hash.c:277:1: internal compiler error: Segmentation fault
That one is VERY bad! It means that the compiler crashed π¦
I suggest that you go ahead and report a bug on https://bugs.launchpad.net/gcc-arm-embedded/+filebug
LikeLike
Pingback: Using FreeRTOS with newlib and newlib-nano | MCU on Eclipse
Hey Erich,
I just ran through this today, in an attempt to fix my problems with _REENT_SMALL in MCUXpresso 10.2.0. I can say that since you wrote this things have gotten slightly better – I used q4-2014 and didn’t have to do any of those patches.
The biggest problem I ran into is that for some reason it was using a version of newlib.h that was an empty placeholder, not the configured one, so it was throwing errors about LDBL_EQ_DBL because it had the wrong floating point setup. I deleted the file and it built successfully. For some reason make -j kept quitting when there didn’t seem to be an error, but make completed. Maybe a dependency problem with the parallel build?
Now my problem is that they took out (or hid) some of the options I’m looking for in MCUX and I’m not sure I’ve got it set up right. I’m getting a bunch of undefined references to intrinsic floating point functions like __aeabi_ddiv. They all seem to be related to doubles so I’m guessing this is where it uses software emulation to fill in for the single-precision FPU on the Cortex M4F. I’m not sure if I’ve failed to build the library properly, or if I need to do something else to link in the emulated floating point functions. I’m also missing __errno() and I’m not sure where that is supposed to come from.
Any ideas?
LikeLike
Oops, typo – it’s q4-2017, the current version.
LikeLike
Figured it out after a night’s sleep… I don’t know exactly what MCUX is doing with the GUI configuration for libraries and library paths, but setting my libraries in the .ld file did the trick:
GROUP (
“libgcc.a”
“libc_nano.a”
“libm.a”
)
As a side effect, it did help me locate several places in my code where I thought I was using all single-precision floating point but had some doubles slip in. That ought to speed up my Goertzel filter implementation considerably – I’ll have to benchmark the improvement.
And now I have sources for debugging the standard libraries! Thanks for another great post.
LikeLike
Oh, that’s great! I saw your questions earlier today, but I have been so busy that I had to save this off for tonight. So looks like your are all set and up and running, congratulations!
The libraries should have been included by default by the internals of gcc/ld, but looks like such a group might be necessary, thanks for that hint!
LikeLike
Stupid question maybe, but how did you get Eclipse to actually use the source? When I built newlib I was mostly interested in reconfiguring it and wasn’t messing with the debugger. It built fine, but adding the path to the debugger’s source paths doesn’t seem to do anything.
Also, when you say that the libraries should have been included by gcc and ld, where do they get that from? Do they expect an environment variable?
LikeLike
First: Congratulations, you posted the 20’000 comment to this blog π π π
About your library: It has been a while when I did a rebuild (I did not do that recently), but because I kept the debug information, the library included the path to the files where the sources are, so the debugger has found the files. I had no need to add a path.
LikeLike
Woo hoo! I don’t think more than 1,000 of those posts were mine. π
I don’t know what the heck is going on today. Looks like the library is fine, though. I’m working on porting a project from CW11 to MCUX and setting it up for newlib. I discovered that this new project *does* show source for newlib, but I can’t figure out what’s different about the projects. The non-working one shows the newlib source path in ‘Source Lookup Path’ in the debug configuration (as does the working one) and that entry isn’t editable, so I know it knows it’s supposed to have source there.
Everything about this has been a tough slog. I was determined to set this project up for the MCUX SDK so I could start migrating to that, but I discovered the pins configuration tool won’t work on this install. It works on another one across the room, though. And for some reason the __stack symbol needed by Crt0.s was coming up 0 (presumably because of Crt0.s’s own weak declaration) until I added a dummy reference elsewhere – and this despite the fact that the other project has virtually the same map file and exactly the same startup code.
Anyway, there seems to be nothing wrong with your instructions. I’ll just keep at it now that I have a working version to compare it against. Thanks!
LikeLike
Pingback: Stack Canaries with GCC: Checking for Stack Overflow at Runtime | MCU on Eclipse
Pingback: assert(), __FILE__, Path and other cool GNU gcc Tricks to be aware of | MCU on Eclipse