Eclipse Project ‘Refresh Policy’: Broken Incremental Build with External Make?

So here is a problem I have observed for a while, and several users have reported that to me: the problem is that for some reasons and only for some projects, when I do a ‘build’ in Eclipse CDT, it does a ‘full’ build. It does not only build the changed files, it builds all or many files in the project.

Building

Building

I expect that only the changed files will be rebuild. Instead, it compiles many files it should not compile/build, with the result that a build takes a very long time. That’s clearly wrong, and not what I expect. So what is causing this problem?

The issue is present in multiple Eclipse version (Kepler, Luna), with different Eclipse distributions (Kinetis Design Studio v2.0.0 and v3.0.0) with different make utilities. In all my projects I’m using the GNU ARM Eclipse plugins, but I doubt that this is the problem. I believe it might be more of a CDT problem instead.

The first observation was: The issue seems not to happen for ‘fresh’ projects, but happens for ‘older’ projects or some projects I have received from other users. It is even more strange, that the number of files unnecessary recompiled seems to be kind of random: sometimes it just triggers a new linking phase, sometimes it seems to affect only one or a few files, sometimes it seems to rebuild pretty much all of them. And things might even change depending on which files I change.

File Dates?

First I thought there might be something wrong with the file dates, as make needs to rebuild something when a source file is newer than the object file, or if the header file is newer than the source file using it. I had once a machine where the host clock was going wild, resulting in strange file time stamps. But on my current machine everything was OK.

Debugging Make Files?

If the files are not newer, maybe some do not exist? Checking them on the file system, object files all where there, all with newer file dates than the source files. So I started to see what kind of debug information I can get from make. Make has a –debug option (see https://www.gnu.org/software/make/manual/html_node/Options-Summary.html) which is very useful for debugging make files.

Running Make with Debug Logs

Running Make with Debug Logs

The only thing I found was that somehow some object files do not exist:

Reading makefiles...
Updating goal targets....
 File 'all' does not exist.
  File 'FRDM-K64F_Demo.elf' does not exist.
   File 'Sources/Joystick.o' does not exist.
  Must remake target 'Sources/Joystick.o'.
Building file: ../Sources/Joystick.c

But they *do* exist before running the build! Strange. So something is deleting them, and somehow it must be the build process? But I have not seen any ‘rm’ or ‘del’ in the make logs.

Build Automatically?

Then, I thought it might be related to ‘Build Automatically’ setting:

build automatically

build automatically

‘Build automatically’ actually does not make any difference, as this functionality seems not be implemented in CDT: that would allow that the system would automatically build my application after I have changed and saved my file. So this was a Red Herring :-(.

External or Internal Builder?

What I noticed was that using the Eclipse Internal Builder does not have this problem. The problem only happened with using the external builder (external make):

External and Internal Builder

External and Internal Builder

With the internal builder, Eclipse uses its own built-in make utility, while with the external builder the build process uses an external make. So I thought that I have found a problem in make: Using different make executables did not make any difference :-(.

Refresh Policy!

Actually, that ‘external builder’ was triggering something. After poking around this issue for a few days, finally I received a tip from Rock Li (thanks!): The issue seems to be related to the Refresh Policy setting in the project, which only affects external builders: If the refresh is set for an non-existing (or wrong directory), other than the project root directory itself, these strange rebuilds of files are happening:

Mismatch in Refresh Policy

Mismatch in Refresh Policy

And to my surprise: Deleting that wrong directory and using the project folder instead fixed the issue!

Matching Refresh Policy Folder

Matching Refresh Policy Folder

So the question is: what is this ‘Refresh Policy’ used for? As indicated in the dialog itself, it is only used if using an external builder (which is the case for my project). That refresh is executed after the build process, to make sure the Eclipse project knows about all the new files (object files, make files, etc) generated.  But if that setting is wrong, wrong things might happen :-(.

And now I realize why I had some strange things with some projects, and not with others. That above project I had probably created as FRDM_K64F_Test, and then later renamed to FRDM_K64F_Demo, but that Refresh Policy setting did not get updated.

Summary

If your external make build in Eclipse CDT is taking too long and building too many files, then check what is specified in the Project settings, under Refresh Policy. If you have a wrong folder listed there, then this is very likely causing that problem.

I’m not sure if this is a problem of CDT, make or anything else. I just notice if I set the folder to the correct one, the issue goes away :-). If anyone has an explanation of what is happening, please comment :-).

Happy Building 🙂

Advertisements

31 thoughts on “Eclipse Project ‘Refresh Policy’: Broken Incremental Build with External Make?

  1. I’ve noticed on Eclipse 3.8 that sometimes cross project references are not refreshing properly. This is a bug. Just manually Refreshing the project and also the associated projects seems to fix this issue.

    Probably your issue is related to this problem.

    I’ve also found that library projrcts don’t automatically have their directory links updated when the project name is changed. Sometimes I have to dive into the XML configs to delete dead references. Another bug.

    But still prefer to use Eclipse 100x more than any proprietary Windows based platform.

    Like

    • Yes, it could be related to the same issue. But I’m still not clear why with the wrong refresh the make utility somehow has the object files deleted in my case. Really strange.

      Like

  2. Thanks Erich,

    I got a problem on some imported project that looks like your subject, but it was here Eclipse that didn’t see modified files. I solved the problem by delete debug directory…
    Your paper solve my problem.

    Like

  3. Hi Erich, great find! I expect the most common cause of the Resource name being incorrect is because the project has been renamed and the resource name has not been updated. Have you found an upstream (maybe Eclipse Platform or CDT) bug or filed one?
    cheers john

    Like

  4. Pingback: Kinetis Drone: Sensor Fusion Toolbox | MCU on Eclipse

  5. Hi Erich,

    I’ve had this issue on a project where there was no folder in refresh policy. Adding the folder was not sufficient. I have solved it by building 3 times:
    first with the folder (all unmodified files were recompiled)
    second without (all unmodified files were recompiled)
    then finally with the folder. And then only it worked.

    Does that makes sense to you ?

    Regard

    Like

  6. Pingback: Renaming Eclipse CDT Projects | MCU on Eclipse

  7. Thanks for this. I had exactly he same problem. In my case, I suspect it was because I had copy and pasted another project in workspace, and ended up with the wrong folder specified in the refresh policy settings.

    Like

    • Hi John,
      I have seen many silicon vendor provided example projects having this issue: obviously the FAE or the that group copied projects around and renamed them without knowing what they are doing :-(.

      Like

  8. Yup solved it for me as well. Thanks a bunch.

    As per usual Eclipse standard the bug referenced by john moule has been opened for years. I always wonder what really changes between Eclipse major versions (except for the name and start up image). That being said I still find it the best all-round IDE for C development.

    Like

      • Eclipse (Mars V4.5.2) fails to update “Refresh Policy” both when renaming projects and when copying and pasting them.

        These are not unusual operations. For example, I do them frequently whilst learning new microcontrollers, with many new projects starting out as renamed copies of existing ones.

        I’m currently learning STM32F7xx + SW4STM32 + STM32Cube + LwIP + RTOS, and even with an i7 and SSD was spending most of my time waiting for Eclipse. Your find has increased my rate of progress enormously.

        Thanks again!

        JB.

        Like

  9. Pingback: Tips for Making Copy of Eclipse CDT Projects Easier | MCU on Eclipse

What do you think?

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.