Eclipse Project ‘Refresh Policy’: Broken Incremental Build with External Make?

Posted on by

So here is a problem I have observed for a while, and several users have reported that to me: the problem is that for some reasons and only for some projects, when I do a ‘build’ in Eclipse CDT, it does a ‘full’ build. It does not only build the changed files, it builds all or many files in the project.

Building

Building

I expect that only the changed files will be rebuild. Instead, it compiles many files it should not compile/build, with the result that a build takes a very long time. That’s clearly wrong, and not what I expect. So what is causing this problem?

The issue is present in multiple Eclipse version (Kepler, Luna), with different Eclipse distributions (Kinetis Design Studio v2.0.0 and v3.0.0) with different make utilities. In all my projects I’m using the GNU ARM Eclipse plugins, but I doubt that this is the problem. I believe it might be more of a CDT problem instead.

The first observation was: The issue seems not to happen for ‘fresh’ projects, but happens for ‘older’ projects or some projects I have received from other users. It is even more strange, that the number of files unnecessary recompiled seems to be kind of random: sometimes it just triggers a new linking phase, sometimes it seems to affect only one or a few files, sometimes it seems to rebuild pretty much all of them. And things might even change depending on which files I change.

File Dates?

First I thought there might be something wrong with the file dates, as make needs to rebuild something when a source file is newer than the object file, or if the header file is newer than the source file using it. I had once a machine where the host clock was going wild, resulting in strange file time stamps. But on my current machine everything was OK.

Debugging Make Files?

If the files are not newer, maybe some do not exist? Checking them on the file system, object files all where there, all with newer file dates than the source files. So I started to see what kind of debug information I can get from make. Make has a –debug option (see https://www.gnu.org/software/make/manual/html_node/Options-Summary.html) which is very useful for debugging make files.

Running Make with Debug Logs

Running Make with Debug Logs

The only thing I found was that somehow some object files do not exist:

Reading makefiles...
Updating goal targets....
 File 'all' does not exist.
  File 'FRDM-K64F_Demo.elf' does not exist.
   File 'Sources/Joystick.o' does not exist.
  Must remake target 'Sources/Joystick.o'.
Building file: ../Sources/Joystick.c

But they *do* exist before running the build! Strange. So something is deleting them, and somehow it must be the build process? But I have not seen any ‘rm’ or ‘del’ in the make logs.

Build Automatically?

Then, I thought it might be related to ‘Build Automatically’ setting:

build automatically

build automatically

‘Build automatically’ actually does not make any difference, as this functionality seems not be implemented in CDT: that would allow that the system would automatically build my application after I have changed and saved my file. So this was a Red Herring :-(.

External or Internal Builder?

What I noticed was that using the Eclipse Internal Builder does not have this problem. The problem only happened with using the external builder (external make):

External and Internal Builder

External and Internal Builder

With the internal builder, Eclipse uses its own built-in make utility, while with the external builder the build process uses an external make. So I thought that I have found a problem in make: Using different make executables did not make any difference :-(.

Refresh Policy!

Actually, that ‘external builder’ was triggering something. After poking around this issue for a few days, finally I received a tip from Rock Li (thanks!): The issue seems to be related to the Refresh Policy setting in the project, which only affects external builders: If the refresh is set for an non-existing (or wrong directory), other than the project root directory itself, these strange rebuilds of files are happening:

Mismatch in Refresh Policy

Mismatch in Refresh Policy

And to my surprise: Deleting that wrong directory and using the project folder instead fixed the issue!

Matching Refresh Policy Folder

Matching Refresh Policy Folder

So the question is: what is this ‘Refresh Policy’ used for? As indicated in the dialog itself, it is only used if using an external builder (which is the case for my project). That refresh is executed after the build process, to make sure the Eclipse project knows about all the new files (object files, make files, etc) generated.  But if that setting is wrong, wrong things might happen :-(.

And now I realize why I had some strange things with some projects, and not with others. That above project I had probably created as FRDM_K64F_Test, and then later renamed to FRDM_K64F_Demo, but that Refresh Policy setting did not get updated.

Summary

If your external make build in Eclipse CDT is taking too long and building too many files, then check what is specified in the Project settings, under Refresh Policy. If you have a wrong folder listed there, then this is very likely causing that problem.

I’m not sure if this is a problem of CDT, make or anything else. I just notice if I set the folder to the correct one, the issue goes away :-). If anyone has an explanation of what is happening, please comment :-).

Happy Building 🙂

40 thoughts on “Eclipse Project ‘Refresh Policy’: Broken Incremental Build with External Make?”

  1. I’ve noticed on Eclipse 3.8 that sometimes cross project references are not refreshing properly. This is a bug. Just manually Refreshing the project and also the associated projects seems to fix this issue.

    Probably your issue is related to this problem.

    I’ve also found that library projrcts don’t automatically have their directory links updated when the project name is changed. Sometimes I have to dive into the XML configs to delete dead references. Another bug.

    But still prefer to use Eclipse 100x more than any proprietary Windows based platform.

    Like

    Reply ↓

    • Yes, it could be related to the same issue. But I’m still not clear why with the wrong refresh the make utility somehow has the object files deleted in my case. Really strange.

      Like

      Reply ↓

  2. Thanks Erich,

    I got a problem on some imported project that looks like your subject, but it was here Eclipse that didn’t see modified files. I solved the problem by delete debug directory…
    Your paper solve my problem.

    Like

    Reply ↓

  4. Hi Erich, great find! I expect the most common cause of the Resource name being incorrect is because the project has been renamed and the resource name has not been updated. Have you found an upstream (maybe Eclipse Platform or CDT) bug or filed one?
    cheers john

    Like

    Reply ↓

  7. Pingback: Kinetis Drone: Sensor Fusion Toolbox | MCU on Eclipse

  8. Hi Erich,

    I’ve had this issue on a project where there was no folder in refresh policy. Adding the folder was not sufficient. I have solved it by building 3 times:
    first with the folder (all unmodified files were recompiled)
    second without (all unmodified files were recompiled)
    then finally with the folder. And then only it worked.

    Does that makes sense to you ?

    Regard

    Like

    Reply ↓

  9. Pingback: Renaming Eclipse CDT Projects | MCU on Eclipse

  11. Thanks for this. I had exactly he same problem. In my case, I suspect it was because I had copy and pasted another project in workspace, and ended up with the wrong folder specified in the refresh policy settings.

    Like

    Reply ↓

    • Hi John,
      I have seen many silicon vendor provided example projects having this issue: obviously the FAE or the that group copied projects around and renamed them without knowing what they are doing :-(.

      Like

      Reply ↓

  12. Yup solved it for me as well. Thanks a bunch.

    As per usual Eclipse standard the bug referenced by john moule has been opened for years. I always wonder what really changes between Eclipse major versions (except for the name and start up image). That being said I still find it the best all-round IDE for C development.

    Like

    Reply ↓

      • Eclipse (Mars V4.5.2) fails to update “Refresh Policy” both when renaming projects and when copying and pasting them.

        These are not unusual operations. For example, I do them frequently whilst learning new microcontrollers, with many new projects starting out as renamed copies of existing ones.

        I’m currently learning STM32F7xx + SW4STM32 + STM32Cube + LwIP + RTOS, and even with an i7 and SSD was spending most of my time waiting for Eclipse. Your find has increased my rate of progress enormously.

        Thanks again!

        JB.

        Like

        Reply ↓

  15. Pingback: Tips for Making Copy of Eclipse CDT Projects Easier | MCU on Eclipse

  18. Dragging up an old post! I’ve been trying the Mac version of MCUXpresso recently and had this “rebuild all every time” issue. I’d imported a project that worked fine in Windows but was doing full rebuild in Mac.
    Finding this article, and checking “refresh policy” it’s blank (nothing listed) in both Windows and Mac, so apparently Windows treats that as “finished” while Mac treats it as “not finished” in some weird way!

    All sorts of other Mac / Windows issues too, but they don’t belong here.

    Like

    Reply ↓

    • strange. But as as don’t own a Mac, I cannot check. Maybe there is something wrong with the file dates (maybe just a lucky punch)? Other than that: does it happen just for that project or for others (what would be the difference)? Maybe it is one of these forward/backward slashes problem?

      Like

      Reply ↓

      • All of my MCUXpresso Windows projects have the refresh policy empty with nothing listed. On the Mac side I have a couple brought in sample projects which came in with the refresh policy set like you describe here – and I was forced to do the same in the test project I copied over from Windows.

        Like

        Reply ↓

        • So it seems the curiosity is why the Windows version is perfectly happy with nothing listed in the refresh policy (good news for me!)

          Like

        • I would say that having nothing in there is not ok. It should list its own project. I only can speculate why this would result in different behaviours on Win and Linux. But my guess is that all this ‘check for time stamp’ is heavily OS related, and it seems that Windows deals with this for sure differently, and somehow it works. So: does it work if you have the project listed on Mac in the dependency?

          Like

  20. Pingback: Tutorial: Rename, Copy or Clone Eclipse Projects with MCUXpresso | MCU on Eclipse

What do you think?

This site uses Akismet to reduce spam. Learn how your comment data is processed.