PSoC Creator build with workspace in network folder

Tip / Sign in to post questions, reply, level up, and achieve exciting badges. Know more

cross mob
GiFR_1080991
Level 2
Level 2
10 sign-ins 5 replies posted 5 sign-ins

Hi All,

I'm using PC 4.4 under Windows 10 with a workspace placed in a share folder (filesystem on local network).
In PSOC4 projects I'm experimenting that after a successful "Clean&Build" , if I edit a source file correctly listed in source tree (i.e. main.c) then a following "Build" doesn't regenerate a new linked HEX file.
The same if I edit a source file in "Generated_Source".
The modified source is recompiled, but there is no relinking.
An output message says: "The link step is up to date, no work needs to be done".
I checked that the timestamps of main.c, main.o, main.lst are newer than application .A, .ELF, .MAP, .HEX.
I also tried to force timestamps from cmd shell in several ways, but same result.
If the same workspace is copied into a local folder (C:\...)  it works as expected, with "Build" that regenerates HEX after a source change.

Any idea?

Giuliano

 

0 Likes
1 Solution
Rakshith
Moderator
Moderator
Moderator
250 likes received 1000 replies posted 750 replies posted

Hi Guiliano, 

Thank you for the update. I did get a response regarding this from the internal team.
When using more than one machine, the times need to be synced to the millisecond. This is true for build tools, including Make, CMake, and even Visual Studio. This means using NTP on all systems involved, using the same time server for time resolution. Furthermore, depending on how far off the time is at the first sync, it may take a while (up to a week) until all systems are perfectly in lock step.

In some cases, times may be cached in the tool based on the local system time but this should not be an issue if the systems are in sync. This could be the reason why you are facing the issue. 

It is however recommended to edit the project locally, build and then upload it to a shared repository.

Thanks and Regards,
Rakshith M B

View solution in original post

15 Replies
Rakshith
Moderator
Moderator
Moderator
250 likes received 1000 replies posted 750 replies posted

Hi Giuliano, 

You are observing this issue in all projects in the shared folder, is that correct? You can try creating an empty project, modify the main.c, and build the project to test this out.

We have seen some issues with PSoC Creator projects on a shared folder which is why we recommend keeping the PSoC Creator in local directories.

Can you please try deleting the build artifacts (.A, .ELF, .MAP, .HEX) and rebuild the projects? Does this work as expected? If so, a workaround could be to create a prebuild step to delete these files before the start of the build process.

Thanks and Regards,
Rakshith M B
0 Likes
GiFR_1080991
Level 2
Level 2
10 sign-ins 5 replies posted 5 sign-ins

Hi Rakshith,

I confirm that it is a general problem that affects all projects in shared folders.
I tried also creating an empty project  and I can confirm the issue.

For your information in my case the shared folder is the personal user area and it is mounted from a "Windows Server 2016" using DFS (Distributed File System).
I have been working with PSoC Designer/Creator on shared folders since 2008 without other mayor issues in the recent past.

I also confirm that just deleting the HEX file triggers the rebuild.
I created a workaround as you suggested with a prebuild step and it fits the job.

Anyway I would propose to file a ticket on this subject, because the misbehavior is critical and could pass undetected. In my case I spent a couple of days debugging a not working booloader/bootloadable project, before realizing the root cause.
i.e. when changes are to be applied to code into "Generated_Source" there is need of an incremental rebuild without a clean step.

Thanks and Regards,
Giuliano

0 Likes
GiFR_1080991
Level 2
Level 2
10 sign-ins 5 replies posted 5 sign-ins

Hi again Rakshith,

I've got news...
I understood and tested that a difference between local storage and network storage is that by default the file attribute "LastAccessTime"  is always updated on local storage, but it is NOT updated on network storage.

This can be shown using PowerShell command:
    > Get-ItemProperty .\main.c | Format-list -Property * -Force

After manually forcing  the update of timestamp I was able to trigger the re-linking.
This can be done using PowerShell command:
    > Set-ItemProperty -Path .\main.c -Name LastAccessTime -Value (get-date)

This means that PSoC Creator is checking dependencies looking at file attribute "LastAccessTime" instead of "LastWriteTime"... this is fundamentally wrong, therefore in my opinion ... it's a BUG!

"LastAccessTime"  is updated together with"LastWriteTime" on local storage, for this reason it works.
Instead "LastAccessTime"  is never updated after file creation on network storage, for this reason linking is skipped.

Let me know what you think and if this finding could be forwarded to the Developer Team so that there is chance it is considered for a fix.

Thanks and Regards,
Giuliano

 

0 Likes
Rakshith
Moderator
Moderator
Moderator
250 likes received 1000 replies posted 750 replies posted

Hi Guiliano, 

Thank you very much for sharing the results of your tests. This will definitely help.

I definitely agree that this is a difficult error when you are trying to debug a project. I have created an internal ticket forwarding all the information shared to the internal team.

Thanks and Regards,
Rakshith M B
0 Likes
Rakshith
Moderator
Moderator
Moderator
250 likes received 1000 replies posted 750 replies posted

Hi Guiliano, 

I received an update from the internal team regarding this issue. They looked into the issue and PSoC Creator is querying the last write time of the file. However, updating the last write time depends on the OS and might be inaccurate to improve performance. It might be caching till the last access time is updated - https://docs.microsoft.com/en-us/dotnet/api/system.io.file.getlastwritetimeutc?view=netcore-2.0#rema... 

By default, the last access time updates should be enabled and we suspect that it is disabled in your case. The steps to enable the last access time updates here might help - https://www.tenforums.com/tutorials/139015-enable-disable-ntfs-last-access-time-stamp-updates-window...

Thanks and Regards,
Rakshith M B
0 Likes
GiFR_1080991
Level 2
Level 2
10 sign-ins 5 replies posted 5 sign-ins

Hi Rakshith,

I understand the answer, but sorry if I don't agree.
There should be another place where the wrong check is performed or other reasons.

In my case DisableLastAccess = 2 (managed by system).
As already reported above, LastAccess is updated on local storage (NTFS), but not on network storage (DFS).
This is the default behaviour on DFS filesystem, likely to reduce data replication traffic on network shares.
Other filesystems (NTFS, NetApp, etc.) could behave or be configured differently.

The explanation about a later update of LastWriteTime because of caching doesn't hold.
It's true that there could be a small delay in updating of LastWriteTime, but it is not causing the issue.
I can prove it as shown in the following.

For the very basic empty example, see below all the time stamps for source main.c and Design01.elf

GiFR_1080991_7-1621520603257.png

GiFR_1080991_6-1621520561951.png

You can see that the source main.c  has been edited at 16:22 (LastWriteTime).
LastAccessTime is frozen to the CreationTime at 15:52 (as default on my network storage).

Instead the ELF file has been written by a previous build at 16:20 (the same for files .HEX, .MAP, .A, .DEPS) .

Now, if a Build is launched (at 16:27) the following output is produced:

GiFR_1080991_8-1621520865552.png

You can see that the main.c is recompiled, but the link step has been skipped.
In other words, files main.o and main.lst have been updated, instead files .HEX, .ELF, .MAP, .A, .DEPS not updated.
This can be repeated again and again and the linking never happen.
No network caching or delays are involved, it's a step by step process and you clearly see the timestamps above.

Now, let's force update of LastAccessTime on main.c using shell command:
> Set-ItemProperty -Path .\main.c -Name LastAccessTime -Value (get-date)

GiFR_1080991_9-1621521482710.png

Now LastAccessTime is 16:37, while LastWriteTime is still 16:22 as before.

Now, if a Build is launched (at 16:40) the following output is produced:

GiFR_1080991_10-1621521622382.png

You can see that the main.c is recompiled again and the linking is now correctly performed.

Overall the above tests clearly show that there is a step in the build process that decides to skip the linking by looking at the LastAccessTime of source files.
I believe that LastWriteTime is checked on main.c to trigger its recompilation.
But there should be another case for which the check is done another way and the last build step is skipped.

Thanks and Regards,
Giuliano

0 Likes
Rakshith
Moderator
Moderator
Moderator
250 likes received 1000 replies posted 750 replies posted

Hi Giuliano, 

Thank you for sharing the logs. I forwarded the logs to our internal team.

Powershell might not use the same API to fetch the access and write times of the file. Can you please try using Process Monitor? It should allow you to see what the system returned at the time of the call.

The team did a search through the source files to see if there are any calls for the last access time and they were unable to find one. All calls were for the last write time. PSoC Creator should not be able to fetch the last access time. So the incorrect value is most likely returned by the .NET API used or the OS in the case of your network folder.

Thanks and Regards,
Rakshith M B
0 Likes
lock attach
Attachments are accessible only for community members.
GiFR_1080991
Level 2
Level 2
10 sign-ins 5 replies posted 5 sign-ins

Hi Rakshith,

I attached log file from "Process Monitor", where I filtered only events for "psoc_creator.exe" and path "main.c".
It covers the operations performed by "Build" after a re-saving of main.c (with the link step skipped).

Hope this could be useful

Thanks and Regards,

Giuliano

0 Likes
Rakshith
Moderator
Moderator
Moderator
250 likes received 1000 replies posted 750 replies posted

Hi Giuliano, 

Can you please also share the logs for main.o file? In fact, it would be better if you share the logs filtered just for psoc_creator events. We can then add the filters for the specific files if needed.

Thanks and Regards,
Rakshith M B
0 Likes
lock attach
Attachments are accessible only for community members.
GiFR_1080991
Level 2
Level 2
10 sign-ins 5 replies posted 5 sign-ins

Hi Rakshith,

I attached full log file from a new run.
Thanks and Regards,
Giuliano

 

0 Likes
GiFR_1080991
Level 2
Level 2
10 sign-ins 5 replies posted 5 sign-ins

Hi Rakshith,

I update you with my last findings because I would not want you all to spend further time on the wrong path.

This week I asked to my admins to also check on the issue, from their point of view.
This night they made some change to the storage servers, still I don't know the details.
After that change the behaviour has fundamentally changed:
- manually updating LastAccessTime of source is no more a valid workaround.
- build just after editing still skips the linking.
- build repeated after more than 30 seconds finally perform the linking.

I found that local machine time is behind 30 seconds with respect to server storage.
It means that, from the point of view of local machine, the files are created "in the future".
After manual alignment of machine time with storage, the building behaviour is finally correct!
I'm sure this was not sufficient before the last changes to storage setup.
Therefore is seems that the issue was related to how the server storage was setup and also quite obscure to debug.
Once I know more detail on what changes have been applied, I will post them.

Still there is some residual non-robustness... I mean, if time alignment between local machine and storage is not perfect (i.e. within 1 second), then the linking could be skipped again.
If everything that is time-evaluated for linking is located strictly within ".\CortexM0p\ARM_GCC_541\Debug\" it should not happen, in my opinion.

I mean, if time misalignment between local and share storage is so critical, likely there is somewhere a time comparison between an file on share and a local file (or machine time).

Could it be that some intermediate result for linking is stored in a temporary directory on local storage?

I thank you all for the time you spent on this issue and the suggestions.

Best Regards,
Giuliano

0 Likes
GiFR_1080991
Level 2
Level 2
10 sign-ins 5 replies posted 5 sign-ins

Hi Rakshith,

I have an incremental understanding on the matter...
With the last changes made by our IT, now I have the network storage that is time-aligned with my machine.
Still there is a skew of less of 1 second, but on timestamps it can result as a 1 second of difference.
What I see now from PSoC Creator is that the build always skips linking on first try after source editing, then it succeeds on second try.
The criticality is so subtle now that if I try to get log with Process Monitor, then the issue doesn't show anymore because of the added delay of tracing.

This would confirm my hypothesis on previous post, that likely Creator is comparing timestamps between network and local files, or between network files and machine time. It would be a criticality in case of network share, because as I see,  there could be a not negligible skew in timestamps from share storage, also in best case.

Let me know if you could made some check in this sense, possibly using the logs that I shared previously.

Thanks and Regards,
Giuliano

0 Likes
Rakshith
Moderator
Moderator
Moderator
250 likes received 1000 replies posted 750 replies posted

Hi Guiliano, 

Thank you for the update. I did get a response regarding this from the internal team.
When using more than one machine, the times need to be synced to the millisecond. This is true for build tools, including Make, CMake, and even Visual Studio. This means using NTP on all systems involved, using the same time server for time resolution. Furthermore, depending on how far off the time is at the first sync, it may take a while (up to a week) until all systems are perfectly in lock step.

In some cases, times may be cached in the tool based on the local system time but this should not be an issue if the systems are in sync. This could be the reason why you are facing the issue. 

It is however recommended to edit the project locally, build and then upload it to a shared repository.

Thanks and Regards,
Rakshith M B
GiFR_1080991
Level 2
Level 2
10 sign-ins 5 replies posted 5 sign-ins

Hi Rakshith,

thank you for your further checks and answers.
I generally agree with the scenario that you depicted.

We use NTP, but I however think that something is not optimal here, infact  I'm in the loop with my IT to possibly have a better sync to storage.

About the requirement for a Time Accuracy of 1ms, here I'm less confident. In theory it should be possible with a perfect setup on systems with Windows Server 2016. With previous Server 2012 and 2008 the best accuracy was in the tens of milliseconds range. Maybe there is also impact from the widespread virtualization of services.
Let's see, these are issues that I have to clarify internally.

About your words:


In some cases, times may be cached in the tool based on the local system time but this should not be an issue if the systems are in sync. This could be the reason why you are facing the issue.

Here I agree also, but I think there is a different view of the same facts.
With a dependency chain between initial/intermediate/final files (source >>> object >>> image), if all the files are located within the same filesystem, the eventual time skew with the machine time should be irrelevant.
If someway the tool is caching the local time for making time comparisons between the Build directory files , I think it's a criticality that then requires to workaround with a huge precision in the network timing accuracy.
I'm using other environments that builds with "make" and also Visual Studio and I don't see a similar behaviour.

I mean, one thing is to compromise with a criticality/workarounds, another is an ideal robust solution.
My intention here was to possibly give a hint to possibly remove the weakness at the root.

On the other hand I understand that maybe my issue is not so general.

Thanks for  your attention and support.
Best Regards,
Giuliano

 

Rakshith
Moderator
Moderator
Moderator
250 likes received 1000 replies posted 750 replies posted

Hi Giuliano, 

Thank you for the response and your cooperation throughout. 
I will forward your suggestions to the internal team so that they can evaluate your request. 

Thanks and Regards,
Rakshith M B
0 Likes