Recommendation on NAS share with many files for monitoring

Discussion related to "Everything" 1.5 Alpha.
Post Reply
sbrblz
Posts: 8
Joined: Mon Sep 26, 2022 6:12 am

Recommendation on NAS share with many files for monitoring

Post by sbrblz »

I am using 1319 portable version.

Using a shared NAS, there's about 930,000 files on it. The NAS is not mapped to a drive letter, it has many shared folders under its name. I tested adding the main NAS share to the index, it takes about 3 minutes to go through all the files I believe. Monitoring for files seemed to work at first, especially when I tested adding files to a share, but later in the day when I batched renamed some files in Everything, the change was not picked up by the program itself, I had to run a manual rescan. After the manual rescan, I saw in the index logs tons of updates that the monitoring did not pickup, basically it's as if the monitoring stopped detecting any changes for most of the day.

Would adding all the individual shared folders in the NAS allow Everything to better monitor for changes as opposed to just adding the NAS name? If so, can we have the option to change the settings for all folders added under indexes options at once instead of one by one, such as rescan options etc.
I did try rescan on full buffer option as well.

I test on my own NAS as well and while monitoring pickups changes for files I add on the same computer, it's hit or miss if it will pickup changes from other devices. If monitoring is not meant to be 100% reliable, would adding full rescans periodically like every 30 minutes etc be the recommended option? Using the fast rescan option, it seems to take about 2 minutes. It wold probably be faster adding all folders separately and changing the rescan settings for the most frequently used shares only.

Edit: After doing some testing, it seems changes made to subfolders of a shared folder are not detected, only changes made to the root of a share. This seems to match with this post. I suppose it's a limitation of smb server/client dynamics. Maybe adding all shares with more frequent rescan settings for the more frequently modified folders would seem like the way to go.

To recap:
Everything will (generally) detect all changes to shared folders and subfolders made by the same computer it is running on.
Everything will detect changes to the root of a shared folder by another computer/device.
Everything will detect changes to subfolders of a shared folder made by another computer/device if that folder is also open on the same computer Everything is running on.
Everything will not detect changes made to subfolders by another computer/device if that folder is not already open on the same computer Everything is running on.
Last edited by sbrblz on Tue Apr 04, 2023 12:37 am, edited 1 time in total.
therube
Posts: 4977
Joined: Thu Sep 03, 2009 6:48 pm

Re: Recommendation on NAS share with many files for monitoring

Post by therube »

I believe it's going to depend on OS/file system (i.e., NTFS) used on the NAS (or lack thereof)?

(The rest, I'll defer to others...)
void
Developer
Posts: 16735
Joined: Fri Oct 16, 2009 11:31 pm

Re: Recommendation on NAS share with many files for monitoring

Post by void »

Everything uses ReadDirectoryChanges to monitor changes to NAS shares.

To view the changes Everything receives from ReadDirectoryChanges, please run Everything in debug mode:
  • In Everything, type in the following search and press ENTER:
    /debug
  • Detected changes are shown in Cyan color.
It will be up to the NASes implementation of ReadDirectoryChanges on what files/folders changes are detected.
It sounds like your NAS does not support the bWatchSubtree parameter for ReadDirectoryChanges.



There is an alternative to ReadDirectoryChanges in Everything 1.5 called SHChangeNotify.
This should detect all local changes made by the shell.
However, you will most likely run into the same issue of missing changes to unwatched subfolders on remote volumes.

To disable ReadDirectoryChanges and use SHChangeNotify:
  • Copy and paste the following into your Everything search box:
    /read_directory_changes=0
  • Press ENTER in your Everything search box.
  • If successful, read_directory_changes=0 is shown in the status bar for a few seconds.
  • Restart Everything.
read_directory_changes



Consider rescanning your NAS share every hour:
  • In Everything, from the Tools menu, click Options.
  • Click the Folders tab.
  • Select your NAS share.
  • Schedule how often a rescan occurs, eg: every 1 hour.
  • Click OK.


Or schedule a background rescan immediately:
  • In Everything, type in the following search and press ENTER
    /rescan \\nas\share
    (where \\nas\share is your nas share or network drive, eg J: - It should match your folder index under Tools -> Options -> Folders)
Post Reply