Asking for a precision, is the index refreshed instantly or at intervals?

[Oliverh]

Pardon me to bother with this question, I have doubts and I would like to know how much I can rely on Everything for minute activities.

When the FAQ says:

Does "Everything" monitor file system changes?
Yes, "Everything" does monitor file system changes.
Your search windows will reflect changes made to the file system.

Does that mean Everything is supposed to refresh itself instantly (in its search results) when we make changes to the monitored drives? (me: all local NTFS SATA)

I thought that was the case, but just a few minutes ago I found that a directory renaming I did wasn't taken into consideration so Everything was showing as result something that didn't exist anymore.
Forcing the reconstruction of the index in the options did the job, but that made me doubt. Maybe the index would have refreshed itself after a pair of minutes -- no idea. Maybe that was a bug (I'm with the latest stable, 1.4.1.877 (x64) windows - there was no error log file in the install dir.), all programs will experience bugs on the occasion on occasional random PCs.
It would be good to know, for desktop uses, if Everything is likely to miss changes made within the last minutes, or if it usually does keep track of it instantly

Thanks if someone can tell!

[NotNull]

There was another person who had issues with renaming files in Everything. (Search for "rename" on the forum will bring those (2? 3?) threads up, I guess. That was promptly solved by @void.

It was fixed *after* version 1.4.1.877.
Try the new stable version Everything 1.4.1.895 from the frontpage (www.voidtools.com)

(And let us know if this fixed your issue too, or that you experience some other issue.

[void]

Everything will detect changes instantly. After all changes are processed, Everything will wait for 1 second before checking for more changes.

Everything should be detecting all changes made to NTFS volumes. There could be an issue reading the USN Journal which might cause Everything to miss changes. Also there is a bug in Everything 1.4.1.877 when exiting Everything it would miss changes, it was fixed in 1.4.1.895.

If Everything is missing changes often, I would need to see your debug logs. USN Journal read errors are reported in your logs.

[Oliverh]

Thank you for your replies, Void and NotNull!

Then, apparently, I'm in front of a program bug.

I was first secretly relieved to find it was a program version problem, so I upgraded to the latest stable, 1.4.1.895 (the Help > Check update menu entry told me I had the latest minutes ago, go figure).
But even with 1.4.1.895 the problem is still present.

I wouldn't want to waste your time and interest, really, I'm fine with having a little problem, it's not the end of the world. Nobody's got 36 hour long days (I wish I had.)
But if you are interested in trying to find why it happens because you're interested in eliminating it, I don't mind going through testing.

What I confirmed myself is that
- the problem only happens for the files present on one precise partition. The drive that contains this partition has two other partitions, and them they don't have this problem.
- that partition contains a bit of everything, but in total (control-A: properties), there are over 332k files and 31k folders, for a total of 793 GB. NTFS.
- I checkdisked that partition (default options in windows10) and no errors were found
- In the program options, every time there was that button, I clicked to reset the options, just in case, no change.
- if I force the reconstruction of the index, I'm once again shown proper results... until I repeat the testing, haha.

What I haven't tested: are proper results shown eventually? Like, after 30 minutes, 2 hours, or something? It's something I'll be testing for the next days.

The details of the problem
- if I delete a folder/file already present in that particular partition, Everything will still show that partition as present
- If I search for a name that is not present "right now" in that partition, Everything shows nothing. Now, if I create a file/folder with that name in the windows Explorer, Everything won't still find that result
- however, Everything will mention something else once the directory/file is created, its name will be listed in appdata roaming microsoft windows recent in .lnk form
- another kind of test, I now cut-and-paste that new directory into another partition, and then, instantly, the proper result is found in addition to the .link result ( screenshot: https://imgur.com/a/TCHa0 )

Error logging: I tried to "create" something not cluttered,
- I activated the error log just now with /debug_log as a search
- I searched for something not existing anywhere yet, a randomly chosen string, "yetanothertestcomeon" -- no result found, no surprise
- I created a directory with that name on the partition that has the problem, in the Windows explorer
- there is now one result in the Everything windows, but it's yetanothertestcomeon.lnk in c:\users\ my name \Roaming\Microsoft\Windows\Recent
- now I cut and paste the "yetanothertestcomeon" directory from the problem partition, into another partition: Everything instantly shows the directory as being present in that partition, and *still* shows the .link from c:\users\ my name \Roaming\Microsoft\Windows\Recent
- last stop, opening the error log from %TEMP%\Everything Debug Log.txt and saving it to disk
I attached this error log in .zip format.

I don't see what else to do at this point.
Once again: I can live with this problem, it's no big hussle, it's not like it's a life and death issue.

[NotNull]

That's some serious research you have done!
There is semi-good news: there is a workaround possible, but that's when everything else fails. Keep that in mind, to keep up the good spirit


There is *still* some (possible) information missing (see end of message), but I'll try with what you provided (and that is a lot ..)

From user-settings to system-settings:

Do you have any exclusions that might get in the way?
Check Menu:Tools > Options > Indexes > Exclude.


What are the Everything settings of your H-drive (where you created the new folder)?
In Menu:Tools > Options > Indexes > NTFS , click on the H-drive i the Local NTFS volumes box.
Below this box you see the settings for this H:-drive.
All checkboxes should be enabled and "Include only" should be empty.
(Compare this to your other disks)


Are you running Everything as Administrator?
Under Menu:Tools > Options > General:
- Enable Everything Service
- Disable Run as administrator


Did Windows add the new folder to the USN journal?
Some background information: Windows keeps track of files in 2 (hidden) places: (1) MFT si the "global address book" of all your files and (2) the USN journal. Here Windows keeps track of all the changes to the drive. Backup programs use this to know what to back-up. And it is used to self-repair the"global address book".
Every NTFS drive has it's own MFT and USN journal. When you re-index a disk, the information comes from the MFT; if you add/delete/modify files, that info comes from the USN journal. Apparently here is the issue somewhere, so chck the USN journal first, to see if your folder is there (otherwise Everything can't find it, obviously)

- Start CMD as administrator and issue this command:

Code: Select all

fsutil.exe usn queryjournal "H:\path\to\Your Folder"

Either you get some mumbo-jumbo as result (File is in USN journal) or an error (file is not in USNjournal)


ONLY If file/folder is NOT in USN journal:
Repair USN journal
As the USN journal is also used to rebuild your "global address book", check first if your filesystem has issues (beyond what CHKDSK finds), by reading the SMART data of your disk. I don't expect problems, but beter be sure. You can read SMART data of your disk with utilities provided by your disk manufacturer or use an all-round tool like CrystalDiskInfo (that's what I use). If your health status isn't "Good", don't proceed.

You can delete the USN journal of your H: drive with this command:

Code: Select all

fsutil usn deletejournal /D H:

After that, reboot your machine. A new USN Journal will be created automatically.


Good luck!

P.S. Missing information:

- Normally the debug log starts with some information about Everything. In my case, it looks like this:

Code: Select all

Everything
Version 1.4.1.877 (x64)
Windows NT 6.2
Processors 2
IsAdmin 0
AppData 0
Service 1
cmdline "C:\Tools\Everything\Everything.exe"

That info is useful.

[Oliverh]

Allright

Do you have any exclusions that might get in the way?
Check Menu:Tools > Options > Indexes > Exclude.

The checkbox to activate the list of exclusions is ticked, but below there is nothing listed at all, it's an empty box.

I unticked the box just in case, tried with creating in Explorer and then searching in Everything for a new unique-name directory in the partition with the problem, and, ah, no, problem still present.

What are the Everything settings of your H-drive (where you created the new folder)?
In Menu:Tools > Options > Indexes > NTFS , click on the H-drive i the Local NTFS volumes box.
Below this box you see the settings for this H:-drive.
All checkboxes should be enabled and "Include only" should be empty.
(Compare this to your other disks)

AAAAAAAAAAAAAAH!!! Damnit!

In this one, in the list of the NTFS drives,
(I'm reading this in French, so I'm trying to guess what the English version is like, sorry about that)
- every partition that didn't have the problem had, (x) ticked: monitor changes, and ( ) unticked: include USN journal in recent modifications
- the partition that did have the problem had ( ) unticked: monitor changes, and (x) ticked: include USN journal in recent modifications

Aaaand case solved, phiew.

I have no idea why the settings were different specifically for that directory. I'm confident I did click to restore settings to defaults among my tests (and I witnessed those changes in the interface for instance, as my "show nothing unless there are results" default option was gone - I restored it, though ^^).
Maybe Windows had its say for some reason.
Or, more likely, I managed to screw it up like a pro without noticing, and no idea how and why O_o

Well, I guess this is it.

One again, thank you very much for your help and congratulations on your patience

[NotNull]

One again, thank you very much for your help and congratulations on your patience

No problem, at all. And ... you deserved it!
Right after you posted your first question on this forum, you solved another user's problem. I absolutely can't remember what it was all about, but that stood out. In a positive way.

[void]

Thanks for the debug logs, once I looked through the logs I thought, is monitoring even enabled? Glad it is working now.

[NotNull]

@void:
Would an excluded file be shown in the debug log? I didn't see any reference in the log, but wasn't sure)

[void]

Yes, you should see one of the following messages (in red) if a file is excluded:
unable to add ntfs file: "filename.txt": parent 0000000000000000 not found (only when the file's folder is excluded)
unable to remove ntfs file: "filename.txt": file not found
unable to update file: "filename.txt": file not found
unable to update file: "filename.txt": parent 0000000000000000 not found

No message is shown when a specific file is excluded by filename.

[NotNull]

OK, got it. Thanks!!