Everything does only find something

[php]

It evidently was not such a good idea of mine to hijack another thread with my problem, so I get a new one. First post over there, replicated here:

Re: Many files not found on local NTFS HDD

Postby php » Mon Nov 03, 2014 1:08 pm
Hello, I've got the same or a similar problem, so I checked all of the above and forced an index rebuild, and in debug mode, there was no error message re my hdd.

Both Win XP SP3 and Everything are fresh installs from Oct, 2014.

I've got my stuff on an external hdd, of which some directories are properly indexed by Everything, and others are not, and after rebuild of index, the SAME folders as before are missing from the search results lists (I sort the results by path, so don't miss the hits, but it's evident the respective folders are not there).

When I look up the folder properties (by Attribute Changer 7 which shows more than "Properties" in Windows), read-only, system, hidden, compress and archive are unchecked, and "Index" is checked"; this is identical to other folders which show in the list.

Thus, it seems Everything 1.3.4.686 (x86) does not work properly in the combination XP-and-external-drive?

Perhaps a hint: In spite of speaking of "directories" (plural) above, in fact, I am not sure if plural is correct, BUT Everything leaves out my most important folder there, which is called "1", and which by this is first on my list of folders "1", "2", etc. in that external drive, so perhaps it's a bug in the code (array, list, first element not being retrieved)?

[php]

Now, I did some further investigation. I went so far as to rename my main folder "x:\1\" to "x:\9\", then did a forced rebuild in Everything, then searched again for parts of file names in that folder: Nothing, not Everything.

I then searched for parts of file names in my folder "x:\2\" and its subfolders"; in this folder "2", Everything traditionally had found things, but now I'm positive it just shows SOME of the hits for some string in file names, not showing other hits with the same substring in the file name, and for other strings, it does not show anything, whilst those files being very well present in their respective subfolders.

As said, all this after a second forced rebuild of the Everything index, and after having checked and rechecked Everything exclude/include options.

It goes without saying that all these files NOT FOUND by Everything work fine, in their respective applications, and in any file manager.

Thus, it seems evident that at least in the combination XP with external hdd (ntfs of course, usb, always connected, 1000 giga (internal hdd is just 80 giga)), Everything should be considered as a tool that will find SOME of your files, if you're really lucky.

Can you do anything about this, or must this be considered as a permanent bug now?

(EDIT: Clarification: no European characters involved, all tries with just abc...xyz chars.)

[void]

Everything should find all your NTFS files.

Do you have any search options enabled in the Search menu?

Please make sure match case, match whole word, match path, match diacritics and regex are all disabled.
Please make sure the Everything filter is selected.

have you changed the Everything filter?
Please make sure the Everything filter has an empty search from Search -> Organize Filters -> Select Everything Filter -> Edit -> Search.

Please try disabling exclude hidden and system files from Tools -> Options -> Exclude.

Please check if Everything is finding orphaned files and folders by running Everything in debug mode and forcing a index rebuild from Tools -> Options -> Indexes -> Force Rebuild.
Look for the lines:

Code: Select all

removed 8 orphaned files in 0.025197 seconds
removed 5 orphaned folders in 0.004219 seconds

What is displayed in the debug console after the rebuild is complete?

[php]

Thank you for trying to help, but I have a very simple question: Why are the defaults totally different from what is intended to find "everything"? Would it not be highly preferable to set the defaults to the values you then ask for when "Everything" does NOT find "everything"? Would it not be sensible to do the defaults in a way that "everything" is found, and from THEN the user would restrain the search scope if he wanted to?

For example, the search options you mention cannot be found under "Options"-"General"-"Search" but will be found under Options-General-Home (!), and then, it's always "Use last value" instead of "Disabled", and the "Search" option there (i.e. in Home, not in Search, is "Custom" by default, the only other option would be "Use last value" (ironically).

Another detail: Your shortcut for debug mode is the least practical one for any non-English speaker, since most non-English keyboards do not have a "curve" key; mine does not even make it available by AltGr-something. So I had to change the ini file, but do NOT know if this ini file is correctly read, since I NEVER get to that special "debug mode window". (Yes, I had closed Everything, by File-Exit, then changed the ini file, then reopened Everything: NO "debug mode windos", NO textlines in green after forced rebuild.

Also, when I run Everything from system (by Start button, then "Run" or "Execute" or something, I don't have an English Windows), by

c:\programme\Everything\Everything.exe -debug (and Return)

I do NOT get any "console" or other "debug window" or such, I only get the normal Everything window, nothing else: I never see the Dos Window in your link.

And I NEVER see anything from my x:\1\, and in my x:\2\ folder, I see SOME hits for a string, and other hits are not displayed, the strings being just abc chars, within file names which also just comprise abc chars, no äöü, no ABC, just abc, as simple as that.

And as for "orphaned" files, whould could that mean after now 6 or 7 forced index rebuilds?

So at the moment, it's as said yesterday, and with Everything options that should not hide anything. And no debug mode (yesterday, I had thought I was in debug mode, without being there. So why even debug mode is not accessible?) As said, fresh Win XP 3, and fresh Everything, without resetting anything to anything "exclusive".

[therube]

> Why are the defaults totally different from what is intended to find "everything"?

(I've never been clear on what it is you're trying to do... anyhow...)

By default everything is set to find everything.
So if your search term is "abc"
It will find "abc" & "abc123" & "abc.123" & "123abc"... IOW, any file or directory containing the string "abc".

That is the default.

If you want to only find the whole word "abc", then there are methods to do that.


And a default is just that a default.
And while you may want the default as given, the next person may not.
With Everything, there are options to change behavior, so you can have it your way & next person can do what suits them.


Where a particular option may be found or how or if it is implement are all determined by the powers (power, actually) to be. Though that is not to say that others are cut out of the procedure, as plenty of others suggestions have been implemented.


> [keyboard] shortcut for debug mode...

Debug mode can also be started from (desktop) shortcut (or similar) with the command-line option -debug. The ~ key is (probably) interchangeable with the unshifted version of the same key, so you might check that. (As it is, on my XP system, I can't effect debug mode using the keyboard shortcut.) (Don't think there was any option to remap the shortcut to any other key, but not sure?)


> I had to change the ini

Any manual changes to the .ini need to be made with Everything (the program not necessarily the service) completely closed.


> Everything.exe -debug (and Return)
> I do NOT get any "console" or other "debug window" or such

Strange.

[php]

Debug mode

"Any manual changes to the .ini need to be made with Everything (the program not necessarily the service) completely closed."
As said by me, I closed ET by File-Exit, before hampering with the ini file.
And yes, run ET -debug will not show a particular window, SHOULD it show that, or only after there would be errors to be notified?
And as said, ET debug is not accessible by its unfortunate shortkey. In fact, any shortkey for that would only be accessible from WIHTIN ET, so there is no need to choose such an exotic one, right?

Defaults

I'm not complaining about your defaults. I just mentioned and tried to explain that from the other user and from me, in order to "find" any files, you ask that we reset numerous options to values which are NOT the defaults. Thus, my point was, the defaults - whatever you choose them to be - should be in a way that ET should find everything, anywhere.

abc, etc

We speak of different such strings, for different searches. BUT as I tried to explain, the SAME string "abc" is found in some file names (=name part of the file name, not suffix part, nothing extraordinary, just the plain, simple things!), and the SAME string "abc" is NOT found in some OTHER file names within even the same folder / subfolder. In fact, many files contain "net" within the file name (NOT as suffix), and SOME of them are listed by ET, and others are not, and as said, a whole directory, x:\1\ is not listed at all, i.e. NOTHING I search there will ever be listed (and when I rename the directory, nothing is listed either).

And as said, all these files which ET does NOT find, can be opened without any problem, and are listed in any file manager. Only SOME of them are "system" and even "hidden", since I run 1-click Autohotkey macros to "format" them this way, in order to get them differently colored in my file managers, BUT MOST ARE NOT, and ET will not find them though, and as said, in x:\1\, NOTHING even is found, 98 per cent of my files NOT being "system" or "hidden". And yes, I have got all options in ET as you said, so everything should be found.

I'm perfectly willing to try out other things / hints from yours, but if you cannot imagine a possible solution, this is to be considered a bug, with XP at least, i.e. file search by ET would then to be considered perfectly aleatoric, at least for non-c: drives, so people should know they cannot rely upon ET in those circumstances - and probably with more "modern" Win versions, too.

Often, it's not necessarily a "bug" in some program, but interaction between that tool and some Windows internals, but free or not, people should know if they can rely upon a tool, or if they can not.

[therube]

> ET -debug will not show a particular window, SHOULD it show that

Yes, it should show that, just like the picture in the link above.

Maybe run a CHKDSK on your drives?

http://www.voidtools.com/support/everyt ... _shooting/

[php]

From your link, I followed this:

Recreate the USN Journal

To force the NTFS driver to rebuild the USN Journal:

In "Everything", from the the Tools menu, click Options.
Click the NTFS tab.
Select the NTFS volume .
Uncheck Enable USN Journal Logging.
Click Apply.
Check Enable USN Journal Logging.
Click OK.

With the effect that now, I only get hits for "net" on "c".

Also, "Database location" was gone, i.e. the field was now empty.

So I entered a new database location, then did a new "Force Rebuild".

This only took some seconds, against many seconds for previous forced rebuilds of index, and now, with the database, it's exactly as without it:

hits on c: only. This means that setting of USN Journal Logging for x:, then checking it again had as affect that x: is of no more interest to ET. (Of course, I don't know what "USN Journal Logging" is about.)

So, from a totally aleatorix external hdd processing, we got to processing of just c: and nothing else anymore; my initial idea that ET could have a problem with external drives, seems to become confirmed.

You know, there is traditional, false "knowledge" in web fora that ET uses that ntfs file allocation table (or whatever it is called: MIT?) exclusively. Then, others intervene with saying, no, there is an index, too, but very fast since for file names only, not for content.

It seems that what you do in the background, for building these indexes and so on, is technically quite demanding and not totally robust as soon as it gets to external drives?

Perhaps from my USN Journal logging OFF-ON switch and that now, x: is not accessed by ET anymore, you can deduct what's going on (in part)?

Under normal circonstances, chkdsk would have been a good idea, but as said, all these files are accessible from anywhere else.

And if my MIT on the external hdd was damaged, that would show immediately, right? It's just that ET does obviously NOT access that MIT directly, for hit tables, but for building indexes, and builds the hit lists from its indexes. It seems that index building is not entirely robust, as soon as it gets to external hdds?

I also have been tempted to reinstall ET, but it does not crash or something, it runs smoothly, it just doesn't work correctly.

[therube]

> but as said, all these files are accessible from anywhere else

Though that doesn't mean it's in a consistent state.

To me I look at it like you have files that should be seen & aren't.
And -debug isn't working for you.

Bug in the program?
Certainly possible, but if it were happening to me & given that what you are seeing is not typical...

[void]

Please try running Everything in debug mode from a command prompt:

• Completely Exit Everything.
• From the Start menu, click Run...
• Type in cmd and press enter.
• Navigate to your Everything installation folder, eg:

  Code: Select all

  cd C:\Program Files\Everything

• Run Everything.exe with the -debug command line option:

  Code: Select all

  Everything -debug

What is displayed in the console after you have forced a rebuild from Tools -> Options -> Indexes -> Force rebuild?

Does Everything find all your files if you uncheck Exclude hidden files and folders and uncheck Exclude system files and folders from Tools -> Options -> Exclude?

[php]

Thank you for the instructions, will follow them after posting. GOOD NEWS from my side, anyway, but some quite interesting problems remain to be resolved yet!:

I ran chkdsk, which worked fine, except for the fact that I got (after hampering with USN Journal from ET as described above) a short message re the USN Journal - I've never got any such message in my entire life (but, on other occasions, lots of other, more "normal" chkdsk messages) - since those messages immediately disappear together with the "Dos" window, I can't say about the wording.

I then ran ET again: No hits except for c: (btw., just a very little fraction of the hits I should have got on c:, see below).

I then ran the traditional command line command: path\Everything.exe -debug AND ditto -debug -verbose - for both, I did NOT get any "dos"/command window.

I then DE-installed ET, from the dedicated deinstall.exe in its program folder.

I then downloaded the 1.3.4.686 install file again, first line of the installs on your home page, and installed it.

Here, while installing, I remarked
- Associate EFU files with ET (whatever that might be; so you ask for a decision here not-expert users (like myself) are unable to decide upon)
- Automatically index fixed NTFS volumes (I suppose "fixed" means "internal" = c: and other drives / partitions within the casing and running any moment, this would exclude usb drives even if they also run anytime) -
I left everything on defaults.

Then, I tried again for the above: path\ET -debug and -debug -verbose; I did NOT get any dos/command window from this!!!

So we've got the problem here that ET's debug mode is far less accessible than you might expect; as said, with my new ET installation (as before with the old one), I let everything on defaults.

Then, I searched again, and ET (as expected) said it was building up the index, this took a minute or so (USB 2.0 with almost 1 tera), and this was a similar time to the "forced rebuilds" before, with the previous installation.

Then, I got thousands of hits for "net", INCLUDING x:\1\, and hundreds or thousands for c:, where with the previous installation (no chkdsk for c: in the meantime!) I always had got SOME DOZEN hits for "net" on c: only. This means the previous ET installation only got SOME hits on c:, too, whilst now it gets PLENTY of hits for c:, without me doing anything re c: (I will run a chkdsk for c:, but never did in the month I got my new Win installation).

This means: Whilst for x:, we don't know after all, we are positive (I include you in this "we" since I don't lie, so please BE sure about this) about c: ET first just finds some files there, and now the new installation finds 10, 20 or 50 times more there, without c: itself being in this phenomenon for anything.

Now for the SORTING of the results: It's a total mess. I click above the path column in order for the hits to sort by path, and there is SOME sorting only, i.e. there are some 30 or so c: hits, then some dozen of x: hits, then c: hits again, and so on, i.e. it is absolutey impossible to get to the relevant hits, since the number of hits is about 3,600 (I know this number by "select all", then getting "n objects selected"), and I checked again:

- some c: hits
- some x: folder a hits
- some x: folder b hits
- and so on
- many c: hits
- many x: folder c hits
- many x: folder d hits
- many x: folder a hits
- and so on

This means there is clearly sorting within x: but SOME hits of SOME folders in hits x: are displayed as a block within the c: hits (and cutting this c: block in half by this), and then follow the other x: folders, INCLUDING those folders of which some hits had been displayed before (!). Btw, I also took the time to check if the "hits-on-top" were doubles/replicates of the ones below, i.e. if the same hits appeared again, but no, which means ET does not seem to replicate hits, but just does not sort them all correctly. In other words, some folders are replicated, but their content is not.

Of course, we are speaking of 3,600 hits here, so it seems ET will not sort by path correctly anymore when there are "too many" hits.

Now the underlying problem for the "just some c: hits, too" phenomenon: It's obvious an ET installation is "able" to seemingly work faultless, with in reality leaving out most hits, even for hits on c: - as said, I never hampered with c:, or with ET settings, and now the new installation, with the same default settings as before, seems to find them all (with no chkdsk or whatever in-between); also, all my "forced rebuilds" before had not done anything about this problem (I always got some dozen only of hits for "net" on c:), whilst "forced rebuild of index", for me, included anything (incl. index for c:), to the contrary of that "USN Journal" thing, which was specific to each drive.

So, there seem to be some problems in ET, but I very much hope my descriptions above will help to find them. Also, the biggest problem of them all is a double one:

- the SILENCE of ET when it doesn't grasp (for whatever reason that might have) all possible hits (cf. the c: problem, let alone the x: problem where other causes might have been involved): The user thinks that's all, when in fact ET even could have left out 9 tenth of the hits! and
- the SILENCE of ET when there obviously have been serious installation problems: as said, nothing crashes, no error message, nothing: the user thinks everything and "Everything" is ok! (Again, no connection with chkdsk / orphans and such, cf. the c: problem!)

Now, I'll try to get to the debug mode, by your new instructions, and then I'll do an add-on (edit) this post.

EDIT: I first tried again the other way above: As before, ET is opened (which proves I write the path correctly), but no command window is opened. I then tried your instruction to directly open ET in debug mode from within its program folder: Immediate success, i.e. ET AND the command window both open.

I see in this window regular messages "update filesystem C: (but nothing about x:)

Please let me know for any research I could make on my pc in order to investigate those things further!

[void]

Now for the SORTING of the results: It's a total mess. I click above the path column in order for the hits to sort by path, and there is SOME sorting only, i.e. there are some 30 or so c: hits, then some dozen of x: hits, then c: hits again, and so on, i.e. it is absolutey impossible to get to the relevant hits, since the number of hits is about 3,600 (I know this number by "select all", then getting "n objects selected"), and I checked again:

Folders are listed first and then files.
Are you seeing something different?

If you think you are still missing files on your X: drive, what is displayed in the console after you have forced a rebuild from Tools -> Options -> Indexes -> Force rebuild?