Many files not found on local NTFS HDD

If you are experiencing problems with "Everything", post here for assistance.
Post Reply
toriatama
Posts: 25
Joined: Sun Oct 12, 2014 4:49 am

Many files not found on local NTFS HDD

Post by toriatama »

Hello,

At first, thank you so much for making such a wonderful tool which save so much time! Everything is really a great program, I cannot use windows search anymore!

Here is my problem:

Everything seems not able to search properly many files and folders on my local D drive. However I did not set any exclusion for D or for any kind of files. Moreover, D drive has a lot of files, but Everything cannot find them. I had this problem long time ago, it then disappeared with an previous update, and now the problem is back. I noticed that D drive is not totally ignored since Everything can find some files, but only a very small amount.

Tech info:
Windows 7 Ultimate 64bit (updated), French version, Processeur Intel(R) Core(TM) i7-2600K CPU, 8 Go RAM.
PC has 3 HDD, all formatted with NTFS, plus one external HD. Local Drive D size is 1To.
Everything version: Version 1.3.4.686 (x64)
Everything is set to run as administrator, service is not enabled.

What I have already tried:
- Force Rebuild database
- in NTFS option enabling/disabling USN Journal and "include in database option".
- I have increased USN Journal Maximal size for D to 232704Ko (originally 32704Ko), no effect.

Remark: my local D drive has a LOT of files, around 700Go, including all kind of documents and media, even files with unicode names (chinese, japanese, etc. but usually they were not a problem for Everything, I can search in many languages without trouble). Some files and directories paths are also very longs, for example: 129 characters. Windows 7 handles everything pretty well, so I guess that Everything should work properly.

Thank you for any advice!
void
Developer
Posts: 16841
Joined: Fri Oct 16, 2009 11:31 pm

Re: Many files not found on local NTFS HDD

Post by void »

Please check if Everything found a problem with the drive:
  • In Everything, Press Ctrl + ~ to open the debug console.
  • From the Tools menu, click Options.
  • Click the Indexes tab.
  • Click Force rebuild.
  • In the debug console, look for following lines (in yellow):

    Code: Select all

    removed 5 orphaned files in 0.001166 seconds
    removed 3 orphaned folders in 0.000304 seconds
  • How many orphaned files and folders did Everything find?
  • It is normal for a few files and folders to be orphaned (internal NTFS files), but if there are a lot (> 500) it may indicate an issue with your drive.
  • If there are many orphaned files, please make a backup and check the volume for errors.

Everything supports indexing unlimited length file names and paths and supports unicode.
Everything does not support hard links. Only the first hard link instance will show up in Everything.
toriatama
Posts: 25
Joined: Sun Oct 12, 2014 4:49 am

Re: Many files not found on local NTFS HDD

Post by toriatama »

Thank you very much for your help.

I've followed your indication but in the In the debug console and found this lines:
removed 245510 orphaned files in 0.007614 seconds
removed 12829 orphaned folders in 0.001653 seconds
Then I've checked the disk using Windows maintenance tools and another disk utility, they both found no error on the disk (I did not make a complete physic/sector check since the volume is working properly and no error was detected).

Then I've rebuild the cache again. Same issue, most files on D are not detected, many orphaned are found:
removed 245465 orphaned files in 0.007308 seconds
removed 12814 orphaned folders in 0.001494 seconds
Anything else I can try?
void
Developer
Posts: 16841
Joined: Fri Oct 16, 2009 11:31 pm

Re: Many files not found on local NTFS HDD

Post by void »

Thanks for the reply.
However I did not set any exclusion for D or for any kind of files.
Can you confirm in Tools -> Options -> Exclude that:
  • Exclude hidden files and folders is unchecked.
  • Exclude system files and folders is unchecked.
  • Exclude folders is empty.
  • Include only files is empty.
  • Exclude files is empty.
Is Include only set for any volumes in Tools -> Options -> NTFS?
toriatama
Posts: 25
Joined: Sun Oct 12, 2014 4:49 am

Re: Many files not found on local NTFS HDD

Post by toriatama »

Exclude hidden files and folders was checked.
Exclude system files and folders was checked.

Exclude folders is empty.
Include only files is empty.
Exclude files is empty.


I had checked the first options (hidden files&folders and system files&folders) because I thought it was safe and convenient to hide system and hidden files.

Now I've unchecked them both and Everything can find all my files! Great! The culprit was "Exclude system files and folders" (checking or unchecking hidden files/folders does not change anything for my issue). It's a bit odd since my files and documents on D are not located in any system folder, many are in custom folders not connected with the system. For example:
D:\MyFiles
D:\WorkInProgress
etc.

Is it a small bug? Or did I misunderstand the "Exclude system files and folders" option? Anyway many thanks for your help!
void
Developer
Posts: 16841
Joined: Fri Oct 16, 2009 11:31 pm

Re: Many files not found on local NTFS HDD

Post by void »

These folders may have the system attribute bit set.

Check the folders attributes with attrib:
From a command prompt, run:

Code: Select all

attrib D:\MyFiles
This will list all the attributes for D:\MyFiles.
S=System.

To remove the system attribute, from a command prompt, run:

Code: Select all

attrib -S D:\MyFiles
If the folder is hidden, you may need to remove the hidden attribute too:

Code: Select all

attrib -S -H D:\MyFiles
You may need to re-index after removing the hidden or system attribute.
You can force a re-index by clicking Tools -> Options -> Indexes -> Force rebuild.
toriatama
Posts: 25
Joined: Sun Oct 12, 2014 4:49 am

Re: Many files not found on local NTFS HDD

Post by toriatama »

To remove the system attribute, from a command prompt, run:

Code: Select all

attrib -S D:\MyFiles
That's solved my problem! Thank you so much! I don't why windows changed my folder into System Folder, that's weird, but now I removed the system attribution and Everything is working fine, even with the option "ignore system files & folders" enabled.

I am very happy! Many many thanks for your kind help!
php
Posts: 7
Joined: Mon Nov 03, 2014 12:53 pm

Re: Many files not found on local NTFS HDD

Post by php »

Hello, I've got the same or a similar problem, so I checked all of the above and forced an index rebuild, and in debug mode, there was no error message re my hdd.

Both Win XP SP3 and Everything are fresh installs from Oct, 2014.

I've got my stuff on an external hdd, of which some directories are properly indexed by Everything, and others are not, and after rebuild of index, the SAME folders as before are missing from the search results lists (I sort the results by path, so don't miss the hits, but it's evident the respective folders are not there).

When I look up the folder properties (by Attribute Changer 7 which shows more than "Properties" in Windows), read-only, system, hidden, compress and archive are unchecked, and "Index" is checked"; this is identical to other folders which show in the list.

Thus, it seems Everything 1.3.4.686 (x86) does not work properly in the combination XP-and-external-drive?

Perhaps a hint: In spite of speaking of "directories" (plural) above, in fact, I am not sure if plural is correct, BUT Everything leaves out my most important folder there, which is called "1", and which by this is first on my list of folders "1", "2", etc. in that external drive, so perhaps it's a bug in the code (array, list, first element not being retrieved)?
Post Reply