1.5.0.1335a Doesn't index all files in virtual folder.

[SeanBannister]

The notes for 1.5 mention support for indexing virtual folders, I added my box.com drive (which is mounted using their software version 2.30.87) to Everything's list of Folders. However I'm only seeing 11% of files indexed.

I ran some tests to figure out why this might be and noticed if I access the folders that are missing the files I can then Rescan and the files get indexed. I presume the way that Everything crawls directories isn't triggering the Box.com software to pull the directory listings from their server on all occasions?

I've tried running a Rescan multiple times to test if Everything can find more files on its own each time it rescans but it doesn't.

My rudimentary fix is opening a command prompt and running "dir /S" in the box.com virtual folder which recursively lists every file and I guess adds them to the cache in the box.com software which Everything will then index.

My current settings: https://imgur.com/a/ttJL1s4

[void]

Hi SeanBannister,

Thank you for the bug report.

Everything shouldn't be using virtual folder indexing here. (IShellFolder::EnumObjects)
Everything should be using just regular folder indexing. (FindFirstFile)

There's most likely a retry error or a directory downloading error.

Could you please check your debug log:

• Avoid using dir /S so indexing fails.
• In Everything, from the Tools menu, under the Debug submenu, click Start Debug Logging.
• In Everything, from the Tools menu, click Options.
• Click the Folders tab on the left.
• Select your Box folder and click Rescan Now.
• Wait for the rescan to complete (progress is shown at the top right or in the status bar)
• In Everything, from the Tools menu, under the Debug submenu, click Stop Debug Logging.
  --This will open your %TEMP%\Everything Debug Log.txt in Notepad
  Please search this file for:
  FindFirstFileEx <error-code> <path>
• Is there an error code? what is the error code? how many are there?

[SeanBannister]

Thanks for the reply. As a fellow software engineer I wanted to help as much as possible so I'm running a fresh version of Everything portable to eliminate as many variables as possible.

I followed your instructions and unfortunately couldn't find any "FindFirstFileEx" references in the file. I've emailed you a copy of the log file in case it helps you. Let me know if there's anything else you need me to test.

[void]

Thank you for the debug logs SeanBannister,

The logs show FindNextFile failing.
No reason is given in the logs.

Everything 1.5.0.1336a adds a retry to FindNextFile on some failure conditions.
This version also adds more debug information.

If the issue persists, please let me know.

[SeanBannister]

Thanks, I've installed this version, I saw your instructions in your email but decided to do a completely fresh install and also reinstalled box.com just in case.

Interestingly its still scanning and has been for about 2 days. The box drive I'm indexing has about 3 million files so its always taken a while to index, I used to do it with Everything 1.4 by mounting it as a drive letter. But this seems to be slower, you mentioned this version retries on error so I'm wondering if its that.

I'll keep you posted.

[SeanBannister]

So... I reinstalled Box and removed all copies of Everything (1.4 and 1.5) and found I could complete 1 full scan of my Box drive, really need to test again but it's a multi day endeavor. Sorry I should of done this in the first place but I was also experiencing the same problem when scanning my FTP server so I've emailed you the debug log for this failing. Both of these have 3 million+ files and take multiple days to complete indexing so I think this might be why I'm experiencing these errors and other users aren't, it often takes a day before it fails.

As a temporary fix for this I found if I mount the FTP with WebDrive Everything can complete a scan successfully. For testing purposes I ran both a regular FTP scan and a WebDrive FTP scan at the same time with two separate copies of Everything and the regular FTP failed while the WebDrive succeeeded.

This has made me realize Everything fails silently when it doesn't complete a scan. The only reason I realized it wasn't completing is I was moving from 1.4 to 1.5 and created a fresh index so I was able to scroll through the parent directories and see some weren't indexed. I'm worried in the future I wouldn't realize if a scan failed and in my use case this could cause me to miss some important files I need to access. Would it be possible to display an on screen error message when a scan doesn't complete successfully?

Also while testing this I thought I'd be able to turn on "Fast rescan" to skip the already indexed files, and rescan the files that were failing. It appears there might be a bug here because it won't rescan the missing folders. Is this bug because it's saved the date the parent directory was scanned before it finished scanning it so because it failed on a child directory the next time it gets to that parent it thinks it was completely scanned even though it failed halfway through? Or maybe it's just failing at exactly the same file every time? I've also emailed you the debug log for this.

Thanks for your help with this.

[void]

Would it be possible to display an on screen error message when a scan doesn't complete successfully?

I will look into logging any errors.

For now, you'll need to enable debug logging.

Is this bug because it's saved the date the parent directory was scanned before it finished scanning it so because it failed on a child directory the next time it gets to that parent it thinks it was completely scanned even though it failed halfway through?

Yes, that is possible.

Please disable fast rescan.

Or maybe it's just failing at exactly the same file every time?

Also possible.
Verbose debug logs would pick this up.