1.5 not always updating file changes right away

[JTCGiants56]

Sometimes I will rename or delete a file in 1.5, and the file will revert back to the old name, or will stay in the list. If I search the new name or if the file is deleted in 1.4, it will show the updated name or that the file is deleted if I had deleted it. If I wait a bit (sometimes 30+ seconds), it will have finally updated with the changes in 1.5.

This doesn't happen everytime I make a change, but frequent enough that I'm posting this. I also never have this problem in 1.4, and every change made is reflected right away.

The only real difference between the two versions as far as my settings, is my use of several properties that i'm indexing in 1.5.

Other than that, I'm at a loss. Any ideas?

[void]

Property indexing may cause this.

What properties are you indexing?
What's shown under Help -> Troubleshooting information?
What type of indexes are slow to update? (FAT/network drive/NTFS volume?)



Debug logging may help to capture the issue.



Everything 1.5.0.1362a may help as a successful file operation will now update your FAT/remote/folder index immediately.

[JTCGiants56]

Property indexing may cause this.

What properties are you indexing?
What's shown under Help -> Troubleshooting information?
What type of indexes are slow to update? (FAT/network drive/NTFS volume?)



Debug logging may help to capture the issue.



Everything 1.5.0.1362a may help as a successful file operation will now update your FAT/remote/folder index immediately.

I'm indexing several video related properties such as frame rate, width, length, etc and these only apply to video type files.

This will occur even when properties indexing (shown by the bottom right progress bar) is not running.

These are all NTFS drives.

Everything: 1.5.0.1361a (x64)
OS: Windows NT 10.0 22631 (x64)
Admin: 1
Service: 1 (connected / installed and running)
Command line:
Binary: C:\Program Files\Everything 1.5a\Everything64.exe
Profile: C:\Users\x\AppData\Roaming\Everything\Everything-1.5a.ini
Database: C:\Users\x\AppData\Local\Everything\Everything-1.5a.db
Instance: 1.5a
Config: allow_multiple_windows=1
Config: show_mouseover=0
Config: check_for_updates_on_startup=1
Config: highlight_max_or_paths=256
Config: offline_alpha=128
Config: language=1033
Config: show_size_in_statusbar=0
Config: auto_include_removable_volumes=1
Config: auto_remove_offline_ntfs_volumes=1
Config: auto_include_fixed_refs_volumes=1
Config: auto_remove_offline_refs_volumes=1
Config: find_first_file_path_not_found_retry_timeout=30000
Config: input_stream_buf_size=0
Config: output_stream_buf_size=0
Config: icon_blend_hidden=1
Config: thumbnail_medium_text_lines=2
Config: thumbnail_large_text_lines=2
Config: filelist_thumbnails=1
Config: open_many_files_warning_threshold=16
Config: set_foreground_window_attach_thread_input=0
Config: allow_multiple_windows_from_tray=1
Config: search_edit_move_caret_to_selection_end=0
Config: search_edit_drag_accept_files=0
Config: focus_search_on_activate=1
Config: keep_result_focus_in_view=0
Config: full_row_select=0
Config: scale=1.200000
Config: paste_new_line_op=0
Config: match_whole_filename_when_using_wildcards=0
Config: size_number_format=4
Config: exclude_files=*.jpg;*.lnk;*.jpeg;*.torrent;*.symlink;*.png
Config: index_date_created=1
Config: fast_date_created_sort=1
Config: fast_date_accessed_sort=1
Config: index_attributes=1
Config: fast_attributes_sort=1
Config: fast_extension_sort=1
Config: db_update_thread_priority=-15
Config: index_recent_changes=1
Config: refs_file_id_extd_directory_info_buffer_size=0
Config: folder_update_thread_mode_background=1
Config: monitor_thread_mode_background=1
Config: content_buf_size=0
Config: content_include_only_folders=C:\;X:\E_DRIVE_xv1\Dropbox\[[TEXT FILES]]
Config: content_exclude_folders=C:\Program Files;C:\Program Files (x86);C:\ProgramData;C:\Users\x\AppData;C:\Users\x\Documents
Config: content_include_only_files=*.txt
Config: content_exclude_recall_on_data_access=0
Config: show_copy_path=1
Config: context_menu_parent_folder=4
Config: filter_visible_count_max=0
Config: filter=EVERYTHING
Config: preview_icon=1
Config: findbar_highlight_all=0
Config: ntfs_volumes=

[void]

Thank you for the information.

monitor_thread_mode_background=1
folder_update_thread_mode_background=1

Could you please try disabling thread_mode_background.

Everything will not detect any changes until there's no disk IO.

To disable thread_mode_background:

• In Everything 1.5, from the Tools menu, click Options.
• Click the Advanced tab on the left.
• To the right of Show settings containing, search for:
  thread
• Select monitor_thread_mode_background.
• Set the value to: false
• Select folder_update_thread_mode_background.
• Set the value to: false
• Click OK.

This will occur even when properties indexing (shown by the bottom right progress bar) is not running.

The progress is only shown in the status bar on first index.
Everything will be reading the properties for new files in the background without showing any progress.
Everything will be re-reading the properties for modified files in the background without showing any progress.



The Everything Service might be failing.

Could you please check your event viewer for any Everything Service issues:

• From the Start menu, search for: eventvwr
• Click Event Viewer.
• In the Event Viewer, on the left, navigate to: Windows Log -> System
• On the right, click Filter Current Log....
• Change <All Event IDs> to: 7034
• Click OK.
• Are there any errors listed for the Everything Service?

[tuska]

Admin:   1
Service: 1 (connected / installed and running)

[JTCGiants56]

Thank you for the information.

monitor_thread_mode_background=1
folder_update_thread_mode_background=1

Could you please try disabling thread_mode_background.

Everything will not detect any changes until there's no disk IO.

To disable thread_mode_background:

• In Everything 1.5, from the Tools menu, click Options.
• Click the Advanced tab on the left.
• To the right of Show settings containing, search for:
  thread
• Select monitor_thread_mode_background.
• Set the value to: false
• Select folder_update_thread_mode_background.
• Set the value to: false
• Click OK.

This will occur even when properties indexing (shown by the bottom right progress bar) is not running.

The progress is only shown in the status bar on first index.
Everything will be reading the properties for new files in the background without showing any progress.
Everything will be re-reading the properties for modified files in the background without showing any progress.



The Everything Service might be failing.

Could you please check your event viewer for any Everything Service issues:

• From the Start menu, search for: eventvwr
• Click Event Viewer.
• In the Event Viewer, on the left, navigate to: Windows Log -> System
• On the right, click Filter Current Log....
• Change <All Event IDs> to: 7034
• Click OK.
• Are there any errors listed for the Everything Service?

Thank you, I've made the changes in advanced settings and will test it out. Out of curiosity, is there any downside to disabling these options?

I did not see any errors for everything in the event log.

Admin:   1
Service: 1 (connected / installed and running)

I don't know why it is running as admin. I've checked the properties of the everything.exe and run as administrator is not checked off. It currently has the "-" check to run as administrator in everything's general settings, and everytime I uncheck it and restart everything, the "-" is set automatically again.

[tuska]

I don't know why it is running as admin.
I've checked the properties of the everything.exe and run as administrator is not checked off.
It currently has the "-" check to run as administrator in everything's general settings,
and everytime I uncheck it and restart everything, the "-" is set automatically again.

Please have a look here:
Common causes for Everything running as administrator

[JTCGiants56]

I don't know why it is running as admin.
I've checked the properties of the everything.exe and run as administrator is not checked off.
It currently has the "-" check to run as administrator in everything's general settings,
and everytime I uncheck it and restart everything, the "-" is set automatically again.

Please have a look here:
Common causes for Everything running as administrator

Thanks ,I tried those without luck. In processexplorer everything is showing with no parent processes and the integrity at high.

@Void, unfortunately disabling those two advanced settings and upgrading to the latest beta has not solved the main issue.

[tuska]

I don't know why it is running as admin.
I've checked the properties of the everything.exe and run as administrator is not checked off.
It currently has the "-" check to run as administrator in everything's general settings,
and everytime I uncheck it and restart everything, the "-" is set automatically again.

Please have a look here:
Common causes for Everything running as administrator

Thanks, I tried those without luck. In processexplorer everything is showing with no parent processes and the integrity at high.

Have you also seen/checked the post a little further down below the linked post

[void]

The indeterminate "-" check means Everything is not set to run as administrator.
However, Everything was launched as administrator by the parent process.

Please check tuska's post above as to why the parent process is running elevated or why Everything is launching Everything as administrator.

@Void, unfortunately disabling those two advanced settings and upgrading to the latest beta has not solved the main issue.

Could you please capture some debug logs:

• In Everything, from the Tools menu, under the Debug submenu, check Start Debug Logging.
  --- wait for Everything to miss changes.
• In Everything, from the Tools menu, under the Debug submenu, click Stop Debug Logging.
  The Everything Debug Log will open in Notepad.
• Please save this file to the Desktop and send to support@voidtools.com

The logs will show why Everything is missing changes.

Privacy

Out of curiosity, is there any downside to disabling these options?

I did not see any errors for everything in the event log.

None, I have heard of issues where Everything doesn't update network drives when these settings are enabled.
Nothing of interest is written to the logs when these settings are enabled.

THREAD_MODE_BACKGROUND_BEGIN

[JTCGiants56]

The indeterminate "-" check means Everything is not set to run as administrator.
However, Everything was launched as administrator by the parent process.

Please check tuska's post above as to why the parent process is running elevated or why Everything is launching Everything as administrator.

@Void, unfortunately disabling those two advanced settings and upgrading to the latest beta has not solved the main issue.

Could you please capture some debug logs:

• In Everything, from the Tools menu, under the Debug submenu, check Start Debug Logging.
  --- wait for Everything to miss changes.
• In Everything, from the Tools menu, under the Debug submenu, click Stop Debug Logging.
  The Everything Debug Log will open in Notepad.
• Please save this file to the Desktop and send to support@voidtools.com

The logs will show why Everything is missing changes.

Privacy

Out of curiosity, is there any downside to disabling these options?

I did not see any errors for everything in the event log.

None, I have heard of issues where Everything doesn't update network drives when these settings are enabled.
Nothing of interest is written to the logs when these settings are enabled.

THREAD_MODE_BACKGROUND_BEGIN

I've captured this moment of it erroring out in the log, let me know if you still need the full log:

Code: Select all

2023-12-15 21:15:27.110: 081c000000000045: bad al retrying...
2023-12-15 21:15:27.375: 081c000000000045: bad al retrying...
2023-12-15 21:15:27.641: 081c000000000045: bad al retrying...
2023-12-15 21:15:27.906: 081c000000000045: bad al retrying...
2023-12-15 21:15:28.172: 081c000000000045: bad al retrying...
2023-12-15 21:15:28.438: 081c000000000045: bad al retrying...
2023-12-15 21:15:28.703: 081c000000000045: bad al retrying...
2023-12-15 21:15:28.969: 081c000000000045: bad al retrying...
2023-12-15 21:15:29.234: 081c000000000045: bad al retrying...

And it repeats that several times after rebuilding.

[void]

Thank you for the logs.

2023-12-15 21:15:27.110: 081c000000000045: bad al retrying...

The NTFS attribute list is invalid.
This can happen with fragmented files.
It can take up-to 60 seconds to sync up correctly.



To disable waiting for a valid attribute list:

• In Everything 1.5, from the Tools menu, click Options.
• Click the Advanced tab on the left.
• To the right of Show settings containing, search for:
  ntfs
• Select get_ntfs_file_record_retry_timeout.
• Set the value to: 0
• Click OK.

I don't recommend doing this as Everything will hard link filenames and file sizes when files are modified.

[JTCGiants56]

Thank you for the logs.

2023-12-15 21:15:27.110: 081c000000000045: bad al retrying...

The NTFS attribute list is invalid.
This can happen with fragmented files.
It can take up-to 60 seconds to sync up correctly.



To disable waiting for a valid attribute list:

• In Everything 1.5, from the Tools menu, click Options.
• Click the Advanced tab on the left.
• To the right of Show settings containing, search for:
  ntfs
• Select get_ntfs_file_record_retry_timeout.
• Set the value to: 0
• Click OK.

I don't recommend doing this as Everything will hard link filenames and file sizes when files are modified.

Thank you. Other then disabling this, I'm guessing the only solution is to defragment all my drives, or possibly a chkdsk can help? Also, why would this issue not happen in 1.4?

[void]

I'm looking into ways to improve this delay..

Defragging might help, but it could be caused by a os file with several hard links.



Everything 1.4 will often miss filenames and file sizes.

[JTCGiants56]

I'm looking into ways to improve this delay..

Defragging might help, but it could be caused by a os file with several hard links.



Everything 1.4 will often miss filenames and file sizes.

Understood, thanks. To note, the files I'm renaming are not in hardlinked folders, and all the drives I'm renaming files on are non-OS drives. Although I do have several hardlinked folders scattered around. I'm unsure if that matters in this circumstance.

Appreciate your work on this app.

[void]

It's not the file you are renaming.

It's another file.

Everything won't detect your rename until the other file updates.

[void]

Everything 1.5.0.1364a improves monitoring NTFS volumes.

Everything will now use OpenFileByID to read changes to files instead of reading directly from the NTFS MFT.

Everything should now update instantly.



The OpenFileByID API call was broken on Windows 10.
Using this API would cause Windows Explorer to stop updating.

Everything will not use this API call on Windows 10 18362 and earlier.
Windows 10 18363 fixed the issue.



Everything will continue to read the NTFS MFT directly for updates on Windows XP and earlier.
(and on Windows 10, 18362 and earlier)



The OpenFileByID API call can be disabled with the advanced ntfs_open_file_by_id setting.

To disable the OpenFileByID API call:

• In Everything 1.5, from the Tools menu, click Options.
• Click the Advanced tab on the left.
• To the right of Show settings containing, search for:
  ntfs
• Select ntfs_open_file_by_id.
• Set the value to: false
• Click OK.

[raccoon]

Can OpenFileById be used to improve reading changes within FAT32 and exFAT volumes? I can't find anything saying that this API doesn't work for FAT systems. Indeed, it can be used to access Floppy devices according to MSDN notes.

[void]

Short answer: no.

The ID for FAT files doesn't always refer to the same file.



OpenFileById doesn't help with detecting changes.

[JTCGiants56]

Everything 1.5.0.1364a improves monitoring NTFS volumes.

Everything will now use OpenFileByID to read changes to files instead of reading directly from the NTFS MFT.

Everything should now update instantly.



The OpenFileByID API call was broken on Windows 10.
Using this API would cause Windows Explorer to stop updating.

Everything will not use this API call on Windows 10 18362 and earlier.
Windows 10 18363 fixed the issue.



Everything will continue to read the NTFS MFT directly for updates on Windows XP and earlier.



The OpenFileByID API call can be disabled with the advanced ntfs_open_file_by_id setting.

To disable the OpenFileByID API call:

• In Everything 1.5, from the Tools menu, click Options.
• Click the Advanced tab on the left.
• To the right of Show settings containing, search for:
  ntfs
• Select ntfs_open_file_by_id.
• Set the value to: false
• Click OK.

Thanks for your hard work as always void!

[void]

Everything 1.5.0.1367a reverts back to reading NTFS file records directly.


OpenFileByID is not working on some files and missing file sizes.


Everything will fall back to OpenFileByID if reading NTFS file records fails (Attribute list is not flushed to disk).

This should give the best performance and accuracy.
With the revert, Everything should still be updating right away.
You shouldn't see any lag when changing files.



There's a few edge cases where both reading the file record and OpenFileByID fail, but it should be really rare.
If this happens, the file size and date modified will appear blank.
Everything will pickup the correct size and date modified the next time the file is modified, or all handles to the file are closed if the file was currently opened, or when you force a rebuild from Tools -> Options -> Indexes.