What is this "Enable USN Journal" for?

[Biff]

What is this "Enable USN Journal" for?


Better to activate it or not?

[void]

The USN Journal is a log of all the recent changes to your NTFS volume.

It is maintained by your NTFS driver (not Everything)

Please leave it enabled.

[Biff]

It is maintained by your NTFS driver (not Everything)

Sorry, do not know what this is.

Please leave it enabled.

OK, I will do. Whatfor?

[NotNull]

Whatfor?

Windows uses a couple of hidden Windows 'bookkeeping files' to keep track of all files and folders on NTFS disks.
Everything uses (reads) two of these to 'do its thing'.

The first one is the Master File Table, It is the 'addressbook' where all your files are on that disk. This is used for Everythings initial scan of the disk.

The second one is the USN Journal. It is a list of all recent changes to this disk. Reading this file frequently, enables Everything to update its index in real time.


So: Everything needs it for real-time updates.

[Biff]

OK, thank you. So why is it enabled then by default?

It seems it can be enabled only when the drive is online? With all offline drives it seems that it cannot be activated, it is greyed out.

Should I enable this as well(?):

[NotNull]

OK, thank you. So why is it enabled then by default?

It seems it can be enabled only when the drive is online?

It is already enabled by Windows. Windows needs this too.

The USN Journal of that disk is on the disk itself, So when it is off-line, changing the setting is impossible.

Should I enable this as well(?):

In general, this should stay disabled.
If you never use the rc: function, this is not needed
When you keep Everything running (background or foreground) this is not needed.

Some background:
When recent changes are indexed (Options > Indexes: Index recent changes), Everything keeps a separate list of changes to files/folders since Everything was started.

Load USN Journal into the Recent Changes database extends this recent changes list of files with the ones found in the USN Journal (about 2 weeks worth of changes).
If you keep Everything running, there will be nothing extra to add as these changes are already found by Everything.

If you never look for recent changes (rc: function), Index recent changes as well as Load USN Journal into the Recent Changes database then both can be unchecked.

[Biff]

It is already enabled by Windows. Windows needs this too.

Sorry, I meant the option in Everything.

The USN Journal of that disk is on the disk itself, So when it is off-line, changing the setting is impossible.

So there obviously is no option to do possible that would let one enable it when the drive is offline and when it goes online to automatically execute the enabling.

OK, thank you for the background, too.

So then all is checked here what shall be checked. Besides of "Enable USN Journal". I will try to check it for each drive when these drives with that options uncechked are online, if I don't forget.

[harryray2]

Out of interest, any idea where the USN journal is kept, is it a file or several files? I use Win 7

[NotNull]

It is one file, not part of the normalfilesystem as-you-know-it.
It is visible for example in WizTree (when running elevated):

2022-08-27 22_44_56-[C_] Local Disk - WizTree.png (74.88 KiB) Viewed 7474 times

The file $UsnJrnl appears to be of size zero, but the actual content is in an Alternate Data Stream (ADS) named $J:

2022-08-27 22_48_08-__._PhysicalDrive0_1.ntfs_[SYSTEM]_$Extend_$UsnJrnl_.png (23.09 KiB) Viewed 7474 times

(screenshot from 7-Zip's 7zfm.exe)

[Biff]

It is visible for example in WizTree (when running elevated):

That program is incredibly fast, even on my slow computer. Over a million files in a few seconds.

Why do you have an old WizTree, from 2018?

[harryray2]

Thanks, I'll take a look...I did do a search for usnjrnl a while back with everything but obviously no results were returned. I'll give Wizytree a try. it may be useful.

[NotNull]

That program is incredibly fast, even on my slow computer. Over a million files in a few seconds.

That is because it uses "Everything-technology" to get the list of files
(and that is why it has to be run elevated, because it needs access to the Master File Table)

Why do you have an old WizTree, from 2018?

(This is getting even more off-topic...)
My prehistoric laptop that I use for to play with Windows and applications (it is my main Everything-experiment system too) is slowly dying.
So I took another one I have lying around (much faster ) and have yet to update all software...

'll give Wizytree a try. it may be useful.

I use it to find out which folders take up most space. Easier overview compared to Everything.
In the screenshot you see a "File View" too. It is a rudimentary Everything replacement.

[Biff]

That is because it uses "Everything-technology" to get the list of files

The same technology as Everything or technology developped for Everything?

(and that is why it has to be run elevated, because it needs access to the Master File Table)

So that is why Everything has to run elevated as well or with that server.

My prehistoric laptop that I use for to play with Windows and applications (it is my main Everything-experiment system too) is slowly dying.
So I took another one I have lying around (much faster ) and have yet to update all software...

Ah, OK, I understand.

[harryray2]

Presumably, Everything would find the files if I added \\.\Physical Drive0\1.ntfs\[SYSTEM]\$Extend\$UsnJrnl:?

[NotNull]

The same technology as Everything or technology developped for Everything?

The same technology as Everything. Both read the MFT.

So that is why Everything has to run elevated as well or with that server.

elevated or by enabling Everything Service. The Everything Service runs elevated, so the rest (database,GUI,etc) can run as a regular user.

Presumably, Everything would find the files if I added \\.\Physical Drive0\1.ntfs\[SYSTEM]\$Extend\$UsnJrnl:?

Don't think that it is that simple. Everything sees more than it shows, so some parts are filtered out.
Also: the \\.\ syntax is part of the Windows device namespace. I suspect Everything focuses on the File namespace as that is where the 'normal' files are.

[Biff]

The same technology as Everything. Both read the MFT.

That appears to do quite a lot of programs. The ones which are very fast. The programs not doing that (I assume, e.g. FreeCommander) actually not sensibly usable (one might think) are pretty slow, I assume.

elevated or by enabling Everything Service. The Everything Service runs elevated, so the rest (database,GUI,etc) can run as a regular user.

So to have created the Everything Service sounds like a great idea. No admin rights needed. Better security.

[NotNull]

So to have created the Everything Service sounds like a great idea. No admin rights needed. Better security.

Exactly! This void-dude isn't as stupid as we all thought all this time

[harryray2]

Don't think that it is that simple. Everything sees more than it shows, so some parts are filtered out.

Is it technically feasible to have an option for Everything to show everything with nothing filtered out?

[Biff]

Exactly! This void-dude isn't as stupid as we all thought all this time

Oops, holy sheet, I'm not sure if that's an incredible insult like I've never heard before or the opposite. I guess, the opposite. Yes yes, indeed, that is absolutely very true.

[harryray2]

Don't think that it is that simple. Everything sees more than it shows, so some parts are filtered out.

Is it technically feasible to have an option for Everything to show everything with nothing filtered out?

[void]

Everything only indexes and monitors files under the volume root.
(Special NTFS files such as $UsnJrnl are ignored)

I will consider an option to show these files.
However, they will be inaccessible.

[harryray2]

Thanks, an option to show would be useful.

[harryray2]

Notnull, how did you access PhysicalDrive0_1.ntfs_[SYSTEM]_$Extend_$UsnJrnl on 7zip?

Thanks.

[NotNull]

See below:

You need to start 7zfm.exe as administrator for this to work.