Optimal Settings for Monitoring of Future Installations

[hairypaulsack]

Hey all!

First of wanted to thank the developers for the wonderful open software which is a great alternative and a relief for battles with windows indexing hogging up my resources! Second thank goes out to the community and developers for the help and quick responses.

CONTEXT:
I'm a little unsure of the optimal settings to observe changes to the HDD. After reading the support section of the website I came out knowing a little more but not necessarily what to invoke as far as settings. So I have my Program Files and AppData directories on my C:/ HDD with the "Enable USN Journal; Include in database; Include USN Journal in recent changes; Monitor changes". I felt like my question wasn't as simple as asking what is the difference between "Include USN Journal in recent changes and Monitor changes". While typing this again I glanced at the support at (sadly) realized that the answer was right in front of me.

I believe I have it correct please tell me if I am wrong. So it's stated that Everything only monitors changes while its running. What exactly is this in reference to? There was the option in installation that was "Start Everything with Windows" which I did not select because I do not use Everything that much due to still using windows native indexing for certain reasons. I am not sure if everything is running when it is in the system tray as some programs just use that as a quick way to start a program and/or have to program somehow pre-loaded as in the case of programs that utilize a "fast boot" by having something running in the background as a background process. I did indeed take the route to have the service running in the background. This only allows bypass of UAC and does not provide real time monitoring? I guess I can open Everything every time I'm about to install a program but it would be nice to always have that monitored as I do not really care about devoting a few GB's for this reason.

SUMMARY OF QUESTIONS and/or POINT OF CLARIFICATION : (simple true or false response w/ question number would more than suffice)
1. Service is for sole purpose of not needing UAC prompt upon execution of program?
2. If the everything icon is in the system tray is the program actually running or in some pre-loaded state?
3. USN Journal loosely translated means "this is the main thing that does the indexing"?
4. Monitor Changes indexing option is solely in reference to monitoring changes while program is running?
5. Monitor Changes check box's tool tip states that it monitors the alteration/changes of file names. Can I assume this includes the addition and removal as well?
6. Including the USN Journal in recent changes allows me to have real time monitoring regardless of Everything running?

Hope I was detailed enough to properly identify my inquiry as I believe this would be the perfect tool to add to my uber(tool)box I want to ensure that I have it properly setup

EDIT: If anyone has an suggestions for things I didn't mention or anything to aid in my ability to hunt malicious activities it would be greatly appreciated.

Thanks,
Hairy "Paul" Sack

[void]

Currently, Everything requires an active USN Journal to index a NTFS volume.
Even when monitoring is disabled, Everything will still require an active USN Journal just to index the volume.
This is a limitation with the NTFS driver and Everything.
This will change in the next release, an active USN Journal will not be required to index a NTFS volume for the next release.

So it's stated that Everything only monitors changes while its running. What exactly is this in reference to?

Everything monitors changes to the USN Journal while it is running.
The USN Journal is part of NTFS, and is not part of Everything.

A USN Journal is present on most NTFS volumes (XP or before is optional, Vista or later is always active).
The USN Journal logs all changes to the file system by the NTFS driver.
The USN Journal is quite small usually only about 32 MB, this can record up to a couple days of changes (depending on file system usage).
This makes it possible for Everything to update its database when it hasn't been running for a few days without the need to reindex the entire NTFS volume.

I am not sure if everything is running when it is in the system tray as some programs just use that as a quick way to start a program and/or have to program somehow pre-loaded as in the case of programs that utilize a "fast boot" by having something running in the background as a background process.

When you run Everything on startup, the entire program is running, the entire database is loaded into ram and the USN Journal monitoring will also start.
There is no quick start process.

I did indeed take the route to have the service running in the background. This only allows bypass of UAC and does not provide real time monitoring?

The Everything service is a "wrapper" to allow the Everything client to read NTFS master file tables.
The service is stateless, meaning it does not do any monitoring or indexing itself.

I'm probably answering you twice here, sorry..

1. Service is for sole purpose of not needing UAC prompt upon execution of program?

Yes, The Everything service is to help avoid the UAC prompt when requiring low level read access to NTFS volumes.
It is not required if you run Everything as an administrator.

2. If the everything icon is in the system tray is the program actually running or in some pre-loaded state?

Normally, this would indicate Everything is fully loaded, up to date, monitoring new changes and ready to use.
However, This can be changed by using the -load-delay command line option.
Setting a load delay can reduce the impact Everything will have with other apps that may be also loading on startup.
In this state Everything would be considered preloaded.
Opening a search window will force the load to begin.

3. USN Journal loosely translated means "this is the main thing that does the indexing"?

Log of changes to the file system.
It's the "thing" that allows Everything to keep the indexes up to date in real-time or while Everything is not running.

4. Monitor Changes indexing option is solely in reference to monitoring changes while program is running?

When monitor changes is enabled for a volume, Everything will keep that volume up to date.
This will be done in real-time when Everything is running, and when the Everything is not running, the USN Journal will be used to update the index.

5. Monitor Changes check box's tool tip states that it monitors the alteration/changes of file names. Can I assume this includes the addition and removal as well?

Monitor changes includes all changes to filenames, including adding new files/folders and deleting files/folders.

6. Including the USN Journal in recent changes allows me to have real time monitoring regardless of Everything running?

"Including the USN Journal in recent changes" is used for the recent change search modifier. It has no effect on monitoring.
Recent changes can be used in Everything to instantly locate files modified since Everything was started.
When this option is enabled, the entire USN Journal is loaded into the recent change list, so you can instantly locate files modified within the last few days, even if Everything was not running.
I would recommend leaving this disabled if you are not using recent changes as it can use slightly more memory and causes the database to take a few seconds longer to load.

Hope this helps.

[hairypaulsack]

Truly exceptional response that I do not think could have been answered in a more clear concise manner; I truly appreciate the well versed info.

In the manner of the pre-boot/fast boot notion I can say the only reason that my neurons reached to the point of excitation would be in noticing the Solidworks install my work gave me to take home for learning and use has this startup process called "sldworks_fs.exe"; upon googlin' it, primary "preview" results lend me to Solidworks fast start. I could swear chrome has a fast boot process as well. I know this is unrelated but you seem to be very knowledgeable and I was curious when you stated there was no such thing as something pre-reloaded in ram. I was wondering if you could clarify or reiterate. Or did you simply mean that Everything has no quick start process.

Again thanks for the attention to detail and the lengthy response as I greatly appreciated learning the semantics and basic inner workings of the program and file system.. Honestly knowledge for what some might never think yet about truly drives me in life

Thanks
Paul

[void]

I was wondering if you could clarify or reiterate. Or did you simply mean that Everything has no quick start process.

There is no quick start process.

This might be similar to what I would call running the background.
When there are no Everything search windows opened, Everything will remain running in the background.
This is shown by the system tray icon.
There is only one Everything process, this manages all the search windows and running in the background when no search windows are opened.

While Everything is running in the background, it will keep the indexes up to date by monitoring the NTFS USN Journals. It will also allow you to quickly open a new search window.

If Everything was not running in the background, opening a new Everything search window would require reloading the Everything database, which may take a few seconds before your search window is shown.
Whereas when Everything is running in the background, opening a new Everything search window will be instant.

You can toggle run in background from Tools -> Options -> UI -> Run in background.
When this is disabled, closing all search windows will cause the Everything process to end.
The next time you open a new search window, Everything will need to reload the database and make sure the indexes are up to date.