correct format for multiple search syntax

harryray2 · Post by **harryray2** » Sun Oct 10, 2021 3:36 pm

Is <dc:last1mins>|<dm:last1mins> correct for showing both DC and DM at the same time in the same search window?

Thanks.

harryray2 · Post by **harryray2** » Mon Oct 11, 2021 4:35 pm

I don't think this works..Any idea of the correct format please?

Post by **therube** » Mon Oct 11, 2021 4:58 pm

You want AND, so remove | & simply leave a space between (the two functions).

harryray2 · Post by **harryray2** » Mon Oct 11, 2021 5:08 pm

Thanks, that was the first thing I tried but it doesn't seem to work.

dc:last25mins dm:last25mins

Post by **therube** » Mon Oct 11, 2021 5:18 pm

Note that new or updated files should show up.
But stale files (i.e., outside of, older then, the dm:last1minute) are not "automatically" removed.

So if you run,
dm:last1minute
Then create a new file,
DIR > C:\TMP\XXX
XXX should show up,
& will persist past the "1 minute" criteria. Will persist, forever actually - until it is either updated or deleted or you perform some search operation that changes the existing search (like changing from dm:last1minute to 'dm:last1minute E:').

So in that respect, as a "directory monitor", depending on your wants, Everything itself may not be optimum.
And Everything's Index Journal might ? be better in that regard?

harryray2 · Post by **harryray2** » Mon Oct 11, 2021 5:31 pm

The problem is, is that when entered individually the total number of files is 83, but when done together the total is 51 files.

I also tried pausing indexing but got the same results

I then tried opening two individual windows, one with dc:last25mins and one with dm:last25mins..I got the same result.

The results seem to be from the first parameter entered.
So if I enter dm:last25mins dc:last25mins I get the results from DM but if I enter dc:last25mins dm:last25mins, I get the results from DC

That's a nice ider about the index journal but Everything wont allow me to have two journals running at the same time.

Post by **therube** » Mon Oct 11, 2021 5:44 pm

Set search to dm:last1minute
note the files
wait > 1 minute
Change search to dm:last1minute dc:last1min
note the files
- they will be different. Any < dm:last1minute will have dropped off
Create a new file, C:\TMP\XXX
XXX turns up

wait > 1 minute
Open a new window, search, dm:last1minute
Its' results will be different from your original window

I'm thinking it is easier to discern what is going on with sorter period duration (like the 1minute) vs. longer.
Also try to have a file in there that periodically automatically updates. Like your browser session history likely updates every so often. Mozilla (SeaMonkey) in my case, & sessionstore.json periodically updates. (Actually, if you're watching, you'll see a new sessionstore-1.json is created, followed by a delete of sessionstore.json, followed by a rename of sessionstore-1.json to sessionstore.json.)

harryray2 · Post by **harryray2** » Mon Oct 11, 2021 5:50 pm

I've tried everything that I can think of including your suggestions but it's still a case of not showing both DM and DC together.
It's actually dm and dclast1mins that I use most of the time and I have the same issue.

I'll carry on pursuing this but at the same time I'll look into the index journal idea.

I'll write a support question re the index journal a bit later on.

Post by **void** » Tue Oct 12, 2021 3:46 am

The following is correct:
<dc:last1mins>|<dm:last1mins>

From my understanding you want dc:last1mins OR dm:last1mins

Please note this search is not real-time.
That is the current results will not be removed after one minute. (unless they are modified/renamed/deleted)

I don't think this works..

What happens?
What results do you see?

harryray2 · Post by **harryray2** » Tue Oct 12, 2021 7:17 am

Thanks, no, what I'm after is to show DN:last1mins AND DC:last1mins in one search results window.

I'm currently opening two separate windows.

Having both dc: and dm: in one result window doesn't appear to give me the same number of results as when a seperate window is open for each of them.

harryray2 · Post by **harryray2** » Tue Oct 12, 2021 10:22 am

I've tried this a few way but I can't seem to get Everything to show results from both dm and dc at the same time.

Post by **void** » Tue Oct 12, 2021 10:25 am

What results are missing when you search for:
<dc:last1mins>|<dm:last1mins>

Could you please give an example.

harryray2 · Post by **harryray2** » Tue Oct 12, 2021 10:55 am

I'm using dm:last1mins dc:last1mins ( dm:last1mins AND dc:last1mins) to show results from both in the search window.

If I use dm:last1mins dc:last1mins it just shows the dm results
If I use dc:last1mins dm:last1mins it just shows the dc results

Everything is not showing the resulkts from both date modified and date created in one window

Post by **void** » Tue Oct 12, 2021 11:05 am

There might be the active filter macro breaking the search.

Could you please show the debug console from Tools -> Debug -> Console.

Search for the following:
dm:last1mins dc:last1mins

What is in the debug console?

Search for the following:
dc:last1mins dm:last1mins

What is in the debug console?

dm:last1mins dc:last1mins
will show results when BOTH the date modified AND date created occurred in the last minute.

dm:last1mins | dc:last1mins
will show results when EITHER the date modified OR date created occurred in the last minute.

harryray2 · Post by **harryray2** » Tue Oct 12, 2021 11:11 am

Can I save the console or does it have to be a screenshot?

Post by **void** » Tue Oct 12, 2021 11:16 am

To copy the console text to the clipboard:

Right click the console window caption, under the Edit menu, click Select All.
Right click the selected text to copy to the clipboard.

harryray2 · Post by **harryray2** » Tue Oct 12, 2021 11:34 am

I've done them with and without a filter active.

Post by **void** » Tue Oct 12, 2021 11:37 am

Thank you for the logs.

However, you will need to show the debug console first, then perform the dm:last1mins dc:last1mins search:

Please make sure the debug console is shown.

Clear the search.
Search for the following:
dm:last1mins dc:last1mins

What is in the debug console?

Clear the search.
Search for the following:
dc:last1mins dm:last1mins

What is in the debug console?

The lines I am looking for should look something like:

Code: Select all

search 'dm:last1mins dc:last1mins' filter '' sort 5 ascending 0
parse flags 00000000 type 00c01100
TERM last1mins
parse flags 00000000 type 00c01100
TERM last1mins
FOLDER TERM START 000000002ce4d1e8 M 000000000012e0a0 N 000000000012e1c0
000000002ce4d1e8 e01100 M 000000002ce4d5a8 N 000000000012e1c0 OP 234 last1mins
000000002ce4d5a8 e01100 M 000000000012e0a0 N 000000000012e1c0 OP 236 last1mins
FILE TERM START 000000002ce4d1e8 M 000000000012e0a0 N 000000000012e1c0
000000002ce4d1e8 e01100 M 000000002ce4d5a8 N 000000000012e1c0 OP 234 last1mins
000000002ce4d5a8 e01100 M 000000000012e0a0 N 000000000012e1c0 OP 236 last1mins

harryray2 · Post by **harryray2** » Tue Oct 12, 2021 11:57 am

OK, do you need the search with and without filter?

Post by **void** » Tue Oct 12, 2021 12:00 pm

If you are normally using the filter, please send with the filter active.

harryray2 · Post by **harryray2** » Tue Oct 12, 2021 12:09 pm

Thanks, again with and without filter as I use both.

raccoon · Post by **raccoon** » Tue Oct 12, 2021 4:04 pm

I'm just adding an idea to this thread.

harryray2 wrote: ↑Sun Oct 10, 2021 3:36 pm Is <dc:last1mins>|<dm:last1mins> correct for showing both DC and DM at the same time in the same search window?

Would this suit your needs?

recentchange:last1mins
rc:last1mins

It will continually update the display with newly created and modified files, as well as moved/renamed files I believe.
You'll need to enable this option for it to work:

Tools > Options > Indexes > [x] Index recent changes
or type: /index_recent_changes=1

Post by **void** » Wed Oct 13, 2021 12:39 am

Thank you for the debug logs.

The search opcodes look correct.

Could you please send a file list of the results for both the dm:last1mins dc:last1mins and dc:last1mins dm:last1mins searches:

Clear the search.
Search for the following:
dm:last1mins dc:last1mins
From the File menu, click Export.
Change Save as type to EFU Everything file list.
Save as dm.efu
Please send this dm.efu file to support@voidtools.com
---
Clear the search.
Search for the following:
dc:last1mins dm:last1mins
From the File menu, click Export.
Change Save as type to EFU Everything file list.
Save as dc.efu
Please send this dc.efu file to support@voidtools.com

harryray2 · Post by **harryray2** » Wed Oct 13, 2021 9:35 am

Thanks, I'll do that later today.

I've been trying out Raccoon's suggestion (thanks Raccoon).

Is rc: the same as date recently changed, which, in a previous post, you mentioned was a hangover from 1.3.

As I understand it rc list files/folders that have changed since Everything is started.
It looks like, on an initial try-out, that rc works for me if I exit Everything for the period that I want to monitor file/folder modifications and creations then re-srart.

I noticed in the .ini file that there is an entry for "always_update_folder_recent_change" should this be enabled?
I also appear to get more results if I enable "include USN journal in recent changes although I do seem to get a slight performance hit.

Post by **void** » Wed Oct 13, 2021 9:45 am

Is rc: the same as date recently changed, which, in a previous post, you mentioned was a hangover from 1.3.

Yes rc: is an alias for recent change.

As I understand it rc list files/folders that have changed since Everything is started.

Correct.

I noticed in the .ini file that there is an entry for "always_update_folder_recent_change" should this be enabled?

This setting will update the date recently changed property for folders for every change single event. (even when there is no actual change)
This includes folder size changes.
Enabling will reduce the update performance of Everything.

I also appear to get more results if I enable "include USN journal in recent changes although I do seem to get a slight performance hit.

The recent changes database is empty on startup.
Enabling "Include USN journal in recent changes" will load the entire USN Journal from disk into the recent changes database on startup.

This will make your startup slower and increase the memory usage of Everything slightly.

Note: rc: searches are not real-time.
That is, stale date-recently-changed-results are not removed from your search.
Only new/modified files are added and deleted files removed.

harryray2 · Post by **harryray2** » Wed Oct 13, 2021 10:05 am

rc: seems to give the results of both dm: and dc: together, which, although not real time, seems to give me better results than dc:last15mins dm:last15mins.

Does dm also give me the results of created files/folders?

I can't quite see the difference between AND and OR when it comes to using dm and dc together.

"always_update_folder_recent_change" may be useful in certain circumstances, will it give me a noticeable performance hit.

Post by **void** » Wed Oct 13, 2021 10:15 am

Does dm also give me the results of created files/folders?

dm searches Date Modified.
A newly created file/folder will usually have the same date modified and date created.

"always_update_folder_recent_change" may be useful in certain circumstances, will it give me a noticeable performance hit.

It might, it depends on the number of changes made to your file systems.
I would not recommend enabling always_update_folder_recent_change on Windows 10 or later.

harryray2 · Post by **harryray2** » Wed Oct 13, 2021 10:38 am

Thanks for all that...Could it be that the reason I'm having the disparity between the dc and the dm results is that dc is included in dm results?

Post by **void** » Wed Oct 13, 2021 10:45 am

It's not clear what results you are seeing or what results you are expecting.

Date Modified is almost always exactly the same or later than Date Created.
A search for dm:last15mins will most likely include any files created in the last 15 mins.

I same almost and most likely because unzipping a file (or other tools) will set the date modified / date created timestamps.

raccoon · Post by **raccoon** » Wed Oct 13, 2021 12:56 pm

void wrote: ↑Wed Oct 13, 2021 10:45 amDate Modified is almost always exactly the same or later than Date Created.
I same almost and most likely because unzipping a file (or other tools) will set the date modified / date created timestamps.

Notably, Microsoft Windows only preserves the Date Modified timestamp when Copying files, but always applies a new Date Created timestamp to the copy. So if you use conventional Copy to create backups of old documents, you will lose all the original Date Created timestamps. This results in clusters of files that have an older Date Modified with a newer Date Created timestamp.

@void: I remember you saying 1.5 would allow for timestamp math. Is it possible to search for all files that have a newer DC timestamp than the DM timestamp? Is it possible to search for all files that have a DC and DM timestamp that are greater-or-less-than 5 minutes or 1 hour apart (delta)?

Post by **void** » Wed Oct 13, 2021 11:57 pm

Date arithmetic is still on my TODO list.

You can already use simple comparison searches like:

dc:>dm:
dm:<dc:

(I recommend using date modified and date created indexing if you do this)

harryray2 · Post by **harryray2** » Thu Oct 14, 2021 10:22 am

Thanks Raccoon, I'm afraid the Windows timestamp system is giving me a pain in the proverbials...

I use date created and date modified (and now, to a degree, recent change) to monitor the locations of any new installations I make.
The proposed changes to the index journal and real time will help a lot but obviously wont overcome Windows (as I see it) timestamp limitations.

@void
Is there a command line for starting Everything with load usn journal into recent changes database, rather than going into options?
I don't want this enabled all the time as it slows down Everything loading, so I would like to create a shortcut for when I need this.

With the comparison search, is it possible to narrow it down to what happened in, for example, the last 15 minutes. Something along the lines of dc:>dm: last15mins.
Also to view files and folders that have the same modified and created dates?

Thanks for all the answers on this somewhat tedious subject :0)

Post by **void** » Thu Oct 14, 2021 10:34 am

Is there a command line for starting Everything with load usn journal into recent changes database, rather than going into options?

Currently, no.
I will consider an option to do this.

Please consider running another instance that loads the USN Journal into memory.
-no-db might also be useful in this case to avoid updating the existing database.

or, please consider setting the ini setting before launching Everything.
For example, set the load_recent_changes ini setting for a few ntfs volumes:
echo ntfs_volume_load_recent_changes=1,1,1,1,1,1,1,1,1,1,1,1 >> c:\Everything\Everything.ini

Post by **void** » Thu Oct 14, 2021 10:38 am

With the comparison search, is it possible to narrow it down to what happened in, for example, the last 15 minutes. Something along the lines of dc:>dm: last15mins.

maybe something like:
dc:>dm: dc:last15mins | dm:last15mins
-or-
dc:>dm: rc:last15mins

Also to view files and folders that have the same modified and created dates?

dc:=dm:

and to find files and folders where the date modified is NOT the same as the date created:
dc:!=dm:

harryray2 · Post by **harryray2** » Thu Oct 14, 2021 11:49 am

Thanks, really useful...

dc:=dm:
dc:!=dm:

Again, for a certain time period.
I tried a few ways but couldn't get it to work

harryray2 · Post by **harryray2** » Thu Oct 14, 2021 6:21 pm

If the -no-db is used, I presume it just loads into memory and then on exit, unloads, without affecting the existing database file?

I'm afraid I don't understand the:
echo ntfs_volume_load_recent_changes=1,1,1,1,1,1,1,1,1,1,1,1 >> c:\Everything\Everything.ini
Could you explain please?

Thanks, a command line option for the load USN journal will save having to start, enable, exit and restart. and vice versa.

harryray2 · Post by **harryray2** » Sun Oct 17, 2021 12:17 am

Do you mean that:
echo ntfs_volume_load_recent_changes=1,1,1,1,1,1,1,1,1,1,1,1 >> c:\Everything\Everything.ini
has to go in my shortcut or is it for a batch file?

I changed this to:
echo ntfs_volume_load_recent_changes=1,1,1,1,1,1,1,1,1,1,1,1 >> D:\harry\Everything-harry.ini
and tried it in various positions on my existing shortcut, which is:
D:\harry\Everything64.exe -filter everything -instance harry

But I can't get it to work.

Possibly I've misunderstood?

Update: I think I'm getting somewhere, I managed to get it going just using the line you gave me in a bat file.

I want to combine it with my existing command:
D:\harry\Everything64.exe -instance harry -s dc:last1mins|dm:last1mins -filter "without firefox and eset"
into a shortcut.

When I use it in a shortcut and enter it, at the start of the line, into the target dialogue box I get a "name in target box is not valid"
and tried it in various positions on my existing shortcut but I can't get it to work.

I've also put in a batch file:
echo ntfs_volume_load_recent_changes=1,1,1,0,1,1,1>> D:\harry\Everything-harry.ini
to disable (uncheck) the "load USN journal", which appears to work.

Is there then a way, on exiting Everything, to disable (uncheck) the "load USN journal" automatically, again in a bat file and a shortcut with the above parameters?

Post by **void** » Wed Oct 20, 2021 10:31 am

It should go in your BAT file.

It's a little bit of a hack, it just appends
ntfs_volume_load_recent_changes=1,1,1,1,1,1,
to your Everything .ini
(Everything will use the last ntfs_volume_load_recent_changes value in your Everything.ini)

Is there then a way, on exiting Everything, to disable (uncheck) the "load USN journal" automatically, again in a bat file and a shortcut with the above parameters?

Maybe have the bat file wait until the Everything.exe process exits and then append the following to your Everything.ini:
ntfs_volume_load_recent_changes=0,0,0,0,0,0,0,

Your BAT file might look something like:
echo ntfs_volume_load_recent_changes=1,1,1,1,1,1,1,1,1,1,1,1 >> D:\harry\Everything-harry.ini
D:\harry\Everything64.exe
echo ntfs_volume_load_recent_changes=0,0,0,0,0,0,0,0,0,0,0,0 >> D:\harry\Everything-harry.ini

harryray2 · Post by **harryray2** » Wed Oct 20, 2021 11:09 am

Thanks, I think I've done more or less the same thing in my batch file.

Is there a way of putting it in a shortcut in the target box, as far as I can see it only seem to work in a batch file, not a shortcut.

You mentioned about using the -no-db. As I understand it this loads the db into memory but doesn't write to disk on exit. Is that correct? I assume that this doesn't speed up the loading of the USN on start?

I'm also curious to why all the one's and zero's in the batch file when the entries in the ini file are different.

Post by **void** » Wed Oct 20, 2021 11:31 am

You mentioned about using the -no-db. As I understand it this loads the db into memory but doesn't write to disk on exit. Is that correct?

-no-db will never load your database from disk.
-no-db creates a fresh database in memory.
-no-db will never save your database to disk.

I assume that this doesn't speed up the loading of the USN on start?

Correct.
Reducing your USN Journal size may help loading performance.
What is the maximum size for each NTFS volume under Tools -> Options -> NTFS?
(I recommend keeping your maximum USN Journal size at least 32768 KB)

I'm also curious to why all the one's and zero's in the batch file when the entries in the ini file are different.

Each value represents an NTFS volume (as described by ntfs_volume_guid)
1 = load recent changes.
0 = do not load recent changes.

So it's a bit of a hack that we don't check how many NTFS volumes you have.
This should work fine provided you don't have more than 6 NTFS volumes.

For example, the following will work with 6 NTFS volumes.
ntfs_volume_load_recent_changes=1,1,1,1,1,1,

If you have less volumes, the extra values are ignored.

harryray2 · Post by **harryray2** » Wed Oct 20, 2021 11:58 am

Thanks for all that..the final question is to why the echo ntfs_volume_load_recent_changes doesn'y appear to work in a shortcut?

Post by **void** » Wed Oct 20, 2021 10:55 pm

echo and launching a program works best from a BAT file.

Please try making a shortcut to your BAT file.