[BUG][v1.5.1358a] Use of regex in Index->Exclude inconsistent

Found a bug in "Everything"? report it here
Post Reply
DerekZiemba
Posts: 42
Joined: Thu Sep 27, 2018 4:46 pm

[BUG][v1.5.1358a] Use of regex in Index->Exclude inconsistent

Post by DerekZiemba »

I think this is something that broke recently as I didn't notice until now. The regex's in the exclude lists don't work if the regex is quoted. But if they're unquoted then they don't work in the everything search bar.
The screenshot was taken after I wrapping them in quotes & before I figured out quotes are what mess it up.
Initially 2 of the 3 highlighted in "Edit list" were unquoted, but oddly they somehow missed a handful of files. 4 & 85 files respectively. I also discovered if there's a space at the end of the expression, even after $, the expression breaks.
Untitled.png
Untitled.png (690.46 KiB) Viewed 6893 times
The expressions are (now):
  • regex:\\(?=(?:[0-9\{\}\-\.]*?[A-Fa-f]){3,})(?!.+?\.(?i:mp4|msi|png|jpg|jpeg|log|exe|dll|lnk|evtx|db|wdseml)$)(?:[[:xdigit:]\{\}\.\_\-]{16,})(?:|\\.*)$
    ignore if large string of hexadecimal with at least 3 separate occurrences of A-F to avoid hitting dates (long string of numbers), also ignore if it's a useful filetype.
  • regex:[~_+\.\-]?([\{\[\(])?[[:xdigit:]]{8}([_.-]?)(?:[[:xdigit:]]{4}\2){3}[[:xdigit:]]{12,}(?(1)[\}\]\)]|)(?!.*(?:\\|\.(?i:mp4|msi|png|jpg|exe|wdseml)$)).*$
    like above, but targets GUIDs specifically.
  • regex:\\AppData\\.+\\User\s?Data\\.+\.(bin|gif|log|(?:m(?:ap|d5|etadata))|(?:(pn|sv)g)|tmp|woff2?)$
    filter out all the junk chrome & electron apps leave around.
  • regex:\\AppData\\.+\\User\s?Data\\.+\\(CURRENT|L((O(G|CK))|ICENSE)|((Tabs|MANIFEST|SESSION)[_-]\d+)|[A-Z]{2}|[0-9]{8,})$
  • regex:\\(?:m(?:sys|ingw)|Python)[0-9_-]{0,5}\\.+\\(i(?:cons?|mages)|demos?|msgs|ttk|hicolor)\\.+$
    filter out most the un-useful stuff that comes with msys, mingw, & python
void
Developer
Posts: 16753
Joined: Fri Oct 16, 2009 11:31 pm

Re: [BUG][v1.5.1358a] Use of regex in Index->Exclude inconsistent

Post by void »

regex: in the normal search box and regex in Tools -> Options -> Exclude are handled differently.

In the normal search box:

Everything Search Syntax takes precedence.

Space = AND
Use double quotes to escape spaces.

Please note: regex: does eat |
| does not need to be escaped.

Use &quot: to escape a literal double quote (")



in Tools -> Options -> Exclude

For folder filters:
Spaces and quotes are treated literally.

For semicolon delimited lists:
Spaces are treated literally.
Double quotes (") can be used to escape a ;
When using double quotes, a double-double-quote ("") can be used to escape a single literal double quote ("). (Same as CSV)



I'm not seeing any spaces in your regex patterns.

Maybe there's some strange macro expansion going on...
Have you defined an i macro? -does searching for i: match anything unusual?
Have you defined an xdigit macro?

Using double quotes will prevent any macro expansion.
DerekZiemba
Posts: 42
Joined: Thu Sep 27, 2018 4:46 pm

Re: [BUG][v1.5.1358a] Use of regex in Index->Exclude inconsistent

Post by DerekZiemba »

Yeah, originally there was spaces at the end and some of the expressions used quotes. Only today figured out that doesn't work. The reason I bring it up is I haven't changed them for a long time and haven't noticed them not working until recently. Thought maybe this was a regression or something.

Are you saying that macros work in the exclude list?? I thought regex was the only available function. I would love to use `extension:` & the like.

The database finally just finished rebuilding & the expressions no longer missed anything. Everything is properly excluded.
void
Developer
Posts: 16753
Joined: Fri Oct 16, 2009 11:31 pm

Re: [BUG][v1.5.1358a] Use of regex in Index->Exclude inconsistent

Post by void »

Everything was incorrectly trimming spaces at some stage.
The spaces and quotes should be treated literally.
Does the issue persist if you remove the trailing spaces and double quotes?



Macros do not work in Tools -> Options -> Exclude.

ext: will work in Tools -> Options -> Exclude, property filters and filename filters in the next alpha update.
(it will be quirky though as it eats the rest of the filter so make sure it is used last)
DerekZiemba
Posts: 42
Joined: Thu Sep 27, 2018 4:46 pm

Re: [BUG][v1.5.1358a] Use of regex in Index->Exclude inconsistent

Post by DerekZiemba »

Yeah I was able to fix the issue before posting. The database finally just finished rebuilding & the expressions no longer missed anything. Everything is properly excluded. IDK why some files would have ever been able to sneak by though. That doesn't make a lot of sense.

The earliest backup I can find with at least one of the invalid expressions is from March 2022. It may have just been broken for a real long time or maybe it never worked ¯\_( ツ )_/¯. IDK why I only noticed today.
ext: will work in Tools -> Options -> Exclude, property filters and filename filters in the next alpha update.
\(^O^)/ AWESOME!
void
Developer
Posts: 16753
Joined: Fri Oct 16, 2009 11:31 pm

Re: [BUG][v1.5.1358a] Use of regex in Index->Exclude inconsistent

Post by void »

Everything 1.5.0.1359a adds ext: support to filename filters, exclude filters and property filters.

ext: will eat all remaining filter text.

This will be a hidden feature for now.
The UI doesn't support ext: yet. (opening the list editor with ... will split your ext: search into individual items)
DerekZiemba
Posts: 42
Joined: Thu Sep 27, 2018 4:46 pm

Re: [BUG][v1.5.1358a] Use of regex in Index->Exclude inconsistent

Post by DerekZiemba »

This is great news! I didn't realize a new version was out and actually came here to say some regular expressions are seemingly leaking items through again.
----------------------------
> This will be a hidden feature for now.
The amount of hidden gems in this program blows my mind. Nearly everyday I find a new feature I never knew about. Earlier this week while assigning a keybind to "Close WIndow" I stumbled across "Close Tab".... said aloud, "wait wtf there's tabs?"
Something I've always wished for was there all along so I started looking into anything else I might have missed. I never considered this an explorer replacement; but now with tabs, the file tree, and assigning middle click to navigate into folders without having to jump to explorer, it totally can be & for me, is quickly doing so.
----------------------------

So the reason I came here. The same reason as before, but it took me by surprise. This time i had to remove the quotes from expressions in Result Omissions to:
* regex:\\[^\.]+$
* regex:((?:[\w\+\_\-]{2})+(?:[\w\+\_\-]{2}=|[\w\+\_\-]{3})+=)(?-i:\.[a-z][a-z0-9]+)?$
The first targets files without extensions, something your new version will make easier!
The 2nd looks for base64 encoded file names. With base64 lacking a period char there's a lot of overlap between them.

Anway I wrote up a whole thing here and while explaining it I figured out the issue was on my end all along. I was stumped because it seemed like they were partially working / inconsistently working. But it was a combination of settings that made it behave like that.
Post Reply