Generating Thumbnails

General discussion related to "Everything".
Post Reply
null
Posts: 26
Joined: Mon Feb 27, 2023 2:10 am

Generating Thumbnails

Post by null »

Is it possible for Everything to decrease memory usage while creating thumbnails? Presently, I am updating PDF thumbnails for Everything, which consists of over 50GB of PDF files, consuming approximately 96% of my 16GB system memory.

Also, it would be beneficial if Everything could produce or refresh thumbnails even when they are not visible in the viewport. It would be convenient to update previews when displaying all PDF files using the *.pdf filter. As it stands, I need to choose the medium thumbnail view and scroll down for the previews to be generated, requiring me to remain at the computer and essentially do nothing.
void
Developer
Posts: 16751
Joined: Fri Oct 16, 2009 11:31 pm

Re: Generating Thumbnails

Post by void »

I have on my TODO list to limit the thumbnail allocation (most likely to a percent of maximum RAM or a fixed size)
For now, Everything will allocate thumbnails and keep them in memory until you close the search window.

Each thumbnail will consume 256 KB of RAM.

Everything will store thumbnails in memory.
Never on disk.
When you close the search window the thumbnails are lost.

Everything requests the thumbnails from the OS.
The OS does cache thumbnails on disk.



Decreasing the thumbnail load size will help lower memory usage:
  • Copy and paste the following into your Everything search box:
    /thumbnail_load_size=128
  • Press ENTER in your Everything search box.
  • If successful, thumbnail_load_size=128 is shown in the status bar for a few seconds.
    where 128 is the load size in pixels.
thumbnail_load_size



To precache thumbnails out of view, you'll need Everything 1.5 with request extra fileinfo set:
  • Copy and paste the following into your Everything 1.5 search box:
    /request_extra_fileinfo_end=256
  • Press ENTER in your Everything search box.
  • If successful, request_extra_fileinfo_end=256 is shown in the status bar for a few seconds.
    where 256 is the number of thumbnails to precache after the last visible item in the result list.
    ---
  • Copy and paste the following into your Everything 1.5 search box:
    /request_extra_fileinfo_start=256
  • Press ENTER in your Everything search box.
  • If successful, request_extra_fileinfo_start=256 is shown in the status bar for a few seconds.
    where 256 is the number of thumbnails to precache before the last visible item in the result list.


request_extra_fileinfo_end
request_extra_fileinfo_start
null
Posts: 26
Joined: Mon Feb 27, 2023 2:10 am

Re: Generating Thumbnails

Post by null »

Thank you. I will give those a go.
void wrote: Wed Apr 26, 2023 1:16 am Everything will store thumbnails in memory.
Never on disk.
When you close the search window the thumbnails are lost.

I firmly believe that saving thumbnails to disk in a database is essential. I often work with various file formats, including Illustrator, Photoshop, CAD, and GIS files, and the Everything search tool is incredibly efficient in finding these files. In order to avoid reloading delays, caching the thumbnails to disk is crucial.

While I do use image viewers, they don't have search features; however, they store thumbnails in a database and enable optimization. Given the number of files I work with, it's acceptable for the databases of these image viewers to be over 1GB in size. As such, having more than 1GB doesn't concern me, since a single GIS file or files from CAD, Illustrator, or Photoshop can be well over 1GB each, even 10GB. Even though some users may worry about drive space, there should be options to compress, optimize, or alter the quality of cached thumbnails.
NotNull
Posts: 5461
Joined: Wed May 24, 2017 9:22 pm

Re: Generating Thumbnails

Post by NotNull »

Messages deleted. This way of discussing is not acceptable on these forums.

Don't discuss each other. Discuss the subject!


In case there was something interesting/important to report, please try again.
But this time in a much friendlier way ...
null
Posts: 26
Joined: Mon Feb 27, 2023 2:10 am

Re: Generating Thumbnails

Post by null »

NotNull wrote: Wed Apr 26, 2023 2:01 pm Don't discuss each other. Discuss the subject!

Messages deleted. This way of discussing is not acceptable on these forums.

In case there was something interesting/important to report, please try again.
But this time in a much friendlier way ...
Thank You.

I persist in suggesting that Everything could incorporate an elective thumbnail caching feature within its database for those who deem it advantageous. This could be a consideration for future development, as I believe it would prove beneficial. Although file managers such as Explorer exist, they do not provide the same level of speed as Everything. Moreover, the current feature allowing users to move the database to a UNC path is a valuable asset.
void
Developer
Posts: 16751
Joined: Fri Oct 16, 2009 11:31 pm

Re: Generating Thumbnails

Post by void »

I will consider adding thumbnail disk cache to Everything.
Thank you for the suggestion.

For now, is there a program out there that would make a png thumbnail in the same location as your PDF file?
Maybe Everything could be designed to use the png file as the thumbnail instead..
null
Posts: 26
Joined: Mon Feb 27, 2023 2:10 am

Re: Generating Thumbnails

Post by null »

I typically use Xnview which displays PDF, PNG etc thumbnails quite well and saves the information to its database. I have to create favorite locations for my files since it's not a search tool.

The issue I was having with my laptop was limited resources, but now that I have my main system up and running, I have plenty of 64 cores, disc space, and 256GB of memory, so I have no concerns with this system. I just wish Everything (also 1.5a, which I like a lot) ran faster and used more cores. However, there are other users with the same issues and limited resources as I was having while running on my laptop which I still use.

Also, is there a specific reason why Everything does not generate a thumbnail unless the thumbnail is in view? I wonder if there is a way to continue generating content even while the content is not in view?
therube
Posts: 4977
Joined: Thu Sep 03, 2009 6:48 pm

Re: Generating Thumbnails

Post by therube »

(I wasn't familiar...

so Preview of PDF is lazy load only
additionally, the various pages of the pdf itself are also lazy load

so if you have a 500 MB pdf (& only 7 pages at that ;-),
it might be a bit slow

likewise, a pdf with a lot of pages, lazy loads each page as it comes into view,
again potentially slowly at that)
null
Posts: 26
Joined: Mon Feb 27, 2023 2:10 am

Re: Generating Thumbnails

Post by null »

I was speaking in general with my last comment. All files load 1 at a time, but don't start loading until after the thumbnail is in view. I set the thumbnails to medium at a 1440 screen, so could take 10–60 minutes until everything is cached.

But with the PDF issue, why doesn't it just take the first two pages? Why does it need to read the entire document? I have many PDF files well over 50 pages, most are over 200 pages, some compressed, some not.
void
Developer
Posts: 16751
Joined: Fri Oct 16, 2009 11:31 pm

Re: Generating Thumbnails

Post by void »

Creating a thumbnail/preview of a PDF file will take a very long time.



Everything uses the system to generate thumbnails for PDF files.
Third party thumbnail handlers for PDF files can be installed.



To precache thumbnails out of view, you'll need Everything 1.5 with request extra fileinfo set:
  • In Everything 1.5, from the Tools menu, click Options.
  • Click the Advanced tab on the left.
  • To the right of Show settings containing, search for:
    request
  • Select request_extra_fileinfo_end.
  • Set the value to: 256
    (where 256 is the number of extra items to precache out of view)
  • Click OK.
void
Developer
Posts: 16751
Joined: Fri Oct 16, 2009 11:31 pm

Re: Generating Thumbnails

Post by void »

I was speaking in general with my last comment. All files load 1 at a time, but don't start loading until after the thumbnail is in view. I set the thumbnails to medium at a 1440 screen, so could take 10–60 minutes until everything is cached.
I will consider loading multiple thumbnails at a time.
For now, Everything loads thumbnails in a single thread.


But with the PDF issue, why doesn't it just take the first two pages? Why does it need to read the entire document? I have many PDF files well over 50 pages, most are over 200 pages, some compressed, some not.
I'm not entirely sure how your PDF thumbnail handler works.
Please try other third party PDF thumbnail handler to see if the performance improves.
null
Posts: 26
Joined: Mon Feb 27, 2023 2:10 am

Re: Generating Thumbnails

Post by null »

Nice.

I don't know about a good thumbnail generator. I've been searching for one for a long time. I've been using SageThumbs, then MysticThumbnails, and PSD Codec. I haven't tried FastPictureViewer Codec Pack yet. Mystic uses too much memory and hogs the cpu. SageThumbs works off and on. PSD Codec I just reinstalled. The reason why I avoid these is because they seldom work and XnView (renders thumbnails for all files) and Faststone Image Viewer (only an image viewer) never let me down. I upgraded to Windows 11 and it seems the thumbnails work better than Windows 10, but I went back to Windows 10 because I cannot move the taskbar to the top without breaking things.

The image below is medium_thumbnail_size=16, thumbnail_medium_text_lines=0, listview_text_padding_bottom=0, request_extra_fileinfo_end=500. Is there a way to eliminate text temporarily while generating thumbnails in Everything? Perhaps add a context (view) menu item for 1x1 pixel representaing each thumbnail (similar to a disk defrag white[working]/blue[complete]) for thumbnail generation for the desired file extension? Then after thumbnail generation the user could move back to medium, large, extra large thumbs. I often clear my thumbnail caches, a lot. The above method greatly reduced the image generation for the psd thumbails.
20230914-September-W38_053546660_Everything64.png
20230914-September-W38_053546660_Everything64.png (800.9 KiB) Viewed 7759 times

Below is a 30 second gif crunching away at generating PDF thumbnails.
20230914-September-W38_060034632_Everything64.gif
20230914-September-W38_060034632_Everything64.gif (479.29 KiB) Viewed 7749 times
therube
Posts: 4977
Joined: Thu Sep 03, 2009 6:48 pm

Re: Generating Thumbnails

Post by therube »

crunching away
As in animated? Cause if so, I don't see it animating?
null
Posts: 26
Joined: Mon Feb 27, 2023 2:10 am

Re: Generating Thumbnails

Post by null »

It's animating.
Post Reply