automatically backup file-list or database after refreshing folders?

[randfee]

Hi all,
everything keeps crashing me quite often when indexing/refreshing network folders (it's running on a win 64 server 2016) - or the server forces a restart due to updates. The folders contain some 15-20 million files and it just takes a while. However. Once it gets killed, one loses all of the newly indexed files unless everything is shut down properly in which case it auto-saves to the Everything.db.

I was wondering if there was a way to create a server-like instance of everything which does nothing but continuously refresh folders and provide an ETP server. The latter is no problem obviously but I'd like it to refresh the given folders and wenever it is done with one it automatically saves to the database or saves the updated file-list (EFU) to a certain folder.

As for the 'clients' on the network, it would be prefereble to use EFU files since they could be accessed even if the 'server' is down, however I don't know what happens if they're being overwritten while some everything-instances access the files, I guess they're just loaded on startup?

regards
randfee

[void]

An "Everything Server" is on my TODO list.

This "Everything server" would manage the main index.
Everything clients will be able to build an maintain their own index from the server index.

For now, I recommend folder indexing.

If you find Everything crashes often, please consider creating a scheduled task to force Everything to flush its database to disk (so it doesn't have to perform a full rebuild the next time you run it):

Create a scheduled task to run:
Everything.exe -update

I have on my TODO list to flush the database to disk periodically.

[randfee]

thanks for the quick reply. Thoughts about it

• we are using folder indexing right now
• why not use EFU files? They are larger, yet they work for the clients even if the ETP server thread crashes or busy (for whatever reason), whereas the flush/update to db only ensures that the database is saved. So I would prepfer this periodical flush/update functionality to also enable updates/flushing EFU files periodially.
• if you create a setting to periodically flush/update the database, please let the user select the period/time/frequency for that
• if clients will be able to build/update their own databases from the server one, how far are you gonna go? Will there be some functionality to check user access rights? That would be neat but a lot more work to implement for you . We don't need it but if it were available we would be happy to pay for it as well. We'd much rather see a simple working solution earlier though than a super-fancy one far into the future

[NotNull]

You can let a separate instance of Everything do the indexing and write the results to EFU files.
The EFU files can now be used as indexes for your ETP Server.
Whenever the EFU files are updated, the ETP Server will re-read them and update the index accordingly.

In a bit more detail:

Configure Indexing

• Copy Everything.exe to an empty folder
• Run (example1)

  Code: Select all

  Everything.exe -instance INDEX -create-file-list "X:\output folder\Program Files.efu"   "c:\Program Files\"

• Run (example2)

  Code: Select all

  Everything.exe -instance INDEX -create-file-list "X:\output folder\Windows.efu"   "c:\Windows\"

  (Note that this will NOT create an Everything.ini and Everything.db; they are not needed.

Configure the ETP Server:

• Remove all current indexes (NTFS-, ReFS-, Folder indexing
• Go to Options > Indexes > File Lists
• Add "X:\output folder\Program Files.efu"
• Enable Monitor changes for this File List
• Add "X:\output folder\Windows.efu"
• Enable Monitor changes or this File List
• Click the OK Button
• Done

When this shows thye expected results on the ETP Server, remove these two filelists and add the actual fileshares you want to have indexed, using the method described above.
Create schduled tasks (or whatever method you prefer) to update the EFU files.
The "Monitor changes" setting will cause the ETP server to check for updated EFU files and update it's database accordingly


Some notes:

• Are you running 64-bit Everything? Indexing 20 million files is about the max a 32-bit Everything can "hold".
• Don't know if it is a good idea to let the all the clients load a 2GB database in memory ...
• This will index Filename (including path), size, date modified, date created and file attributes.
• This is all workaround. Wouldn't you prefer to have your "normal" indexing working?