ETP-server resets connection

[Gisle Vanem]

I've integrated ETP-server support in my envtool search program.
I find it works fine most of the time, but trying to get a list of all *.exe files on a remote server,
the ETP-tranfer gets interrupted. The WSAGetLastError() is 10054 which is WSAECONNRESET

I use a command like:

Code: Select all

envtool.exe --evry --host 10.0.0.37 *.exe
...
Expected 2638 results, but received only 334.

I'd be very happy if someone could confirm this or if it's a bug in my envtool program.
I've put a compiled envtool.exe in my Github project; enter https://github.com/gvanem/EnvTool and click on
"Clone or Download" | "Download ZIP".

Edit: Forgot to say. This is with the latest 1.4.1.877 (x86) version of EveryThing on the server side.
The same version, except x64 is on the client side.

[void]

With the following command:
envtool.exe --evry --host localhost *.exe

Everything receives the following search request:
^.*\.exe$

I'm getting the correct results:
540 matches found for "*.exe".

Are you prematurely closing the socket on the client side?

Does manually entering the EVERYTHING commands in Filezilla work:
EVERYTHING REGEX 1
EVERYTHING CASE 0
EVERYTHING SEARCH ^.*\.exe$
EVERYTHING PATH_COLUMN 1
EVERYTHING SIZE_COLUMN 1
EVERYTHING DATE_MODIFIED_COLUMN 1
EVERYTHING QUERY

[Gisle Vanem]

Thanks for answering.

You only have 540 *.exe .

I've tried with both a higher recv-timeout (10 sec) and 16kByte recv-buffer:

Code: Select all

  DWORD timeout = 100*1000;
  int rx_size = 16*1024;
  setsockopt (sock, SOL_SOCKET, SO_RCVTIMEO, (const char*)&timeout, sizeof(timeout));
  setsockopt (sock, SOL_SOCKET, SO_RCVBUF, (const char*)&rx_size, sizeof(rx_size));

It didn't help. The problem seems to be a timing issue. If I reduce the number of results (s*.exe)
or redirect to file of NUL, all is well.

I suspect it is my simple use of recv(sock, buf, 1, 0). This should be rewritten into something more clever.

[Gisle Vanem]

I've also tried using the ftp-client that comes with Windows. It also shows the same problem.
I.e. ETP-server does not send a 200 End at the end of all results. Verified with this
test-etp.bat file

Code: Select all

@echo off
if %1. == . (
  echo Usage: %0: file-spec
  exit /b
)
echo QUOTE USER foo                    > etp-commands
echo QUOTE PASS bar                   >> etp-commands
echo QUOTE REGEX                   0  >> etp-commands
echo QUOTE CASE                    0  >> etp-commands
echo QUOTE EVERYTHING SEARCH       %1 >> etp-commands
echo QUOTE EVERYTHING PATH_COLUMN  1  >> etp-commands
echo QUOTE EVERYTHING QUERY           >> etp-commands
echo BYE                              >> etp-commands
ftp -n -s:etp-commands 10.0.0.37

(replace 10.0.0.37 with localhost or whatever). With the above, I get for test-etp.bat *.exe, a
Connection closed by remote host. and not any 200 End.

[void]

I tested on a Windows XP VM, so I didn't have many exe files, * also worked fine (about 20,000 files).

Windows 7 x64 ftp client is working ok for me, 18159 exe files on my dev PC, with the "200 End." message received (tested 5 times).

What version of Windows are you using?
Do you have the Windows firewall enabled?


Please try running Everything in verbose debug mode, you will be able to see all the ETP events:

• In Everything, type in the following search and press ENTER:
  /debug
• Type in the following search and press ENTER:
  /verbose

Anything shown in the debug console when your client is disconnected?

To close the debug console:

• In Everything, type in the following search and press ENTER:
  /debug

[Gisle Vanem]

What version of Windows are you using?

Win-10 on both client and server sides.

Do you have the Windows firewall enabled?

Yes. But I'm testing on my LAN for the moment. Do you have a ETP-test server I can test slower transfers with?
That be nice!

Thanks for the tip on /debug and /verbose.
I do have dbgview v4.81 (the latest) from SysInternals, but it's a bit outdated. Do you know of a better one?

[void]

I managed to reproduce this issue consistently with Windows 7 or 10 using the FTP command.
The issue typically occurs at the same send position. Usually after about 10-15 seconds.

Both the client and server are receiving 10054.

I've tried setting large send and recv timeouts with no luck.
I've tried setting keep alive with no luck.
I've tried disabling tcp_nodelay with no luck.

Disabling Windows Firewall appears to fix the issue.
Looks like Windows Firewall does not like lots of data going over port 21.
Changing the Everything ETP port to 21212 while keeping Windows Firewall enabled appears to fix the issue.

Please try changing your ETP port to something else other than 21.

I do have dbgview v4.81 (the latest) from SysInternals, but it's a bit outdated. Do you know of a better one?

Everything has it's own debug console, you won't need a third party debugger.

[Gisle Vanem]

Ok.

I changed to port 2121 and that works very stable; no 10054/WSAECONNRESET codes.
I get good transfer speed too; an
envtool --evry --host 10.0.0.37 *

gives 26 MByte of data in 9 sec (154 000 files).
Which works out as approx. 22 MBit/s over a 54 Mbit/s WiFi. Not bad!