SelectChannelConnector much to slow in windows

test client

test server

jira won't let me attach the 14MB jarfile. But any other 14MB file called big.jar should do as well.

Yug,

as I don't have two windows machines... this is going to take me some time to look at.
Actually... I'll see if Nik can have a crack at this.

This is probably some sort of delay in noticing that the socket is writable?

Yug, can you capture some traffic with ethereal as this may tell us where the
delay is.

I've replicated this already on 2 laptops running on winXP and ran ethereal to analyze what happens between the client and server. However, I can't pinpoint where exactly the selectchannel connector is taking long. I'm attaching the ethereal output files for both selectChannelconnector and socketconnector for anyone with better interpretting skills.

couldn't attach the file because it exceeded the 10 mb limit so i uploaded it here:

http://www.yousendit.com/transfer.php?action=download&ufid=1C706A7D654FC034

i also tried to run the client on a linux machine. here is the ethereal file:
http://rapidshare.de/files/28014771/slowserver-selectchannel18080-linux_client.zip.html

It was slow as well. took 138 seconds but that may be partly due to the slow network here. nevertheless 138 seconds is a big number

Nik,

Can you point to where I can get the capture file of the SocketConnector.

What were the total times for the two captures and what is the file size you are using?

Looking at the trace from the server on windows vs one I did on my linux machine, there are
three things I notice.

1) There appears to be a smaller window on the windows box. IE acks are sent for the
data packet pretty soon after they are sent. With the linux server, data is sent WAY in
advance of acks being received.

2) The windows server never fills up it's sliding window. The linux server sends data
until the window is full and then waits a bit for acks. 3 to 10 ms is the typical wait.

3) Even without a full window, the windows server stops sending data and then there
is a long pause (200ms) until the client sends an ack. It looks like the client is waiting
for either some return data or a full window before sending the ack.

So there is something about how this data is being sent that is making the windows
TCP/IP stack work really really badly.

Nik - we need to do some experiments to see how we can affect this by making variations
in the aspects discussed below.

Because of the size of this file, it is probably being sent by the DefaultServlet
using the Resource.writeTo() method, which eventually uses the
IO.copy method. This has a buffer size of only 8192 for transfers. We should
try a much larger buffer size ... perhaps 32k

We can adjust the configuration of the DefaultServlet so that the file cache
will accept this file size and use a memoryMapped buffer. This should be
the fastest way to send data. Ifthis works, then we will have to review the
concept of file cace with regards to memory mapped buffers... or perhaps
have temporary mem mapped buffers for large files etc.
Look at the code in DefaultServlet.getContent(String,Resource)

Failing that - turn off useMemoryMapped buffers - so the whole massive
file gets loaded into memory and then served from cache. This is not
a good way to proceed, but will tell us if the problem is how we read the
data or how we write the data.

We should also see what the OS has set Socket.getSendBufferSize() at
for the two types of connector. We should then try setting it to bigger
values to see if that makes a difference. We should research how
big this should be, and potentially we should either make it directly
configurable or set it to the same size as the setResponseBufferSize
field

Nik - can you do the experiments - as I still don't have a good windows
machine I can use for this.

thanks

I used a 15.77 MB big.jar.

I adjusted the configuration of the DefaultServlet so that the file cache will accept this size as you suggested. Set it to 16MB. It worked! File transfer was much faster.

Here's the ethereal file: http://rapidshare.de/files/28136625/selectchannelconnector-maxcachedfilesize-16MB.zip.html

meanwhile, I'll continue with the other experiments.

Here's the ethereal file for the socketconnector:
http://www.yousendit.com/transfer.php?action=download&ufid=D5F2E4BA413BEBB5

I just tried setting Socket.setSendBufferSize() equal to responsebuffersize (32*1034) and this is what I got (still slow at 100984 milliseconds at 156 kb/s):
http://rapidshare.de/files/28232735/selectchannelconnector-setsendbuffersize-32k.zip.html

I tried setting setSendBufferSize to 64k and file transfer was much faster. about 1344 milliseconds at 11738 kb/s-is this an acceptable speed level? the selectsocketconnector had exactly the same speed readings as socketconnector.

here's the ethereal file for that:
http://rapidshare.de/files/28232735/selectchannelconnector-setsendbuffersize-32k.zip.html

I believe the static content handle refactor has fixed this problem (not tested
first hand, but has been reported as fixed).

Yug - Can you please test 6.0.0rc2 and reopen this issue if it is not the case.

With the latest svn, the windows-windows transferrates are ok.

SlowClient output windows server - windows client
SelectChannelConnector
http://crushinator:18080/big.jar
bytes read: 14364739
1609 ms
speed: 8927 kB/s
-----------------------------
SocketConnector
http://crushinator:18081/big.jar
bytes read: 14364739
1250 ms
speed: 11491 kB/s

For some odd reason, the SelectChannelConnector is still slow when the client runs on freebsd:

server windows - client freebsd
SelectChannelConnector
bytes read: 14364739
11177 ms
speed: 1285 kB/s
-----------------------------
SocketConnector
bytes read: 14364739
1277 ms
speed: 11248 kB/s

.. but isn't it nevertheless strange that it still is noticable faster when using the SocketConnector?! And in particularly, obviously, on the second scenario windows/freebsd? (what about windows/linux?)

Btw, this would probably be a 100 Mbps network, so that ~11.5 MBps basically is rather close to the theoretical limit (100/8=12.5, plus TCP/IP overhead)?
It would be very interesting if one of you could repeat the tests using a setup where you couple those two machines together with a 1Gbps line, to see what it'd really max out at.
Note that very many boxes these days have a 1Gbps network port, but still most office/home switches are 100Mbps: You could actually just "hotwire" the boxen together w/o using a hub/switch (use static ips). One should load the entire file into memory then, or else the disk will definately limit - and probably use a somewhat larger file too, or else it will go too fast to give reliable measurements.

In a normal (not extremely-high-amount-of-users) environment, it seems like it would be smarter to still stick with the good'ol SocketConnector, rather than with the new'n'flash, obviously still not quite broken in SelectChannel stuff?