Re: squid and tcp splicing ?

On Wednesday 10 September 2003 20.32, J.Smith wrote:

> Apart from CONNECT, the paper mentions the GET method as well. Not
> sure what you exactly mean by "allowing the session to be logged
> correctly".

Squid needs to be told exacly how many bytes was transferred, and when
the transfer to the client finished as this information needs to be
logged for accounting purposes.

But there is other complications as well even if we stay HTTP/1.0.

* delay pools. The delay pool feature of Squid is a feature where the
traffic is throttled to a configurable level. If the traffic is not
seen or noticed by the proxy then this throttling obviously no longer
works.

* persistent connections. Even if the HTTP reply (not counting
CONNECT) is found to be non-cachable Squid can not simply hand the
socket for unconditional forwarding to the kernel. Squid needs to
regain control of the sockets once the transfer has finished allowing
Squid to read the next request from the client or send the next
request to the server. In HTTP/1.0 this is relatively simple as the
message delimiting is trivial (either known length from the headers
or close of socket), but for HTTP/1.1 it gets messier due to the use
of chunked transfer encoding. Also in HTTP/1.1 there are many
situations where the proxy will need to chunk/dechunkt the transfer
encoding.

It is my opinion that there is many better ways of optimizing proxy
class I/O without resorting to kernel level "splicing". For example
it would be very interesting to see a I/O mechanism not involving
copying of data on each read/write combined with a sane event
notification. This would accomplish the same goal with fully
preserved semantics and only marginally increased CPU load compared
to kernel level "splicing". There has been a couple of attempts in
doing this based on AIO but I have not seen any promising results
yet.. (AIO has reasonable I/O primitives but lacks in notification,
and most kernels are not very suitable for AIO).

> > and works with non-blocking sockets.
>
> The splice() implementation in AIX 5.x is supposed to be a
> kernel-based socket-level tcp splice. Not sure about the
> "non-blocking" part, though.

If the splice call is blocking then Squid can not use it as it would
stop all other traffic..

Regards
Henrik
Received on Wed Sep 10 2003 - 14:24:45 MDT